April 2018

Digitization and data analytics will impact upstream operations favorably

On June 13, 2014, the cost of a barrel of West Texas Intermediate stood at $107.49. That year, budgeted, worldwide capital expenditures in the oil and gas industry would approach a record $520 billion. The future looked bright.
Jay Stieneker / North Highland

A new era of increased efficiencies is on the horizon for operators.

On June 13, 2014, the cost of a barrel of West Texas Intermediate stood at $107.49. That year, budgeted, worldwide capital expenditures in the oil and gas industry would approach a record $520 billion. The future looked bright.

However, within eight months, trends would converge to plunge the price of oil, down to $44/bbl. This decline would continue into February 2016, with the price of oil reaching a low of $29.04/bbl. 

Three major factors were behind this precipitous drop. Economic slowdown in China and other developing economies slowed the growth of oil demand around the world. This coincided with increased supply emanating from the U.S., driven by a revolution in shale oil extraction technologies. As many economists and industry experts were discussing the approach of “Peak Oil”—the date of worldwide maximum oil production, due to the depletion of world oil supplies—the U.S. unexpectedly increased its production from less than 5 MMbpd in 2007 to 9.4 MMbpd in 2015. 

Historically, as a leading global swing producer, Saudi Arabia would intervene in these types of situations by decreasing production and keeping the price of oil stable. This time would be different. Saudi Arabia decided to maintain its market share and not to decrease its output. The price of oil was allowed to fall, which ushered in a new era for the upstream sector. 

Thus, the industry must meet significant challenges to realize new efficiencies and increase productivity. This will allow it to be profitable at lower oil prices without compromising on safety and environmental concerns. Not only are market forces to be reckoned with (e.g., fall in OECD demand, increased supply, loss of demand due to future events, such as the possible rise of electrification in powering transportation), but there is now external competition from alternative energies, and the possibility of increased regulation due to climate change emission laws.

A potential answer to these threats is the introduction of digitization and data analytics within the industry, Fig. 1. Not only do these technologies promote new ways of working, support cost efficiencies, and help optimize business processes, they also bring the added value of safer operations and reduced risk within the value chain.


Fig. 1. Implementation of digital technologies can automate engineering decisions, as illustrated by the chart.


Digitization is typically thought of as the automation of processes or the introduction of technology to increase productivity. The reality is that this is just a part of the greater picture. Digitization brings transformation into the organization, itself, Fig. 2. The business evolves into a culture of continuous improvement, raising goals beyond just efficient execution and delivery excellence. Business processes shift from being solely a human endeavor and organizations leverage data and digital tools better, allowing workforces to focus on the most critical elements of any given work stream. 


Fig. 2. Digital transformations holistically leverage the capabilities and possibilities of various digital technology solutions in a strategic and systematic way, to have a profound impact on the business.


Data analytics is an outgrowth of digitization, and refers to techniques to extract and analyze information to enhance productivity. It is used to reveal patterns and correlations between data that lead to better-informed and enhanced decision-making. In many cases, these data already exist within an organization; they just need to be identified and correlated to reveal previously unknown connections. Data analytics can be used to not only gain insight into current business relationships, but also to make forecasting easier, through predictive modeling.

Digitization lays the groundwork for analytics. If digitization is implemented with analytics in mind, data are gathered from various sources (e.g., meters, databases, the cloud) that power the analytics machine. Together, data and analytics have the potential to unlock value by introducing cost savings and significant improvements in productivity and revenue.

This article will elaborate on technologies that lay the foundation for digitization and analytics; discuss the transformation of the organization to effectively take advantage of these advances; and provide examples in oil and gas operations where digitization and analytics can be deployed.


Several new technologies have been introduced to become the backbone of digitization. The following items are some examples. 

Cloud computing is the delivery over the Internet of computing services, including servers, storage, databases, networking and software, typically via service providers. This lowers cost by eliminating the need for organizations to procure and maintain their own data centers. It also has the advantage of ensuring that business resources, such as storage or bandwidth, are scaled properly and can be added or subtracted, based on need.

Big data is the concept of managing and analyzing vast quantities of data that cannot be stored on traditional servers and databases. Formerly, the bulk of business information was stored in documents and structured databases. This has evolved to include a myriad of unstructured data sources, including audio and video, photographs, simulations, 3D models and real-time operating data, among others. Big data brings together large, scalable and reliable distributed computing platforms, which can parse and analyze extremely large data sets in a relatively short period of time. 

Mobility allows users to view and interact with data anytime and anywhere. Previously, data could only be analyzed at a fixed location, typically on a desktop computer. With the advent of smartphones and tablets, data are available anywhere, as long as there is an Internet connection. This access allows for the entry or reference of data at worksites, drilling platforms and on the move, instead of the old, limited way of relying on notes, which may be out-of-date.

Internet of Things (IoT) is the concept of connected objects and devices with embedded sensors that can generate and transmit data through a network to a central controller or other objects. They have the capability to provide pertinent information to data consumers, whether human or machine, to give insights into the condition of any system with connected monitoring sensors. 

Robotics is the use of machines specially designed to automate and perform human tasks. Robots support the performance of complex tasks and are often guided by an external controller or internal software. They are extremely good at performing repetitive tasks with great speed and accuracy, and working in hazardous environments. 

Artificial Intelligence (AI) is the ability for machines or computers to virtually learn. It is the latest technology making wide sweeps through the oil and gas industry and has the potential to make a great impact. Typically, software is designed via an algorithm to perform some task or function. Data is input to the algorithm and an output is the result. AI changes this model by learning from inputs and outputs to build an algorithm. 

AI systems are trained by presenting the machine with conforming or “good” and non-conforming or “bad” data sets. The AI’s machine learning programming is then able to determine the attributes of good vs. bad data and develop an algorithm to evaluate any new data. This way of working holds great promise in areas as diverse as reasoning and decision technologies; language; speech; vision recognition and processing technologies; human interface technologies; and new computing architectures and devices.

The convergence of these technologies is driving digitization analytics in the oil and gas industry today. Application and IoT sensor data are being accumulated in greater and greater amounts, now that it is cost-effective to store these data in the cloud. Big data management systems and AI are being used to mine and analyze data for previously unknown correlations that help optimize business processes. Meanwhile, robotics is increasing productivity and keeping the workforce out of harm’s way. Mobility is enabling the workforce to interact with data anytime, anywhere.

Digitalization and analytics enable organizational transformation. Digitalization is not just about deploying technology; it is fundamentally changing the way that organizations are going about their work. As mentioned earlier, workforces are transitioning from focusing on execution to analyzing, embedding and sustaining continuous improvement within an organization. As a result, businesses gain efficiencies and accuracy through organizational change.

Two examples show how that is happening: 

The introduction of “self-service” concepts within functional areas is promoting a merging of IT and business processes. Previously, if a business wanted to make a technology change, the IT department would be in charge of execution. Requirements would be gathered, IT would develop a solution, and the business would test the result upon completion. If there were issues with the solution, a trouble report would be generated, and a discussion would usually ensue on whether the requirements were being met. This is a rigid, cumbersome process, at best, and does not consistently lead to successful outcomes for businesses. 

With the advent of new technologies, such as graphical coding interfaces, departments outside of IT are now able to generate their own solutions to address various opportunities within their organizations. Detailed technical skills are no longer necessary for solutions to be built and tested. This allows for those individuals, who are closest to a given set of data and processes, to rapidly prototype and develop answers to their most pressing questions without needing to translate the requirements to an IT organization. The walls are coming down between IT and other business divisions, as what were once traditionally thought of as IT skills are now being embedded in all areas of a business.

Another organizational change area that enables digitization is “Agile Processes Improvement” and system development. “Agile” is derived from the software development sector, but it also can be used to drive continuous improvement in other areas. The strength of Agile centers is continuous feedback to build a better product or project implementation. Project managers are replaced by “Scrum Masters,” and the product managers lead self-directed teams to ensure that the business is involved on a daily or near-daily basis. Agile is also based on the premise that requirements constantly evolve and cannot be fixed effectively at the beginning of a project. This allows flexibility to be built into processes, which ensure a business receives a result that addresses the actual needs at hand. 


To better demonstrate where digitalization and analytics can improve safety, promote operational efficiencies and reduce risk, let’s look at a few examples. 

Integrity management and smarter inspection technologies. Robotics, big data and analytics pave the path forward for new ways of inspecting pipelines and production facilities, in the subsea area and onshore. This is leading to significant cost-savings, keeping the workforce out of harm’s way, and increasing the speed of inspections, all while revolutionizing reliability in the oil and gas field. 

Reliability is now finding inventive ways to implement risk-based approaches, where inspections are conducted, based on the analytical determination of what could possibly fail, as opposed to time-based interval inspections. This allows operators to pinpoint what should be inspected, and at what interval, instead of conducting inspections that may be redundant and unnecessary. This is accomplished by using knowledge and data about the vulnerabilities faced by each specific part of the system, which is provided by sensors and robotic data. These data, collected in large volumes, are stored on big data platforms and analyzed to determine system or equipment inspection types and frequencies that help maintain overall operational integrity. 

Robots can venture and collect data from locations where it is either dangerous or ineffective for human crews to reach. Autonomous underwater vehicles inspect pipelines, risers and subsea trees, using not only traditional visual recordings but also laser, sonar and advanced imaging technologies that give greater insight into the condition of critical system parts, even those that are subterranean or underwater.

Remotely operated drones conduct fly-by inspections at surface facilities. The drones reach hard-to-reach areas and can replace helicopter monitoring, making inspection processes safer and faster. 

Work management and optimizing crew performance. Analytics can be used, to great effect, to maximize work management personnel that maintain facilities. This can be accomplished through the analysis of data in a Computerized Maintenance Management System (CMMS). Even relatively simple analyses can ensure that the optimal amount of maintenance work is scheduled, level-loaded and efficiently carried out. 

One metric that helps determine if an asset is under-maintained or over-maintained is the percentage of safety-critical equipment. Safety-critical equipment is deemed vital to a facility, itself, and is maintained with more rigor than other types of equipment. Like facilities tend to have roughly the same percentage of safety-critical equipment distributed throughout the asset. If that percentage falls below, or above, a certain threshold, it can indicate that further investigation is needed to right-size the maintenance work. The asset may be under-maintained, if the amount of safety-critical equipment is low, which could impact safety, and over-maintained if the amount of safety-critical equipment is high, which decreases efficiencies and increases costs.

Another metric to analyze is the number of maintainable items in the CMMS. Over time, non-facility equipment tends to creep into the CMMS. This adds hours to maintaining the asset that aren’t strictly dedicated to facility maintenance and can lead to overstaffing.

The analysis of maintenance plans can level-load workforce scheduling to ensure that maintenance activities are distributed evenly across the year. If maintenance plans are higher in one particular timeframe as opposed to another, the facility will have to be staffed to the maximum amount of work, leaving crews idle during less intensive times. 

Detailed analysis of work management data, through analytics, can lead to significant improvements in productivity. 

Reliability with condition monitoring and predictive analytics. Rotating equipment on assets is crucial to power generation, gas compression, gas and/or water injection, and the exporting of oil and gas. Because of the criticality of this equipment, it is imperative to know the health and status of key components of the rotating equipment. This is analogous to IoT. Sensors send data to big data environments, where the information is analyzed to ensure that rotating equipment is running correctly.

Once rotating equipment is instrumented, these data are transmitted to central control centers that monitor the performance of each piece of equipment. Over time, signatures are developed and analyzed to predict possible failure in equipment that appears to be running correctly. Real-time monitoring supports proactive intervention and maintenance activities that correct the functioning of the equipment and help prevent lost production or catastrophic failure. 

Also, the fleet of rotating equipment can be monitored and analyzed to better understand vendor performance, equipment reliability in differing climates and operating environments, and maintenance best practices. Data analytics reveal whether one vendor is consistently over-performing another vendor; if one make and model is performing well in hot, dry climates but not in colder environments; or if a make and model are performing well in one region as opposed to another area, even if both have similar operating conditions. These observations drive procurement and maintenance decisions that can significantly improve the bottom line.

Production performance management and optimizing production facilities. Production performance management can reduce operational risk, decrease down time, improve operational efficiencies and reduce maintenance costs through the real-time monitoring and presentation of operational data. This allows engineering teams to quickly respond to issues as they occur, or in the case of predictive analytics, before they occur.

These systems integrate cloud, big data and analytics to provide real-time feeds and visualizations that enable engineers to detect performance issues. Case management tools can also be made available to incorporate learnings from past operational issues. 

Events, such as equipment performing outside its operational limits or trending toward its limits, can be identified and remediated quickly before causing other issues that could ripple across facilities. Production anomalies can be troubleshooted and managed through resolution. 

The system also enables the interaction between different functions, such as mechanical engineers and processes engineers, who may collaborate to determine the root causes of unplanned events. Previously, teams may have provided solutions isolated to their domain, and this siloing simply delayed resolution, because the interaction between functions was not clearly understood. Now, cross-functional teams, enabled by new ways of working, interact with each other, based on common data that help isolate and repair issues within a production facility.


The oil and gas industry finds itself at a unique period in its history. With increasing competition and potential for regulation in a low-revenue environment, the industry needs new ways to improve processes and increase efficiencies without compromising safety, Fig. 3.


Fig. 3. Digital transformation in oil and gas has the potential to unlock trillions of dollars of value for fi rms, the industry, and global society during the next decade.


Because the industry is asset-based, it is vital for players to be able to analyze and predict how a facility and its components are going to behave over time. Armed with this knowledge and information that is provided by digitization and analytics, better decisions can be made to optimize processes, increase production and improve reliability and safety. 

Fig. 4. Companies still have not realized the entire potential of digital technologies.


Foundational technologies, such as the cloud, big data, mobility, IoT, robotics and AI, enable digitization and analytics. But successful implementation of these solutions requires more than simply introducing new technologies into a business, Fig. 4. An organization realizes complete transformation through concepts, such as self-service and the Agile process that take proper advantage of technology and free up workforces to examine, analyze and respond to information, through a mindset of continuous improvement. Digitization and analytics are key areas that will support performance improvement throughout the industry. wo-box_blue.gif


About the Authors
Jay Stieneker
North Highland
Jay Stieneker is an expert practitioner with North Highland. He has more than 20 years of experience in operations, capability improvement and software development in the oil and gas and aerospace industries. Specific areas of expertise include strategy development; digital transformation; data quality and analytics; Agile Process Improvement; and business process improvement and implementation.
Related Articles
Connect with World Oil
Connect with World Oil, the upstream industry's most trusted source of forecast data, industry trends, and insights into operational and technological advances.