Four steps to extract the full potential of a novel data platform

With good reason, the journey to the cloud for oil and gas companies has taken some time; for many, their journey has only just begun. Plenty of organizations in the energy industry are actively exploring cloud strategies through mechanisms, such as SaaS offerings or lift-and-shift solutions. However, sheer data volume, the importance of security, challenging egress strategies, and data sovereignty requirements have posed challenges to adopting and benefiting from a digitally transformed cloud ecosystem.

Oil and gas companies aren’t just extracting hydrocarbons—they’re mining massive quantities of data. According to IBM, the average oil platform has 80,000 sensors, producing two terabytes of data every single day. Sensors populate every oil and gas operation, and a paper published in Petroleum in 2020 estimated that the industry produced as much as 2.5 exabytes per day. To put that data velocity into perspective, the human species in its collective history had created five exabytes of data by 2003. Oil and gas operators are also some of the biggest consumers of high-performance computing resources of any industry, and energy giant BP owns one of the most powerful supercomputers in the world, so it can meet its own data-processing needs. Significant obstacles can produce substantial rewards, however, and cloud capabilities offer incredible opportunities for the oil and gas industry.

With the flexibility of the cloud, organizations can scale and adjust their computing resources to meet dynamic needs instead of paying for the static processing capacity and disk storage often found in overbuilt or underbuilt on-premises, or co-located data and computing centers. Data stored in the cloud could also power future innovations, both within a business and outside it. These advantages were compelling enough for a collection of oil and gas leaders to create a new, community-driven consortium in partnership with The Open Group, called the OSDU Forum in 2018. According to the group’s website, “the Open Subsurface Data Universe™ Forum provides a vendor-neutral environment for the development of an open data platform and ecosystem. The OSDU Forum is open to stakeholders from oil and gas operators, service companies, technology providers, software companies, academia and others.”

The overall mission is “to deliver an open source, standards-based, technology-agnostic data platform for the energy industry that stimulates innovation, industrializes data management, and reduces time to market for new solutions.” In March 2021, the OSDU Forum delivered the first production-ready release of the OSDU data platform, called Mercury. With open access to a wealth of data, the OSDU promises to help spur the growth of an entirely new ecosystem in oil and gas, and more importantly, in the broader energy industry.

Fig. 1. No matter what methods are used, a data platform like OSDU should be viewed as a market of data technology designed specifically for the end-user instead of a catalog of all the available inventory.

A seismic impact. Because the OSDU data platform makes valuable energy data available in the cloud, it’s encouraging other companies across the energy value chain to move their commercial digital offerings to the cloud as well, Fig. 1. If the industry is using a common, cloud-based data platform, independent software vendors in the oil and gas space, for example, are strongly incentivized to move their own products from desktops to the cloud to take advantage of the robust resources inherent in cloud-based stacks. Furthermore, combining the expansive security model of OSDU with the nature of the cloud, new opportunities arise in collaboration across the energy value chain. Long gone are the days of sharing data via email or expensive data portals, as OSDU provides the very mechanisms necessary to create hub-and-spoke models that enable your teams to share and collaborate on data in a single location across teams, partners, suppliers, and vendors.

Armed with cloud data, operators, themselves, are no longer tied to the static, on-premise computing resources they’ve historically relied on, and they can trade in that traditional hardware and instead unleash infinite cloud computing power on some of the world’s most complex problems. The OSDU is the catalyst for this transformation, because it’s a central, open hub connecting different players and products in the oil and gas space, but it’s also powering another kind of transformation.

Many international oil companies are changing their business models, becoming integrated energy companies, as they look to expand their portfolio of energy beyond oil and gas. Shell is one such company, and ambitious government emissions targets and shareholder pressures are steering it away from traditional profit centers and toward liquified natural gas and other lower-carbon sources of electricity.

OSDU might have “subsurface” in its former name, but at its core are the mechanisms to serve as a single, crowdsourced data platform for the energy industry—regardless of business model. That flexibility means the platform can adapt to house different kinds of data, empowering organizations as they transition to new energy generation methods. The Open Group even goes so far as to include this capability in its platform mission, voicing a commitment to support a wide variety of data types that may emerge in future business domains.

All in the approach. Whether it’s being used to bring cloud capabilities to the oil and gas industry or enabling the transition to cleaner forms of energy production, the utility of the OSDU data platform depends largely on the angle from which companies approach it. Many organizations still view it as a place to bring data, unify its format, and make it discoverable and usable by end-users, applications, business process, and data analytics workflows. To be fair, it’s a common and accurate perspective, because those goals are what a typical data platform aims to accomplish. However, this vision falls short of extracting the true value of OSDU.

Yet in my nearly two decades in the industry, it has been my experience that even the most powerful and far-reaching data platform will inevitably fail to meet the end-user’s needs or earn their trust when it’s used as just another data platform. In general, a data management organization looks to bring as much data into a platform as possible, with the belief that a data-rich environment offers limitless possibilities for the end-user. That idea is predicated on another fallacy—that all data are good data.

Take the Google Search engine, which is a powerful tool that links those seeking information with websites providing that information. It uses sophisticated algorithms to determine the likelihood that the search results’ content matches the context of the words entered by the end user. As a result, despite millions of matches, more often than not, the first page of results provides you with the search result you were seeking. Now imagine performing that same search without the intelligence of the algorithm extrapolating context from a few search words. Clearly, it’s possible for a platform to be rich in data but far from valuable to the end-user.

In fact, bringing all available data into a platform can sometimes create more issues than solutions. If the goal is to provide business users and their applications access to the expansive data assets of the organization, the data consumption capabilities must be designed, based on the needs of the data consumers—the end-users.

Fig. 2. The average oil platform has 80,000 sensors, producing two terabytes of data every day. Sensors populate every oil and gas operation, and it’s estimated that the industry produces as much as 2.5 exabytes per day.

Dealing with data consumption. Different data are used in different ways. Some are consumed by users, for example, while some are consumed by applications that the users rely on to serve up information and perform various other analytical and intelligent functions, Fig. 2. There’s a big difference in these two use cases, and using different tiers of data presentation (as outlined in Pariveda’s Modern Data Enterprise framework) could help organize and segregate various data products—the units of value that satisfy a user’s data needs.

No matter what methods are used, however, a platform like OSDU should be viewed as a market of data products specifically designed for the end-user, instead of a catalog of all the available inventory. OSDU isn’t an infinite space to satisfy our data hoarding desires; it’s a curated ecosystem of data products designed to deliver value, based on the end-user’s business cases. For an organization to extract as much value as possible, it’s critical to start at the end—the end-user’s needs. This isn’t to say only blessed and high-quality data should live in OSDU. Curation is not a mechanism of quality; it’s a mechanism of ensuring that what’s present has a specific purpose linked to business outcomes.

IT and the data governance organization have their own ideas of an effective OSDU implementation, but they’re not the ones who will ultimately be relying on the platform. Some might say OSDU has done its job of encapsulating the complexity of data collection and aggregation if its end-users are none the wiser, but we must evolve beyond this mindset.

Fig. 3. The OSDU data platform is more than what its name implies. Instead of a mere data platform, it’s a unique innovation in an industry that has struggled to overcome significant hurdles and benefit from the savings and scalability of the cloud.

Instead, OSDU implementations should follow right-to-left thinking, starting with the context of end-user needs and workflows, and then move backward—through the data platform to the data transformations to the data enhancements to the data modeling and, finally, to the data sources. Those end-users are geoscientists, climatologists, hydrologists, petroleum engineers, and more; engaging them will help data management organizations define the most useful and significant data products. In other words, OSDU can serve as a bridge between an organization’s data customers and its data curators.

Behind the why. There were certainly conversations happening between data management organizations and end-users before the creation of OSDU. However, the cloud-native data platform’s unique combination of flexibility and consistency can help evolve those conversations, reducing time spent on “how” and moving more heavily into the “why” of data products, Fig. 3. When data curators know more about what their data customers in the business need, they can leverage this knowledge and partnership to produce more transformative insights by shaping and aggregating the data in such a way as to ease the burden of consumption.

As leaders and executives in the oil and gas industry look to nurture these kinds of outcomes and maximize the benefits of OSDU for all the different participants in the energy value chain, they should adhere to the following four steps closely.

1. Plan an adoption journey that looks beyond the platform. OSDU is much more than just a data platform, and any adoption journey should be planned with its broad applicability in mind. Instead of handing the implementation reins over to IT and the data management organization, keep the business value of an effort top of mind and work right-to-left to make sure end-users are at the heart of any initiative. Define end-user needs clearly and explore how OSDU can meet those needs.
   The platform’s support of the vastness of data type storage capabilities (exciting side note: real-time data capture and streaming are currently in the works) will be particularly important to earn end-user buy-in. In terms of the organization’s broader adoption, look for business cases at the edge of existing data ecosystems, and then go one step further to see what use-cases turn up.
2. Engage all areas of the business from day one. An OSDU implementation isn’t merely an IT project, a data management project, a data science project, or a business project—it’s all of them and more, and should be undertaken as such. If an OSDU data platform implementation team lacks members from any of these groups, take a step back and add some depth and expertise to the roster before proceeding. Once OSDU is implemented in an organization as a mere data platform, it will be difficult to go back and change entrenched notions about its utility and purpose, so start off on the right foot. Explore the concepts of emergent teams, and apply them to your journey.
3. Plan the journey, but stay flexible. The OSDU data platform is gaining traction, but it’s still in its relative infancy. Expect it to continue to grow rapidly and expand to touch a variety of business domains. Some of these initiatives will, no doubt, provide value that nobody saw coming. Approach OSDU with an entrepreneurial mindset and stay open to new possibilities that emerge, no matter how wild they might seem initially. Mistakes are part of the entrepreneurial journey, so fail quickly, learn as much as possible from the experience, and go back to the drawing board for another attempt.
4. Give back to the community. As you learn, consider pushing those learnings into the OSDU data platform community. More often than not, enhancements to the core OSDU data platform’s capabilities won’t constitute a competitive advantage for your enterprise. OSDU is where it is today, because generous individuals and organizations put in countless hours and then donated their efforts for the benefit of an entire industry. With any community-sourced platform, it’s critical to actively manage the competition and collaboration polarity to reach an equilibrium of value contribution and value extraction. Achieving that balance requires an organization to give back and help build an ecosystem that benefits all users—even their competitors. A rising tide lifts all ships.

The OSDU data platform is much more than what its name implies. Instead of a mere data platform, it’s a revolution in an industry that has struggled to overcome significant hurdles and benefit from the savings and scalability of the cloud. With an open, cloud-native data platform, OSDU will empower an entire ecosystem of cloud-based applications and innovations, driving significant value in the oil and gas industry and the energy sector as a whole.

To take full advantage, however, adopters will need to resist the temptation to pigeonhole the OSDU and isolate it to an IT and data management solution. By taking a holistic view of its implementation, engaging end-users and other key stakeholders in the project, maintaining a willingness to experiment, and acknowledging a responsibility to the broader community involved in the effort, companies can extract maximum value from the OSDU data platform for years to come. And better yet, let it aid your organization in its energy transition journey.

About the Authors

Alan Henson

Pariveda

Alan Henson is a principal at Pariveda, a consulting firm driven to create innovative, growth-oriented and people-first solutions. Mr. Henson’s experience includes leading operations for a multi-national software company, while managing and delivering multiple large-scale software development projects. He is also proficient at developing business intelligence solutions, building virtual reality software focused on training and talent development, and implementing software packages in the upstream oil and gas industry with an emphasis on data management, data mining and analytics. alan.henson@parovedasolutions.com