Driving impact by scaling data products – best practices from Gartner

Data products, highly-consumable, business-focussed data assets, are widely seen as central to increasing and scaling business data use. By providing data in a format that is easily understandable by business users, and is machine-readable by AI, data products unlock real value from an organization’s data.

Chief Data Officers and other data leaders understand the importance of data products. But how do they turn theory into practice and actually create and share data products successfully at scale? Where should they start and how can they maximize sharing?

To help, Gartner has produced a range of presentations and reports, including a new study on data marketplaces. In this blog we share some of these best practices to help organizations drive impact through their data products.

According to Gartner’s Chief Data and Analytics Officer (CDAO) Agenda Survey, 50% of organizations have already deployed data products, with another 29% committed to piloting or considering deployment within the next year.

However, to unlock real value, organizations need to industrialize and scale their production, and ensure that they are discovered, understood, and consumed by their target audiences. This requires a comprehensive strategy and processes, backed by an end-to-end technology infrastructure that covers their creation, deployment, and ongoing maintenance and adaptation. Organizations need to move from one-off data projects to a factory approach that standardizes processes to deliver consistency and volume.

As well as scaling production, organizations also must scale consumption. In a new study, Data Marketplaces Are Key to Faster Data Product Adoption and AI-Ready Data Delivery, Gartner analysts Richa Jha, Ehtisham Zaidi, Robert Thanaraj, and Michele Launi outline the benefits of data marketplaces. Providing a single, centralized space for data sharing, they break down silos and enable secure,scalable data product sharing, helping transform data into a core strategic asset. Huwise is listed as a representative stand-alone data marketplace vendor in the study.

A data marketplace is a collaboration platform connecting data producers and consumers, enabling them to share integrated and semantically enriched data internally.

One of the key reasons for the growth of data products is the wide range of business-critical uses cases they meet:

Too much data is either stuck in departmental silos or is only understandable and usable by technical experts. Data products break down these barriers by ensuring that trusted data is widely available in business-focused formats that meet specific organizational needs. Delivered through intuitive, self-service data product marketplaces they maximize usage across the business.

Data teams currently spend enormous amounts of time responding to business requests for data. Often, these overlap, leading to duplication and wasted time. Data products are designed to meet specific business requirements for large groups of users. This reduces the workload on data teams and enables them to focus on higher-value activities.

Effective business decision-making can be hampered by inconsistent data and terminology, meaning information cannot be confidently compared between departments. Data products provide agreed, consistent, and high-quality data, acting as a foundation for better-informed decision-making and collaboration.

As regulations increase around data, organizations need to be able to audit and track its usage across the organization. Data products include governance frameworks to protect sensitive information, and provide transparent data lineage to monitor and show compliance.

To be successful, AI requires high-quality, well-structured data. Data products provide this source, giving immediate access to governed, compliant data. Usage can be tracked and audited, aiding transparency and ensuring effective AI governance.

Organizational data doesn’t just benefit the business internally. It can be used to drive new revenue streams through data monetization, with information supplied to third-parties. Data products simplify monetization by packaging data into clear, updatable products that can be easily bought and consumed by external customers.

Gartner identifies three main types of data product, each of which meets specific business needs:

These typically aggregate data from multiple systems (such as CRM or ERP) to create a unified view of areas such as finance or procurement. Alternatively, they may productize external data, such as demographic or zip code data, so that it can be used across multiple downstream use cases. This ensures consistency and prevents duplication.

These deliver a single, agreed view of core business entities, such as customers, products, suppliers, or citizens. These ensure that there is consistency across the business and between departments, breaking down silos and enabling accurate analytics. These data products provide a building block that can be used within business processes and more complex data products.

These deliver actionable insights from data for business users. They combine relevant data, business logic, and interfaces to ensure data can be understood, queried, and consumed without requiring technical skills. They can be delivered in the form of interactive dashboards, analytical models or embedded applications.

Enabling the successful adoption of data products is more than a technical challenge. Simply building a data product doesn’t guarantee its usage or tangible business value. Four further obstacles must be overcome:

• Data products must be aligned with a clear business need, and developed in conjunction with users. Failing to meet user requirements will lead to underutilization and a lack of ROI.
• There must be strong governance and security to control access to the information contained within data products. Data quality must be continually certified, sensitive information must be protected, and regulatory compliance safeguarded.
• Data products must be easy to discover and consume. Without the right delivery methods or clear product descriptions, potential users will struggle to find relevant data products, again reducing engagement and adoption. As Gartner emphasizes, data marketplaces are central to delivering consumption.
• There must be clear collaboration across the organization to build a strong data culture. This requires training, education, and the breaking down of silos, both between departments, and technical and business teams.

Based on its experience, Gartner recommends adopting a clear framework and process to both begin and scale data products within the organization.

Lydia Ferguson, a Gartner analyst specializing in data product strategy, recommends a “crawl, walk, run” approach to data products. Begin with a single data product to meet a well-recognized, high-value need, and gain the buy-in of relevant business users. Scope out the requirement and if it can be met through a data product. Based on user specifications, create a first prototype or Minimal Viable Product (MVP) and adapt it based on feedback.

At the same time ensure that the data product delivery framework is put in place. Adopt standardized templates (including data contracts), metadata and semantics so that future products can be built using the same processes.

Importantly, organizations should appoint a data product manager to orchestrate the creation of data products. This cross-functional role bridges the needs of business users, technical and governance teams, focusing on delivering and driving the adoption and maintenance of a specific data product. Once the first product has been launched and adopted, more can be researched and built, creating a wider portfolio that maximizes value.

Clearly a successful data product relies on high-quality, compliant and business-focused data. That requires a technical reference architecture designed to turn raw data into understandable data products. Gartner recommends a five phase reference architecture that spans the end-to-end data journey:

1. Acquire – access data from data sources across the organizatio
2. Integrate – bring data together, through techniques such as replication or virtualization
3. Organize – centralize data in a single platform, such as a data warehouse or data lake
4. Analyze – apply analysis to create a meaningful, comprehensive and accurate data asset
5. Deliver – provide the resulting data products through the right channel for users, such as an intuitive, self-service data product marketplace

Building a data product that is then not used, or where utilization drops over time, undermines the whole program and its objectives. Even the best-designed and most relevant data product won’t be consumed if it is difficult to find or hard to use. That means that organizations must focus on how data products are delivered and shared with all users to ensure their adoption at scale.

The key principle to follow is that user needs and preferences must dictate delivery methods. Different audiences will require different ways to access data. For example, application developers, AI and technical experts will want to be able to connect to data products through APIs. Business users will want a more visual, business-focused experience, such as via dashboards.

All of these needs can be met through a single, centralized data product marketplace. This provides an intuitive, self-service experience, based on e-commerce marketplace principles that make discovery, access, and consumption of data products simple and seamless. Capabilities such as AI-powered search and comprehensive metadata connect users easily with relevant data. Clear descriptions of data products, including details of their owners, build trust and confidence, while security and governance is enforced through granular access controls. Unlike a data catalog, which is focused on technical users, data marketplaces make data available to the entire business, democratizing access and driving consumption.

Gartner highlights four key capabilities for successful data marketplaces:

• Self-service, intuitive data discovery and seamless, secure access for business users and teams
• Strong data preparation capabilities to streamline data integration and cleansing
• The ability to share data securely, backed by automated governance to control access and provide an audit trail
• Usage tracking and e-commerce style features that drive engagement and collaboration, such as ratings and reviews

Data products have to perform across two key dimensions. First, they have to be technically robust and be based on reliable, high-quality and up-to-date data. Secondly, they have to be useful to the business, with strong adoption rates and satisfied users to drive business value and ROI.

That means that alongside technical performance and data observability, organizations must focus on collecting usage information and feedback. How many people are using the data product, and is this changing over time? Which departments are they in? How many people discover the data product through search, but then fail to access it? Combining this information with direct user feedback helps understand whether the data product is actually meeting a business need, or if it has to be adapted or even replaced.

Building data products is an ongoing process. Based on the foundations, processes, and templates put in place, organizations need to continually scale production, working with the business to identify, test and deploy new data products, creating exponential value and increased data consumption.

An effective data product strategy requires organizations and data teams to adopt a product, rather than a project mindset. They have to put in place replicable processes to build data products at scale, identify real business problems, work with users to solve them through data, and then ensure that products are easily accessible and consumable through intuitive, self-service data product marketplaces. Only then will they successfully harness data to improve performance, efficiency and innovation.

As part of its commitment to supporting organizations on their data product journey, Huwise will be exhibiting at the forthcoming Gartner Data & Analytics Summit in London between 11-13 May. Find out more about how we can help transform your data strategy by visiting us on booth 113 or booking a meeting here.