Building data products from external data: use cases and best practices

Data products – high-value, ready to consume, standardized and business-focused data assets – are essential to turning organizational information into business value. They enable business users to consume data at scale by making it available in easy to use formats that don’t require technical skills to understand or act on. This means they unlock the power of data and drive greater insights, improved performance and increased innovation.

More and more organizations are now deploying data products, making them discoverable and accessible through intuitive, centralized, self-service data product marketplaces. And while data products can be built from internal information, increasingly organizations are also relying on external data sources for their data products.

These can be combined with internal data to provide context and understanding (such as adding geographic, market or statistical information), or be standalone interfaces into external data either purchased or provided by partners. With this trend continuing to accelerate, we explain how to successfully harness external data and turn it into valuable data products.

At Huwise we define a data product as:

A data asset that meets high standards around its presentation, readability, and quality in order to enable its consumption by a large number of users. To establish complete trust, it must always provide access to its source data and include a data contract specifying the product owner’s commitments through a SLA and terms of use, and be machine-readable to guarantee automated usage.

It is fundamentally different from a raw dataset due to:

• Ease of consumption – it contains everything needed for a non-technical user to discover, interact with, understand and benefit from the underlying data
• Trustworthy – it is reliable and governed by a data contract that specifies how it can be used and how it is managed. Access is provided to the underlying data to reinforce this trust.
• High-value – it meets a specific, ongoing business need within the organization, delivering measurable business impact
• Continuously updated – it is not a one-off report, but is monitored, managed and updated on an ongoing basis
  High-usage – it is used by multiple people and groups across the business, rather than meeting an individual need

Examples of a data product include interactive dashboards, predictive models, or recommendation engines. Data products can be deployed internally, shared with partners or monetized to create new revenues. They can be used by both humans and artificial intelligence (AI).

Data products have the power to transform internal information into business value, increasing data consumption, building a data-centric culture and enabling AI. However, while the volume and range of internal data is growing, relying solely on an organization’s own data can have limitations.

As it is focused on the business itself, it lacks wider context and depth. For example, it may be incomplete in certain areas or be difficult to compare to the wider market. Equally, it can be difficult to visualize and understand, requiring the addition of external geographic, statistical, or business classification information.

Essentially, this means that data products built just with internal data may only give a limited picture of the world. In some cases, such as improving company operations, this may not be an issue, but in other areas, such as predictive forecasting, it can be a problem.

Adding external data will therefore increase usability and the value from data products. This external data could be:

• Market intelligence, such as overall market size or predicted sales, provided by aggregators or data providers
• Complementary data from partners, such as energy usage data provided by utilities to municipalities or cities
• Geographic or administrative data, such as maps and state/county boundaries
• Official statistical information, such as populations, company directories, and business classifications

By harnessing external data within data products, businesses benefit from:

• More complete information enabling better decision-making, especially at a strategic level
• Lower risk as predictions take into account all factors and wider market conditions
• The ability to better understand and manage complex areas, such as Environment, Social and Governance (ESG) that rely on data from across ecosystems and supply chains, ensuring compliance
• Greater collaboration with partners to deliver innovation and efficiency

Data product marketplaces make access, discovery, sharing and data consumption simple and seamless for all users, whatever their level of technical skills or knowledge. When it comes to external data they deliver additional benefits to organizations around data products:

• As they centralize all data in a single place, any purchased external data can be accessed through the data product marketplace. This avoids data duplication, when multiple departments subscribe to the same service independently and therefore reduces subscription costs
• It enables data products to be shared externally, either publicly or with partners/customers. This allows seamless collaboration while opening up new opportunities for both differentiation or monetization, creating new revenues
• For public bodies or regulated industries such as energy, it helps demonstrate transparency, driving stakeholder engagement and supporting wider objectives, such as enabling decarbonization

UK public body the West of England Mayoral Combined Authority makes decisions and investments that benefit people living in the west of the country. This includes driving collaboration to meet goals around net zero and accelerating the recovery of the natural environment.

As part of this objective, it has created its interactive Local Nature Recovery Strategy (LNRS) toolkit data product. This brings together external data from different players across the region, and central government, to provide an intuitive resource that helps farmers, landowners, businesses, local authorities, community groups and citizens to take effective action to restore nature and natural habitats.

Accessed through a drill-down map, users can zoom in to view specific, detailed areas, click to see current biodiversity levels and pictures of local species, and link directly to resources, such as available environmental grants and programmes that can aid nature recovery. Local authorities are also able to use the toolkit as part of the planning process to meet requirements to increase biodiversity.

Incorporating external data into a data product strategy delivers multiple benefits, adding context, giving a more strategic picture and enabling greater collaboration and monetization.

Achieving these benefits and ensuring that data products meet business needs requires a focus on three key best practices:

Reliable, high quality data is vital In order to deliver effective and trustworthy data products. This means that any external data has to meet the same quality standards as your own information, and requires rigorous testing, formatting and monitoring, backed up by concrete SLAs and contracts. As well as ensuring quality, any external data has to be compliant with your own governance standards and regulatory requirements. For example, it is vital that data that contains personally identifiable or confidential information is clearly labelled and anonymized, meeting regulations such as the GDPR and CCPA.

External data has potential uses across the organization, making it an asset that must be fully integrated into your data stack. This allows it to be used within a range of data products, as well as being available across the business through your self-service data product marketplace. It should be enriched and processed to match your internal standards (such as how you structure names and addresses) so that it is completely interoperable with existing systems, and be available in a range of formats that facilitate sharing.

One-off uses of external data, such as within single reports or solely using it within one department don’t maximize the value it brings. Design your data products to ensure that they can scale to be used across the business, without requiring specialist skills. The focus should be on making it easy to discover, consume, collaborate and give feedback around data products by sharing them through your data product marketplace. This maximizes user adoption, increases ROI (particularly if external data is paid for), and delivers real value to the business.

Being able to access your internal data and share it through data products unlocks real value from your information. Supplementing it with external data dramatically increases this value by providing different perspectives, additional information and greater context. However, these data products need to be easily accessible and usable, requiring a self-service data product marketplace to open up new opportunities for improving performance, increasing collaboration and underpinning greater innovation.

Find out more about how Huwise can transform your data product strategy through our centralized, intuitive data product marketplace solution. Learn more by downloading our Data Product Marketplaces Demystified guide.