How AI is transforming our data portal solution and client data projects
Over the past months Huwise has been working to transform its data portal solution by enriching it with AI, helping clients to save time, improve the experience for their users, and reduce the risk of errors within processes.
Progress in Artificial Intelligence (AI) is accelerating rapidly, with the pace of adoption increasing significantly in 2024 with the launch of new applications and technologies based on AI models.
When it comes to data management, which involves many manual, time-consuming tasks, AI is also able to revolutionize processes and tools.
Consequently, over the past six months, Huwise has been working to transform its data portal solution by enriching it with AI. The aim is to help our clients to save time, improve the experience for their users, and reduce the risk of errors within processes.
Transforming the data portal search experience with AI
We are committed to accelerating data democratization for our clients by applying AI within our solution.
An intelligent AI-based search engine
Since December 2023, Huwise portals can harness a new, semantic search based search engine. This goes beyond literal, keyword matches to understand the meaning of terms and their context.
The new search engine provides users on client portals with real-time result suggestions and access to:
- Datasets corresponding to the searched theme
- Data assets corresponding to the searched theme
- Data assets that include the searched keyword or a similar keyword
This new search method delivers more relevant results, faster. It greatly enhances data discovery, providing a smoother and more efficient user experience. For portal administrators, this also means easier maintenance, as there is no longer a need to add an exhaustive list of keywords to metadata — AI handles it all.
Similar dataset recommendations
To improve data discoverability and streamline the experience on our clients’ data portals, we also offer an AI-based automatic data recommendation feature.
Through this all other most relevant datasets are displayed on data asset pages viewed by users.
This approach aims to simplify the data discovery experience for all users, drawing inspiration from the best practices of e-commerce marketplaces. It aligns perfectly with Huwise’s vision of making data access and use as simple and intuitive as possible for a broad audience, from data experts to novices. Each user can easily navigate through all data assets on a portal to find exactly what they need.
Driving data portal innovation through AI
Our choice of migrating to semantic search stems from a continuous desire to improve the user experience within our data portals. Search is a crucial pillar of this experience, enabling users to easily discover the data they are most interested in. With keyword search, users are often frustrated and receive irrelevant or incorrect results if they do not enter exact keywords used to describe data assets. Instead, semantic search offers greater flexibility, generating relevant results even for approximate or imprecise queries or those in other languages.
The semantic (or vector) approach eliminates potential user frustration and significantly improves their search experience. Implementing AI was a no-brainer once we evaluated and compared several AI models, the best of which provided a 64% gain in relevance compared to a keyword search engine.
Enhancing the relevance of our AI-based search engine
Our journey of integrating AI into our search engine has been iterative. It began with a vector approach, which managed approximate queries with spelling errors or synonyms, with variable result relevance. We then adopted a hybrid approach, combining keyword and vector methods, ranking results by semantic similarity and popularity, thus improving their relevance. Finally, we returned to a solely vector approach, enriched with a relevance algorithm to present only the most pertinent results and display nothing if results are insufficient. This latest improvement ensures optimal result quality, considering the approximation or imprecision of user queries.
Our unique and innovative algorithm effectively identifies the best ratio between the number of suggested results and their relevance. Our evolution from a vector approach to a hybrid one, and ultimately back to semantic search supported by this new algorithm, reflects a process of continuous evolution where we re-evaluate previous choices while enriching them with new knowledge. It is very much a scientific approach, where continuous exploration and the integration of new data is essential to delivering ever more effective solutions.
Continuous iteration delivers improving results
To measure the success of our search engine, we use a relevance index that scores results on a scale ranging from 0 to 1. Thanks to our work on different approaches, this index has risen from 0.52 to 0.87 in just a few months.
Although this relevance index is crucial, it is not the only factor we consider to ensure search engine quality. As a solution provider, we have to meet the different and unique needs of all of our clients.
Our challenge is to find the right balance between maximizing the number of relevant results and minimizing less suitable results in order to offer a satisfying experience to all of our users. For example, some clients prioritize the quantity of relevant results, while others place more importance on reducing irrelevant ones.
Clients at the heart of our innovation
Our beta-tester clients have played a central role in the continuous improvement of our search engine. Their feedback throughout the development process, along with the continuous learning of our AI algorithms has helped shape a more user-centric search tool.
The new AI-powered search engine makes it much easier to access and discover data. It suggests relevant data even when approximate keywords are entered, reducing the risk of missing out on the data users are actually searching for. It also enriches the user experience by highlighting data that is interesting but wasn’t directly searched for, creating serendipitous results. It also has a major impact on the quality of our metadata, ensuring we focus even more on its quality and completeness. As beta testers, we found the whole experience useful thanks to the openness of Huwise’s teams when it came to listening to our suggestions and feedback. We’re very interested in seeing the next steps in AI-powered search, exploring the solution's forthcoming AI features, and collecting feedback from our users to better understand how these innovations help them find data on the portal.
AI and data portals: new features coming soon
Our use of AI isn’t limited to search. We now have a range of new AI-based features under development or being tested with our clients, which will soon be available.
Exploring data with AI
Soon, users of Huwise data portals will be able to rely on AI chatbots to explore data. By querying a virtual assistant using natural language, they will receive instant answers in specific visualization formats, such as maps, figures, or graphics.
Building pages with AI
Portal administrators will also soon be able to use AI to create and integrate complete pages on their portal. Whether generating dynamic visualizations, building pages, or arranging elements, a virtual assistant will automatically respond to their prompts.
If you want to learn more about the upcoming AI innovations that are enriching Huwise’s solution, contact us to speak to one of our experts.
Share this post:
Articles on the same topic:
About the author
Anne-Claire Bellec
Anne-Claire Bellec has more than 15 years of experience in marketing strategy. She has previously held roles as Chief Marketing Officer and Director of Communication within both agencies and SaaS companies specializing in data and digital solutions.
More articles
