
The complexity surrounding business support
Business support is at the core of the government’s economic growth mission. To meet the diverse needs of businesses and remove barriers to growth, the government relies on a wide range of digital services. This digital support offer is complex and to be effective, it must be well connected and coherent.
Business needs can be described through research, but data is also needed to deliver value to them through a complex network of services.
Company data tells us who businesses are such as their size, location and financial performance. Advisers use business profiles to identify businesses most likely to benefit from government support. Digital systems use profile data to create personalised workflows and reduce the data entry burden.
Until recently, however, the value of company data was offset by the work required to use it.
Across BIST, teams and individuals:
- sourced, cleaned and linked their own company data, resulting in work duplication and increased infrastructure costs
- captured business data using different identifiers and often unvalidated free-text inputs, producing data that could not be linked nor reused across services
- procured data suppliers to plug holes in data without capacity for reuse nor shared visibility, impacting costs and increasing the risk of vendor lock-in
- copied data without maintaining it fresh, leading to presentation and usage discrepancies across BIST.
Introducing the Company Data Layer
A year ago, we set out to change this. We wanted to create one reusable source of company data that could allow teams from across BIST to access what they need.
We created the Company Data Layer (CDL), a common component for company data that does the work once. It gives services a simple way to find reliable company information, without each team sourcing, cleaning and maintaining its own data. Over the last year, the CDL has demonstrated its value in real services. We have reliably delivered insights to our internal CRM, the application and case management systems for the British Industrial Competitiveness Scheme (BICS), and other services across BIST.
After 6 months of being in beta, we can reflect on the CDL’s impact on our services and look ahead to future possibilities.
How BIST is benefiting from the Company Data Layer
Improving data quality across the support ecosystem
When services can identify businesses reliably, the data they collect becomes more useful and easier to reuse. Easy access to business identification tools, like search and data matching, helps teams capture recognised identifiers earlier in the user journey. This improves downstream data linking and use.
Speeding up digital product and service delivery
Using or searching through company data is now as easy as making an Application Programming Interface (API) request or consuming an integration event. Teams can remove the burden of sourcing, cleaning and storing company data and focus on creating the compelling user experiences that make their products uniquely valuable. Michal Charemza, Lead Data Engineer working on the British Industrial Competitiveness Scheme, says:
CDL has solved the problem of merging Companies House bulk data with its streaming API. Without CDL, we would have been much more nervous about using Companies House data in a high-profile public-facing service.
Martin Tierney is a Senior Software Engineer working on the Data Hub Customer Relationship Management (CRM) tool. He says:
Having a source of truth for company data for the department is proving to be very useful, particularly allowing Data Hub to remove an external dependency and the complexity contained in it.
Revealing data problems and complexity
Having a single source of data can simplify complexity and reveal and resolve discrepancies. Company data can be spread across sources and described in different ways, so simplifying and checking it required great manual effort before the CDL.
Smoother and more effective back-office processes
Improvements to data quality means it is easier to join the dots across the ecosystem to understand users and their activity. Cross-service activity provides invaluable contextual information to BIST advisers who need it to deliver effective support to businesses.
How the Company Data Layer has acheived this
While the Company Data Layer’s value is generated from widespread adoption, its success stems largely from its design features.
Users choose the CDL over alternatives because of extensibility, flexibility and resilience.
The CDL can be extended
We designed it to easily extend to more data sources as we acquire them by:
- identifying when different data sources refer to the same business
- defining rules that flag discrepancies by blending conflicting or redundant information about the same business
- exposing metadata about where business information is coming from and how recent it is
- allowing users to get a default view of a business, or the raw data from each source if they need it
Flexibility is built in
We recognise that different services and users need to consume our data in different ways. The service supports:
- searching for businesses using approximate values
- retrieving links between businesses, such as ownership structures
- sending notifications when business data changes, so services can proactively respond to these events
- recovering the change history to business data, allowing us to reconstruct a business’s history
Soon, we will support a connector for our internal AI tools, and a way of searching for lists of businesses based on customisable criteria. This will allow users to interact with the CDL using queries in English through their AI chat of choice. The combination of these features opens new opportunities, such as allowing our business advisors to find promising leads.
As we expand to more services, we help them access rich and consistent data allowing them to be smarter and more personal for their users. For example, the ability to look at a business’s history is critical for funding and grants leading to better decisions and reducing bureaucratic effort for businesses.
The CDL is designed for resilience
We have invested in the CDL’s resilience due to its importance for our infrastructure. It scales automatically based on demand. We now have a complete suite of alerts notifying the team of various problems, including slow responses, old data, or unusual traffic.
We work with the services using the CDL to ensure we have robust fallback mechanisms. This means that they can continue to operate if the CDL experiences any short-term issues. We now have the confidence to expand adoption at pace and onboard citizen-facing services.
What is next for the Company Data Layer?
Company data is used across government to support businesses and growth. If you work in local government or partner organisations such as the Growth Hub network and Chambers of Commerce, you likely face many of the same data challenges we faced in BIST.
Our ambition is to make the Company Data Layer accessible beyond BIST, so that other departments and organisations can benefit from it. We have built a service that is extendible, flexible and scalable from the ground up.
If you work with businesses, you can help us by sharing your interest in the CDL. Tell us who you are and what challenges you face with company data. We would be excited to showcase our service capabilities and supported use cases. Please get in touch at datamatching@businessandtrade.gov.uk.


Leave a comment