{"id":7542,"date":"2024-10-13T01:01:00","date_gmt":"2024-10-13T01:01:00","guid":{"rendered":"https:\/\/dgen.net\/0\/?p=7542"},"modified":"2025-04-06T21:58:05","modified_gmt":"2025-04-06T21:58:05","slug":"how-can-we-find-the-goldilocks-zone-of-our-national-data-infrastructure","status":"publish","type":"post","link":"https:\/\/dgen.net\/0\/2024\/10\/13\/how-can-we-find-the-goldilocks-zone-of-our-national-data-infrastructure\/","title":{"rendered":"How can we find the Goldilocks Zone of our National Data Infrastructure?"},"content":{"rendered":"\n

(please note my disclosures at the end of this post)<\/em><\/p>\n\n\n\n

This post contains (as-always) personal opinions and thinking-in-progress (strong opinions, weakly held) as I navigate through the maze(s) of data governance. Persistent in my questions are how we apply the Goldilocks principle<\/a> of governance vs innovation: to remain flexible and avoid brittleness.<\/p>\n\n\n\n

Given the conversations I\u2019m in at the moment I believe it very important to have as much debate as openly as possible, as this impacts everyone<\/em>.<\/p>\n\n\n\n

Here\u2019s a question:<\/p>\n\n\n\n

How should the UK implement its National Data Infrastructure<\/strong> so that it provides consistent control, sharing, and security for data, much like other public infrastructures such as roads and utilities?<\/p>\n\n\n\n

\n

\u201cEverything should be as simple as it can be, but not simpler\u201d
[often attributed to Einstein]<\/p>\n<\/blockquote>\n\n\n\n

Having initiated \u2018data as infrastructure<\/a>\u2019 at a political level in the UK in 2013, I\u2019ve been watching and learning on this for a long time. We\u2019ve made remarkable progress in some areas, less so in others.<\/p>\n\n\n\n

Let\u2019s explore, starting with some lessons learned<\/strong>.<\/p>\n\n\n\n

One immediate reflection is that as over 60 countries moved to copy the Open Banking Standard, those who took only \u2018parts\u2019 of the system tended to experience less ideal outcomes (to the point that they often add them back in later to help course-correct \u2014 most notable are those who thought this was \u2018just a tech problem\u2019 and only took the open API principles).<\/p>\n\n\n\n

We can look to past UK data initiatives like MiData and Open Banking and potentially infer that voluntary-only initiatives and centralisation do not work, and that the decentralised nature of Open Banking was more successful. However, these are shallow reflections.<\/p>\n\n\n\n

We can point to three factors beyond centralised or voluntary which are dependent on the use case<\/strong>:<\/p>\n\n\n\n

    \n
  1. Market incentives<\/strong>: there must be an economic argument that policy can then amplify or mandate. If there is no financial incentive, there will be no movement (regardless of central vs decentralised, mandate or not \u2014 mandates are also often ignored). Midata was mostly tech-led, Open Banking was use-case led.<\/li>\n\n\n\n
  2. Removal of friction<\/strong>: Removing transactional friction may seem like something everyone wants, but not if your current business model relies on it. There must be \u201csomething in it\u201d for everyone, or at least a path to cost reduction or a new business model. Removing friction can help everyone go together: this is never a \u2018technology problem\u2019 (e.g. absence of a data ontology).<\/li>\n\n\n\n
  3. Mandates<\/strong>: if the friction is \u2018too high\u2019, regulatory intervention is necessary to mandate participation and move the market. However, if the market can demonstrate self-initiation, then \u2018endorsement\u2019 from government can be sufficient to drive impact. Value creation must be visible in both cases.<\/li>\n<\/ol>\n\n\n\n

    I believe that we need to embrace these factors in our systems design<\/strong>.<\/p>\n\n\n\n

    We need to:<\/p>\n\n\n\n

      \n
    1. work out where the lines are drawn between\u00a0central<\/strong>,\u00a0pre-competitive<\/strong>\u00a0and\u00a0competitive<\/strong>\u00a0areas;<\/li>\n\n\n\n
    2. ask what the role of regulation is, and at what\u00a0granularity<\/strong><\/li>\n\n\n\n
    3. query how things can be\u00a0implemented<\/strong>\u00a0in a way that humans (and machines) can and want to\u00a0adopt<\/strong><\/li>\n\n\n\n
    4. ensure those governing, operating and participating can be\u00a0helped<\/strong>\u00a0in their\u00a0assessments<\/strong>\u00a0of compatibility and interoperability<\/li>\n\n\n\n
    5. work out how to\u00a0prioritise<\/strong>\u00a0and create clear (and stable)\u00a0roadmaps<\/strong>\u00a0that enable investment to be made<\/li>\n<\/ol>\n\n\n\n

      I believe that we can build on the success of sector-specific approaches<\/strong>:<\/p>\n\n\n\n

      The Open Banking model\u2019s success lies precisely in its sector-specific, incremental approach. It focused on the unique needs of banking, where security, standardisation, and customer trust were paramount.<\/p>\n\n\n\n

      Copying the Open Banking model into other sectors (e.g., energy, transport, health) isn\u2019t about duplication but about applying tested frameworks tailored to each industry<\/strong>.<\/p>\n\n\n\n

      \n

      Without considering the market incentives for participation we risk techno-utopian thinking<\/p>\n<\/blockquote>\n\n\n\n

      Instead of trying to create a complex, overarching architecture that risks being too broad, a sectoral approach allows for flexibility and adaptation to industry-specific challenges. This carries a risk of fragmentation, and equally a potential to mandate cross-sector interoperability<\/strong> without defining exactly what this means in detail for everyone. The risk of push back on over-reach is material and catastrophic (in terms of adoption, if it fails). Further, market-incentives will be market-specific.<\/p>\n\n\n\n

      Avoiding Over-Standardisation<\/h2>\n\n\n\n

      National Data Architecture risks over-standardisation and could stifle innovation instead of enhancing it. Different sectors have wildly varying levels of data maturity, hugely variable data governance needs and challenges, and a one-size-fits-all approach is highly likely to impose unnecessary constraints on sectors that are not as ready for them.<\/p>\n\n\n\n

      For example, imposing the level of data governance on the industrial sector that is required in the financial sector is (today) not realistic. We are, however, on the frontline of that journey with ESG reporting today, and it highlights the complexity of that real-financial sector bridging. Incremental improvements, like extending Open Banking principles to energy or transport can allow industries to evolve organically while keeping governance controls and security as a priority.<\/p>\n\n\n\n

      Pragmatism of the Incremental Approach<\/h2>\n\n\n\n

      The incremental, domain-by-domain approach may seem less ambitious, but it offers a practical pathway to innovation that can be adopted today<\/strong>. Trying to design a top-down, comprehensive national architecture from the outset risks paralysis by analysis and over-planning. Learning from Open Banking and rolling out smart data initiatives sector-by-sector allows for continuous improvement and responsiveness to real-world feedback, which may be more effective than a sweeping, coordinated architecture.<\/p>\n\n\n\n

      Interoperability Requires Industry Engagement<\/h2>\n\n\n\n

      Interoperability relies on industry engagement. This is best fostered through practical, sector-specific initiatives like Open Banking. Imposing an overarching system from above risks of alienating key industry stakeholders, who may resist changes that threaten their business models (this happens in every sector already, today). Open Banking\u2019s success came from the balance<\/strong> of regulatory oversight and industry cooperation, a formula that can be replicated in other sectors.<\/p>\n\n\n\n

      Market Failures Can Be Addressed Incrementally<\/h2>\n\n\n\n

      There are (always) risks of market failures such as data hoarding (castle & moat is still the prevalent investment model for data businesses). These issues should be tackled incrementally through sector-specific mandates rather than a top-down architecture, but can be accelerated with top-down principles.<\/p>\n\n\n\n

      The competition fostered by Open Banking and similar initiatives already shows promise in tackling data monopolies. An incremental, practical approach allows policymakers to address market failures as they emerge<\/strong>, rather than trying to solve all potential problems with a one-time top-down design.<\/p>\n\n\n\n

      Trust is Earned, Not Imposed<\/h2>\n\n\n\n

      Trust in data systems cannot be imposed from the top. While Open Banking was catalysed by regulation, it earned user trust through transparency, security, and gradual adoption, not through a top-down imposition. Similarly, rolling out frameworks sector by sector allows consumers, businesses and citizens to see the benefits and gradually gain confidence in the approach. Attempting to \u2018mandate trust\u2019 top-down could backfire, especially in sectors like health, where data privacy concerns are particularly sensitive.<\/p>\n\n\n\n

      At IB1, our Trust Frameworks<\/a> are one tool (of many) that can help foster multilateral collaboration, building on Open Banking principles.<\/p>\n\n\n\n

      Decentralisation Encourages Innovation<\/h2>\n\n\n\n

      The decentralised, minimised<\/strong> and pre-competitive<\/strong> nature of Open Banking has led to its adoption and innovation. A large, centralised approach would have risked locking in certain standards or technological pathways that could hinder future innovations. Allowing sectors to develop their own interoperable standards fosters a more competitive, flexible landscape, where innovations in one sector can inform developments in another, without being dictated by a single framework.<\/p>\n\n\n\n

      \n

      Data governance should aim to balance impact on rights and security while minimising its own footprint and reach<\/p>\n<\/blockquote>\n\n\n\n

      While the idea of a unified, overarching data framework may sound appealing in theory, in practice it would risk being too rigid, slow to adapt, and detached from the unique needs of individual sectors. A more pragmatic approach \u2014 building on the lessons of Open Banking and extending these principles to other industries \u2014 may offer the best balance of innovation, security, and user control.<\/p>\n\n\n\n

      Please feel free to comment, or message me (via LinkedIn<\/a> or directly)<\/p>\n\n\n\n

      Disclosure<\/strong>: I sat on the MiData energy sector board (mostly tearing my hair out on these points); co-chaired the creation of the Open Banking Standard<\/a> (through which I learned so many lessons it could fill a book or two); was founding CEO of the Open Data Institute<\/a>; am co-Chair of the Smart Data Council, and run a non-profit (IB1.org<\/a>) working on data governance at sector and national scale.<\/p>\n\n\n\n

      Thanks to Chris, Frank, Paul, Hadley and others for their feedback and inputs.<\/p>\n\n\n\n

      Useful links<\/strong><\/p>\n\n\n\n

      Lessons learned from Gaia-X Data Spaces https:\/\/www.sitra.fi\/en\/articles\/eight-lessons-from-building-data-spaces\/<\/a><\/p>\n\n\n\n

      ODI on Data Institutions
      https:\/\/theodi.org\/insights\/projects\/rd-data-institutions\/<\/a><\/p>\n\n\n\n

      Icebreaker One on how it implements data sharing
      https:\/\/ib1.org\/what-we-do<\/a><\/p>\n\n\n\n

      Open Banking (implementation entity)
      https:\/\/openbanking.org.uk<\/a><\/p>\n\n\n\n

      Additional narrative on cities
      https:\/\/agentgav.medium.com\/the-porous-city-92ae986cd43c<\/a><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

      (please note my disclosures at the end of this post) This post contains (as-always) personal opinions and thinking-in-progress (strong opinions, weakly held) as I navigate […]<\/p>\n","protected":false},"author":1,"featured_media":7543,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[5,35,29,38],"tags":[],"class_list":["post-7542","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-business","category-culture","category-data","category-government"],"jetpack_featured_media_url":"https:\/\/dgen.net\/0\/wp-content\/uploads\/2025\/04\/1_TRAvxG7V_q3WD4P_5YoHXw.webp","jetpack_shortlink":"https:\/\/wp.me\/pfJFK3-1XE","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/posts\/7542","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/comments?post=7542"}],"version-history":[{"count":1,"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/posts\/7542\/revisions"}],"predecessor-version":[{"id":7544,"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/posts\/7542\/revisions\/7544"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/media\/7543"}],"wp:attachment":[{"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/media?parent=7542"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/categories?post=7542"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dgen.net\/0\/wp-json\/wp\/v2\/tags?post=7542"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}