<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	
	xmlns:georss="http://www.georss.org/georss"
	xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#"
	>

<channel>
	<title>data</title>
	<atom:link href="https://dgen.net/0/tag/data/feed/" rel="self" type="application/rss+xml" />
	<link>https://dgen.net/0</link>
	<description>Designing our data infrastructure</description>
	<lastBuildDate>Mon, 07 Jul 2025 07:59:50 +0000</lastBuildDate>
	<language>en-GB</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=6.9.4</generator>

<image>
	<url>https://dgen.net/0/wp-content/uploads/2017/02/cropped-dgen-D-logo-2017-32x32.png</url>
	<title>data</title>
	<link>https://dgen.net/0</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">232530259</site>	<item>
		<title>Data is everywhere, just not where we need it</title>
		<link>https://dgen.net/0/2025/07/07/data-is-everywhere-just-not-where-we-need-it/</link>
		
		<dc:creator><![CDATA[Gavin]]></dc:creator>
		<pubDate>Mon, 07 Jul 2025 07:59:49 +0000</pubDate>
				<category><![CDATA[business]]></category>
		<category><![CDATA[data]]></category>
		<category><![CDATA[data infrastructure]]></category>
		<guid isPermaLink="false">https://dgen.net/0/?p=7669</guid>

					<description><![CDATA[<img width="1024" height="683" src="https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-1024x683.jpg" class="attachment-large size-large wp-post-image" alt="AI generated image of people searching for data across patchwork fields made of charts and spreadsheets" style="float:left; margin:0 15px 15px 0;" decoding="async" fetchpriority="high" srcset="https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-1024x683.jpg 1024w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-300x200.jpg 300w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-768x512.jpg 768w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-830x553.jpg 830w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-230x153.jpg 230w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-350x233.jpg 350w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-480x320.jpg 480w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni.jpg 1200w" sizes="(max-width: 1024px) 100vw, 1024px" />We’ve never had more data. Some think it’s the new oil, the new gold, the new soil, something of a revolution. Is it? Or are [&#8230;]]]></description>
										<content:encoded><![CDATA[<img width="1024" height="683" src="https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-1024x683.jpg" class="attachment-large size-large wp-post-image" alt="AI generated image of people searching for data across patchwork fields made of charts and spreadsheets" style="float:left; margin:0 15px 15px 0;" decoding="async" srcset="https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-1024x683.jpg 1024w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-300x200.jpg 300w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-768x512.jpg 768w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-830x553.jpg 830w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-230x153.jpg 230w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-350x233.jpg 350w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni-480x320.jpg 480w, https://dgen.net/0/wp-content/uploads/2025/07/blog-diejnwwni.jpg 1200w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p>We’ve never had more data. Some think it’s the new <a href="https://en.wikipedia.org/wiki/Clive_Humby" rel="noreferrer noopener" target="_blank">oil</a>, the new <a href="https://www.weforum.org/agenda/2011/11/data-is-the-new-gold" rel="noreferrer noopener" target="_blank">gold</a>, the new <a href="https://www.ted.com/talks/david_mccandless_the_beauty_of_data_visualization" rel="noreferrer noopener" target="_blank">soil</a>, something of a revolution. Is it? Or are we just still humans stumbling around in the dark?</p>



<p>Earth observation satellites scan our planet, sensors (there are more sensors on Earth than there are people) track the movements of people and things, energy flows, air quality, water usage, and behaviours. Our global financial systems process billions of transactions a day. Most organisations, public and private, are drowning in dashboards, APIs, spreadsheets, data lakes, clouds, compute, applications, and now ‘ai’.</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p>“Every age thinks it’s the modern age…but this one really is”<br><a href="https://vimeo.com/ondemand/dreamsrewiredmovie" rel="noreferrer noopener" target="_blank">Dreams Rewired</a>, Manu Luksch</p>
</blockquote>



<p>And yet, when we need to make decisions that matter — how to finance a green retrofit, where to send emergency flood support, or how to measure progress toward net zero, the data we need is often hard to access or nowhere to be found. Or it exists, but it’s in the wrong format, in the wrong place, scattered across organisations and systems, under the wrong licence, or wrapped around the wrong market incentives, or just poorly governed and hard to trust.</p>



<h4 class="wp-block-heading"><strong>Why is this important to our&nbsp;future?</strong></h4>



<p>We’re in an era where our <a href="https://en.wikipedia.org/wiki/Financial_economics" rel="noreferrer noopener" target="_blank">financial economy</a> is demanding ‘quality’ data from the <a href="https://en.wikipedia.org/wiki/Real_economy" rel="noreferrer noopener" target="_blank">real economy</a>. Our financial systems have developed over centuries to measure, track, verify, use and report on the rate of change of ‘one dimension’: money. As we continue the shift to make our environment measurable, we must get the ‘exchange rate’ right between financial investment and environmental outcomes (climate, nature, air quality, water, biodiversity, and so on).</p>



<p>Right now the ‘exchange rates’ are all over the place and, importantly, as we step forward we need to measure the rate of change of many parameters, not just one: we need a step-change in how we think about ‘data’ and data <em>sharing</em>.</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p>It’s hard to understate quite how far away the real economy is from generating financial-grade data</p>
</blockquote>



<p>We’re still mostly in the 1970s. Environmental disclosures still rely on highly manual processes, despite widespread digitisation and <a href="https://ib1.org/ecosystem/2024-carbon-reporting-solutions-report/" rel="noreferrer noopener" target="_blank">hundreds of application vendors</a> in the market, it’s still mostly clipboards and spreadsheets or, for 99% of the world, nothing at all. This is not a problem of data scarcity, it’s a problem of <em>incentives, structure</em>, and <em>alignment</em>.</p>



<p>We’ve built systems that are optimised for accumulation, not coordination. We have engineered data pipelines that extract value, but not the wiring to deliver reciprocity (shared benefits).</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p>In the language of infrastructure: we’ve laid the cables, but we’ve neglected the grid</p>
</blockquote>



<p>We are told to “make data-driven decisions” while simultaneously being denied the means to do so at scale. Meanwhile, we waste more time (and money, opportunity and emissions) trying to reconcile, reformat, clean, or even <em>find</em> the data than the actual decisions that we are trying to make.</p>



<p>The result is a kind of systemic gaslighting: we pretend that the data is ‘there’ when what we really mean is that it <em>exists&nbsp;… </em>but <em>existing</em> is not the same as <em>usable</em>, and very far away from <em>trusted</em>. Just because something is stored on a server doesn’t mean it’s infrastructure. Just because it’s digital doesn’t mean it’s useful. Just because it’s available doesn’t mean it’s accessible.</p>



<p>McKinsey (<a href="https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-data-driven-enterprise-of-2025" rel="noreferrer noopener" target="_blank"><em>The data-driven enterprise of 2025</em></a><em>) </em>suggests that data professionals still spend up to 80% of their time preparing data rather than using it. This is the same number I have from over 25 years ago.</p>



<h4 class="wp-block-heading">So where do we go from&nbsp;here?</h4>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p>Data is everywhere but until it <strong>flows to where it’s needed</strong> (when it’s needed, and with the right context) we’re not building a digital future we’re just building more digital noise</p>
</blockquote>



<p>We need to stop pretending that technology (including ai) alone will solve this. We must stop framing data as a ‘commodity’ to be owned and sweated, and start treating it as infrastructure to be governed.</p>



<p>We must design for connection not monopoly control, for shared purpose not private hoarding. We must invest in the ‘boring’ bits: governance, legal standards, licensing, permission/consent, metadata, interoperability.</p>



<p>This is the stuff that doesn’t make the headlines, but makes everything else work.</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">7669</post-id>	</item>
		<item>
		<title>What is a data infrastructure strategy?</title>
		<link>https://dgen.net/0/2023/01/06/what-is-a-data-infrastructure-strategy/</link>
		
		<dc:creator><![CDATA[Gavin]]></dc:creator>
		<pubDate>Fri, 06 Jan 2023 01:01:00 +0000</pubDate>
				<category><![CDATA[business]]></category>
		<category><![CDATA[Culture]]></category>
		<category><![CDATA[data]]></category>
		<category><![CDATA[culture]]></category>
		<category><![CDATA[data infrastructure]]></category>
		<category><![CDATA[strategy]]></category>
		<guid isPermaLink="false">https://dgen.net/0/?p=7558</guid>

					<description><![CDATA[<img width="1024" height="772" src="https://dgen.net/0/wp-content/uploads/2025/04/designing-di-1024x772.webp" class="attachment-large size-large wp-post-image" alt="" style="float:left; margin:0 15px 15px 0;" decoding="async" srcset="https://dgen.net/0/wp-content/uploads/2025/04/designing-di-1024x772.webp 1024w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-300x226.webp 300w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-768x579.webp 768w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-830x625.webp 830w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-230x173.webp 230w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-350x264.webp 350w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-480x362.webp 480w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di.webp 1400w" sizes="(max-width: 1024px) 100vw, 1024px" />I’m often asked to unpack ‘data’. There’s a lot of confusion and, often, people think it’s a function of ‘IT’ and just for the geeks. [&#8230;]]]></description>
										<content:encoded><![CDATA[<img width="1024" height="772" src="https://dgen.net/0/wp-content/uploads/2025/04/designing-di-1024x772.webp" class="attachment-large size-large wp-post-image" alt="" style="float:left; margin:0 15px 15px 0;" decoding="async" srcset="https://dgen.net/0/wp-content/uploads/2025/04/designing-di-1024x772.webp 1024w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-300x226.webp 300w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-768x579.webp 768w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-830x625.webp 830w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-230x173.webp 230w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-350x264.webp 350w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di-480x362.webp 480w, https://dgen.net/0/wp-content/uploads/2025/04/designing-di.webp 1400w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p><a href="https://agentgav.medium.com/?source=post_page---byline--c5ac73217d6f---------------------------------------"></a></p>



<p id="d29c">I’m often asked to unpack ‘data’. There’s a lot of confusion and, often, people think it’s a function of ‘IT’ and just for the geeks.</p>



<p id="a5c1">A ‘data strategy’ can often be quite shallow in its thinking and tends not to focus on data sharing—which is a cultural shift in the way collaboration needs to work.</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p id="e603">Collaboration in a data-enabled business needs to be about how humans and machines can combine their intelligence, systems, and accelerate and optimise solutions to problems. At the core of this is the ability to easily access, share and use systems internally and externally.</p>
</blockquote>



<p id="bdbb">Even if there’s an understanding that data should be used at a strategic level, it’s rare that there is a good understanding of data strategy at C-level, and often it’s deferred to CIO/CTO to make decisions that, in my view, should be more collaboratively defined by the whole team, aligned with business targets, and explicit contributions to ROI.</p>



<p id="c3a4">For example, I would like to think that the age of “build a data lake” or “let’s build a portal” as a&nbsp;<em>starting point</em>&nbsp;are over. However, my observation is that we are very far away from that.</p>



<p id="78ac">We need to begin by thinking of data as part of our&nbsp;<strong>infrastructure&nbsp;</strong>and, with this, thinking about how it can be&nbsp;<strong>shared</strong>&nbsp;between systems regardless of where those systems are.</p>



<p id="8911">The Open Data Institute (ODI), frames&nbsp;<strong>data infrastructure</strong>&nbsp;as the “systems, processes, and tools that enable the creation, management, and use of data.” This includes both physical infrastructure (e.g. servers and storage systems) and virtual infrastructure (e.g. software and networks).</p>



<p id="ed28">The definition goes further to include: the people, policies, and practices that support the creation, management, processing and usage of data; roles and responsibilities for managing data; the standards and protocols for how data should be collected, stored, and shared.</p>



<p id="f6dc">A strong data infrastructure is&nbsp;<strong>the foundation</strong>&nbsp;for data management, analytics, visualisation, and value creation. And, in a networked world, the systems for management, analysis, visualisation may exist anywhere.</p>



<p id="b204">A strong strategy must, therefore, address interoperability at its core.</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p id="897b">A&nbsp;<strong>data infrastructure strategy</strong>&nbsp;is a plan for building, managing, and maintaining systems, processes, and tools that enable data creation, management and use in a way that can be&nbsp;<strong>shared across systems</strong>.</p>
</blockquote>



<p id="08f3">It is important to understand the role of&nbsp;<strong>data sharing</strong>&nbsp;in supporting outcomes and impacts and, based on this, to create a framework for sharing data.</p>



<p id="b77b">A well-designed strategy can save time and money by rapidly connecting data and systems together: to help people make better, more informed decisions and use data to drive growth and innovation. It must take into account the types of data as well as the business processes (logistical, operational and legal), and understand the applications that rely on that data.</p>



<p id="77df">Critically, it must consider the organisation’s&nbsp;<strong>data governance</strong>&nbsp;policies and procedures, legal and regulatory requirements.</p>



<p id="b7de">In summary, four core elements of a robust data infrastructure strategy must include:</p>



<ol class="wp-block-list">
<li><strong>Data governance</strong>: policies, procedures, and standards (that can be enforced) to ensure the usability, quality, security, and integrity of data, its usage and sharing (internally and externally to any organisation).</li>



<li><strong>Data access and security</strong>: how data will be accessed and shared, both across organisations and externally, including protecting data from unauthorized access or misuse. This will combine technical and legal measures.</li>



<li><strong>Data analytics and visualization</strong>: understanding the tools used to analyse, process, visualise and share outputs, and the processes and skills needed to use them effectively.</li>



<li><strong>Data storage and management</strong>: where and how data will be stored, the tools and processes needed to manage data throughout its lifecycle, regardless of where the data ‘physcially’ resides. This will include time-based, jurisdictional and related policies.</li>
</ol>



<p id="365d">Note that only point four (and part of point two) of this are functions of ‘IT’. Points one and three span the entire organisation, as managers and as users.</p>



<p id="5d76">To focus attentiion, a&nbsp;<strong>data infrastructure strategy</strong>&nbsp;should directly support goals and outcomes, minimise risks, unlock effective usage and sharing to drive growth and innovation.</p>



<p id="1936">Finally, it would be prudent to begin to include ‘algorithms’ (as opposed to code) as part of strategic development as while in many cases it may be the data that is ‘moving to the processing’, it is also the case that the algorithms may have to ‘go where the data is’ for efficiency and/or compliance reasons. This may be the subject of a future post.</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">7558</post-id>	</item>
	</channel>
</rss>
