Context is Everything

I attend between 8-12 customer meetings every week. I travel across Europe and I meet mostly with large FTSE 100, global Fortune 500 brands and high growth startups. This affords me pretty good insight on the hot topics of tech. 

So what's the craic?

Everyone and anyone operating a business or organisation embracing the digital opportunity recognises that data is the currency of the digital age. People are quite literally obsessed with it- big data, small data, fast data, slow data, my data, their data, private data, public data, data, data. 

Having said this, whilst I think the focus on data is entirely warranted I worry that the industry is already littered with customers who have jumped onboard the data hype wagon, invested in 'big data' solutions (usually Hadoop) and really struggled to generate any value from their data. 

Why?.....
It is simply because Data Science can get extremely complicated- if you let it.

Building the infrastructure to do the data science is the easy bit. Quite literally, anyone can stand up a Hadoop cluster and fill a data lake. For the most part, this seems to be what the 'Big Data' industry has been up to over the last few years. Standing up -full- but fruitless data lakes that customers struggle to question and augment to their advantage. 



Most data science endeavours start out as skunkwork projects built on shoestring budgets by extremely resourceful and talented groups of people who have been forced to think differently - largely because of the significant business and technical challenges facing them. There are lots of cool use case examples and stories of how big data has transformed business models and disrupted industries- none cooler than where the technology itself (Hadoop) came from (Google and Yahoo!).

It's these grand stories that often lead to IT professionals and leaders to overcomplicate their own forays into the big data arena by targeting equally grand use cases that quite often fail to return the value they expected. Moreover, organisationally, traditional IT leaders view Big Data platforms as a centralised capability (much like data warehouses) that is surrounded by a few egg-heads that sit around all day dreaming up algorithms to run against piles and piles of data.

Trust your people - they are assets not just risks..

Big data can actually be used for a variety of pretty mundane things that still add business value. Data itself is meaningless without context and context only comes from the person looking at the data. If you limit who can see your data within your organisation, you also limit the context that can be applied to it and therefore the value that can be extracted. 

Too often is the focus on data sources and algorithms. These are important elements of any big data project but arguably it is the use cases that are more important. You can’t just dream those up - well you can, but you have to be extremely lucky to anticipate the needs of everyone in your organisation. As a first step, think about how you might go about surfacing the data your organisation collects to the people that want to use it. I am not suggesting you give everyone access to your production databases but nothing is stopping you from putting an API in front of your data lake and allowing your people to pull the data out and into their own data sandpit. 

Imagine if everyone in your organisation was given a data sandpit in which they could ingest the data streams that interest them, integrate those data streams onto a common timeline, do some analysis, visualise, search and share their findings? Wouldn’t that be cool? Imagine the use cases that would arise… Imagine the requests for additional capability… 

Rather than locking your data up and leaving it to few smart people to guess what questions your employees might want to ask, set it free, give them some basic self-service tools and create a data democracy & culture. What’s the worst that can happen?

For all of you data huggers and security doomongers - I mentioned the word API earlier in this blog. This means you can control what data people have access to - it doesn’t have to be ALL of it.