Democratisation of Data Science #35 #cong15

By Cronan McNamara

Cronan McNamara #35 Democratisation of Data Science


Data is one of our greatest raw materials and can hold the key to unlocking some our biggest problems. It can be the catalyst for great change in our society.

But … data is a prisoner, it is trapped in silos, restricted, controlled and under the thumb of data owners, often well intentioned, but sometimes with no vision of how to use it.

Data is the new oil – a lubricant of business, an ingredient of everyday life and a potentially explosive agent of change. Oil when liberated in democratised societies has fuelled powerful economic growth and propelled societies to ever greater levels of innovation and civilisation.

However oil is a limited, shrinking resource while data is limitless and exploding in supply. 

In ways we are at the early stages of the data revolution – we have discovered its presence, acknowledge its value and are now starting to uncover new ways of extracting value from it.

Also similar to oil the elite few who really understand it are starting to control its flow and although the majority have it, they cannot make best use of it and the uncoordinated efforts by a few disparate groups without collaboration makes little impact.

For our world to truly grow, data needs to be liberated.  It needs to be democratised. 

Cronan McNamara #35 Democratising

The reality is that change has already gone beyond the point of no return and global levers are benefiting from and impacting on data in new and exciting ways. 

In areas as diverse as marketing, risk analysis, weather forecasting, fraud detection, food safety, nutrition and dietetics, data is being used to provide insights to businesses and consumers to help them to make better decisions. 

We see and experience the impact of data and digital services every day:

  • Globalisation
  • Mobile Computing
  • Sharing Economy
  • Social Media
  • Crowdsourcing
  • Open source software

All of these are fueled by data and give us a glimpse of the power of this raw material.

As a society, we are literally drowning in data. We only scratching the surface with less than 4% of data that gets generated ever being analysed.

Cronan McNamara #35 Democratisation of Data Science3

And this is the key point.  Having data is meaningless until it analysed.  Similar to being surrounded by a thick black sticky oil it adds nothing until it is refined and turned into a value added substance.

This is where the movement to democratising of data gets difficult. We have some tricky challenges ahead of us, of these are a few:

  • Although, there is a huge growth in data companies, data grows faster than we can interpret it and storage capacity is only keeping slightly ahead.
  • There are data analytics software tools, but these are specialised and only useful to those with the right skills (data scientists).
  • We face a real headache with a shortage of data science expertise which can harness this new powerhouse.
  • We need to govern it in an ethical way and build trust between data owners and analysts in order to fulfill its potential.

From where we stand, we see a world of many possibilities where ultimately data becomes monopolised by a few mega data companies. The result of this is that the many good businesses that should be able to benefit from data remain stymied and the wealth of individual data is stolen from citizens.

My vision is different.  I know the power of data and properly used, I believe it holds the key to ensuring better decisions are made and individuals and companies can prosper.

In order to democratise data science, we need to make a connection between analytics experts, data sets and decision makers - an online community sharing mathematical algorithms and raw data, combining it with expert industry insights and reformating it to predict future outcomes and risks. 

  • The true power of data is not just drawing a picture of the past and interpreting the present but in looking into the future.
  • No one person or data set holds all the keys or answers so I believe we need a platform on advanced predictive analysis that brings together a community of data modellers, data owners and business to solve big problems.
  • We need to remove the complexity of coding for individual models by using a series of user friendly building block so generating models rapidly is possible.
  • We need to be able to import very large hosted and stores data sets – both open and proprietary into analytical models.
  • We need to create an ecosystem which allows problem solving where people can build models and crunch data in minutes rather than weeks, but more importantly those without all of the required expertise can solve data problems through collaboration with experts online.

When studying the democratising process, it becomes clear that commerce is a powerful force, so a marketplace is essential to reward data owners and technologists.  This can act as an engine of innovation in data science and a catalyst for rapid problem solving. I see this commercially beneficial relationships with two groups:

  • Those who have expertise in data science and / or who have built up valuable data sets can monetise these while the expert data modelers can build, share and curate models for financial gain. What this means is that businesses who lack these capabilities can immediately benefit from the network.
  • An ecosystem where free and commercially available data sets and models are made widely available where individuals, businesses and governments can unleash the power of seeing new futures and solve big problems.
  • A system developed to the point where technical complexity is not a barrier and a world where collaboration will create wealth, know how and a better future.

We are in the first wave of this democratising process and without some guidance, governance and structure could replicate the mistakes of the past. 

Rather than just setting out a future gazing vision we have built a platform called Expert Models. I will introduce the platform at Congregation. 

At the event, I would love to hear about your data sets and challenges and to get you involved in this movement. Together we can move predictive analytics from its powerful but narrow deployment to truly enhancing a better future for us all.


CongRegation © Eoin Kennedy 2017 eoin at congregation dot ie