Your Reading List & recommendations

Big Data is as much about the people you hire as the data itself

By Gavin Meggs, Director of Data, Insight and Analytics at O2.

In our previous blogs in this Big Data series we’ve looked at defining the value of your Big Data, developing an appropriate data strategy, and the impact that AI has had on maximising insight. Today we’ll consider what most organisations now appreciate as their most important asset – people.

Businesses are facing the realisation that data-related skills are in short supply. What began as a shortage of qualified Data Scientists has extended, according to research by the International Data Corporation (IDC), to include data architects, modellers and experts in data management. The situation is only expected to become more critical.

The reason is simple. There is more data being created than anyone could possibly have predicted as recently as five years ago. Data is now created by a wealth of devices we use day-to-day, from the technology we wear on our wrists, the smart devices in our homes and offices, to our online shopping habits, social platform usage and our browsing habits across smartphones, PCs and tablets.

The rise of the Chief Data Officer (CDO)

If you have found it hard to recruit data specialists, it’s worth asking the question: Who really needs to manage your Big Data? In your organisation, is this an IT role or a business role? To address the shortage, there has been a marked rise in the number of Chief Data Officer (CDO) roles, whose emphasis has been on business operational experience, rather than a traditional IT background.

Instead of focusing on the data itself, organisations are hiring people who understand the challenges the business is facing and how data can be used to address them. But how long will it be before the skill sets needed for a good CDO will be just as difficult to find? Some are already reporting that the skill sets needed are complex, and that the blend of technical capability, business management and problem solving hasn’t typically existed before.

Internal recruitment and retention

With many data skills in short supply, businesses are increasingly looking inside their own organisation for malleable talent, and providing training to internal candidates that have the right combination of computing and business skills.

Companies are prioritising the development of several key skills:

  • Basic programming skills
  • Basic statistical skills
  • Machine learning methodology
  • Data visualisation, (e.g. graphs and charts) and communication skills

Taking someone who already has a clear understanding of the current business needs, and then enhancing their understanding of how data can be used to meet them, has clear benefits. It enables structured business needs to be articulated, which can result in more efficient problem solving. But who the CDO reports to says a lot about how critical the CDO role is becoming, and we are starting to see many report direct to the CEO.

Incentives and benefits

Whilst salary is important, organisations are starting to also appreciate that the current generation of skilled workers have personal motivations that go beyond simply the money they earn. They are motivated by the technology they are using, the difference they feel they can make to an organisation or cause, the extent to which they feel they are learning and developing, and how valued they feel by their employer.

Accordingly, businesses are addressing the skills gap by offering a broad range of incentives and benefits including:

  • Flexible working hours
  • Working from home
  • Paid and unpaid leave
  • Training and development opportunities
  • Secondment opportunities
  • Memberships and subscriptions
  • Discounts and other perks

This illustrates a shift in the way that HR will need to look for new talent, discarding some traditional recruitment methods and considerations like shortlisting purely on the strength of a candidate’s CV. Organisations are starting to appreciate that they must fit around the individual rather than force the individual to fit in to the organisation.

Ironically many HR decisions are themselves relying on big data sets, AI and analytics to “interview” candidates for roles across the business. Video interviews use analytics to understand how truthful an answer might be, how much a candidate believes in the convictions and experience they are discussing, or how much interest they truly have in the role. Big Data and AI might not be taking over our jobs but they are being used to help decide who should be doing them.


Organisations often fail to address the issue of responsibility for how data is to be used, or ask the questions whose answers add true insight and value to an organisation. Big Data projects that lack senior sponsorship, and are driven by IT rather than an operational lead will tend to be far less successful than one with appropriate roles defined across the business.

Recruiting data scientists is not always the right answer either, particularly without a comprehensive brief about the problem you want to resolve. Responsibility for Big Data transcends an organisation, and there are four principles that you should adopt:

  1. Big Data projects must include both IT and business stakeholders.
  2. You must have clear lines of responsibility for managing your data, implementing your organisation’s data framework and for remaining fully compliant.
  3. You should enforce your data processes with technology, but not at the expense of the people you employ to manage them.
  4. You should educate all your employees on the importance of your data, particularly in relation to regulation and compliance.


As data grows, so does the demand for Big Data analytics to gain insight into the data. In recent years, big data analytics has turned to machine learning and artificial intelligence for assistance.

We’re also seeing natural language processing becoming an essential technology for many business functions, from chatbots and digital assistants to compliance monitoring and business analytics. However we are a long way from using technology in preference to people. These emerging technologies still need people, and are not destined to solve the skills shortage that is affecting so many big data enterprises right now.

As well as platforms that ensure secure communications to assist flexible working and data management, O2 can also help with staff incentives. For example, we provide Priority and O2 Open to our business and public sector organisations. Both provide a simple yet effective way for companies to offer their employees discounted tariffs for friends and family, as well as Priority Tickets to thousands of gigs and events across the UK, up to 48 hours before general sale, as well as offers and exclusive deals not available anywhere else. Contact us to find out more.

Do you have data skills gaps? What are you doing to address them? We’d love to hear from you. Get in touch.

All articles


Public sector

Safe & secure


Tech advice

Work smarter