Photos by William Moreland, Virginia Long, Cara Fuller and Kreated Media on Unsplash.
When a company decides to become data-driven, it still won’t happen overnight. Including data science and machine learning in the day-to-day processes is rewarding for sure — but it’s challenging, too. In this article, I’ll outline four typical stages that I see real-life businesses go through.
And just for fun, let’s compare these stages to animals…
So read through and find out what stage your company is in:
- LEVEL 1: Chicken,
- LEVEL 2: Donkey,
- LEVEL 3: Cheetah or
- LEVEL 4: Eagle?
Stage #1: Chicken (the wannabe data-driven)
While many companies realize the importance of data science (according to a 2019 New Vantage Partners survey), 77.1% of the surveyed executives reported that it remained a challenge for them.
That’s a huge proportion.
The truth is: it’s hard to get started with data science and analytics — simply because there are so many opportunities out there. Yeah, we all know buzzwords like “data is the new oil.” That might be true (or not). But it’s certainly not actionable.
The media attention on the fancy things doesn’t help, either.
Many resources say that AI, machine learning, deep learning, and big data will disrupt industries in the near future. But for a real-life business, maybe it’s worth focusing on simpler things first.
Either way, data science is a very broad field, and it’s hard to define what “data-driven” means in your business.
Many companies who want to be data-driven, in fact, just run around in pointless circles… like, you know, the chickens in the backyard.
So before getting started (and before building a data team, for instance), it’s essential to figure out what you want to do with your data at all. You can hire a senior data consultant for that, who’ll help map out the best opportunities. And once that preparation is done, only then it is worth it to start hiring people.
A typical example of this stage could be a startup company that successfully launched and grew over the first few years. For data-driven companies, a broad average is that around ~2-5% of a company is data/analytics-focused, so the first time one should think about hiring a full-time data professional is when the company hits ~20-30 people.
Stage #2: Donkey (Data-driven but with biases and errors — so it’s even worse)
Even when there are clear goals set regarding data projects, the company culture needs to adapt. And that takes time. First, your colleagues won’t know what data science is good for… Then they’ll learn, but they’ll misuse it. A data project can fail in many ways. But the two most common mistakes are:
- an error in data collection (so from false raw data, you will get false conclusions)
- a statistical bias when drawing conclusions (so the decisions are based on misinterpreted results)
I can’t emphasize enough how dangerous it is to make wrong decisions based on wrongly executed data projects.
Note: Well, I feel bad a little bit for the donkey in this metaphor… Donkeys are stubborn but nice. Decision-makers at this stage are just stubborn.
But both above-mentioned problems can be fixed in the long term — if you focus on enhancing the data-driven company culture. Again, it’ll take time.
Even so, during the adaptation process, you should make sure that the data collection processes are continuously reviewed and maintained by developers or data engineers. And that there’s a strong senior data science lead who oversees all data projects and data-related decisions at the company.
There are many examples for this stage… well, I’d rather not name names, not to hurt anyone’s feelings. But here’s the story from the article I linked above… It’s from the health industry:
A well-trained machine learning model based on health data decided to categorize asthma sufferers as low-risk patients and sent them home. (Asthmatics are actually at high risk of complications!) It was strange, and later it turned out that asthmatics were always sent to intensive care before… and that’s why they never died. But the model had only seen the correlation (and not the causation) — and mistakenly concluded that asthmatics were low risk. As the article says: “The model had great accuracy, but if deployed in production, it would certainly have killed people.”
I guess there’s no more to say about the dangerousness of this stage.
Stage #3: Cheetah (Truly data-driven)
After a long journey, you are finally there!
You are truly data-driven. A lot of research shows the quantified advantages of being data-driven. But there’s one that you can’t measure — and it’s probably more important than anything else. It’s this:
You know what you are doing and why.
Feeling secure about your decisions is a feeling that you can’t describe with numbers. You don’t have to copy your competitor anymore. You don’t have to worry about trying out every little new trend you read here and there. You’ll see the facts: what drives your results and what you should do to improve them.
You go fast, and you get what you want.
It’s a pretty nice place to be in. Again, not many companies have made it here to date.
Stage #4: Eagle (Beyond Data-driven…)
There’s one more stage you won’t read about many places. I call it beyond data-driven.
The “eagle” companies who are beyond data-driven are fast and strong — and see the big picture unlike anyone else.
Most importantly, they know that even quantitative and qualitative research have their limitations. They know that you can’t base everything on data: sometimes gut feelings and instincts have to play a role in decision making.
Stage #4 (beyond data-driven) seems similar to Stage #1 (wannabe data-driven) on the surface. In both, you’ll rely on instincts. But there is a big difference! At Stage #4, you’ll be data-informed. Even if you listen to your instincts, they will have a “built-in” data factor, too. So the decision at Stage #4 won’t be based on gut feelings only — it’ll be the right combination of data-driven and human-driven.
Just look at the good old days of Apple. Steve Jobs and his team knew their customers, for sure. They’d seen the data, and they knew what people wanted. But they dared to question the status quo, and they dared to go with their instincts, too. And so they released revolutionary products like the first iPhone. And we all know the results.
Note: Nowadays, Apple seems to have gone back to Stage #3 (data-driven), which is fine, too.
I hope this article helped you to see and understand the different stages and challenges of becoming data-driven. Regardless of where you are right now, try to focus on the next step. Start with simpler projects, enhance the data-driven culture at the company, listen to your data — but dare to listen to your instincts, too!
So are you a chicken, a donkey, a cheetah, or an eagle?
Bio: Tomi Mester creates in-depth, practical, true-to-life online tutorials (and video courses) to help people learn Data Science.