What is causality? And how do I get it? This, and many other answers, are posted beautifully here.
The jump from data to decision means aiming to answer the (general) question of “what causes what”. This question of “causality” is answered best by a well-rounded team including analysts and researchers.
What happens when statistical skills are missing?
So, what happens if you forget the “statistics” part in the Data Science balance? The above linked blog cuts the hype to restore balance between data-technology and statistics. At statistics school, they teach design of experiments as the golden rule to causality. And as a counter-measure to false learning from too “data-centric” learning. These skills are available in form of basic statistical skills such as statistical tests, Simpsons paradox and more “tricky” concepts like Power.
Interpreters of findings can fill the gap
Proficiency with both IT and Statistics as well as business skills are needed to be successful, and learning all this takes lots of time. If the only analysts available are computer science majors (who usually are mostly interested in clever programming), who else is on the scene to help decision makers/managers/CxO:s understand what causes what? “Interpreters” are needed, they must not be expert statisticians, programmers, or business experts, but be somewhere in between to communicate findings.
Some final remarks
While Data Scientists might be catching up on causality, there is a whole host of professional veterans available. Most notably in the life sciences, way ahead for decades in finding fact-based correlations that hold over time, over geographies, over ethnicity, and in many other different contexts. We can dig into their treasure chest at our own leisure to learn their methods.
I remain fascinated (and slightly grumpy-fied) by how the constant reinventing of the “hottest profession in the 21st century” (Data Science) ends up discovering/reinventing age-old questions (and solutions) of basic research. Let’s begin from “standing on the shoulders of giants” before charging into the hottest future on the market!