Data Scientist: A Unicorn?

Finding a good engineer is hard. Finding a good data scientist doubly so.

A couple of months ago, Josh Wills, Director of Data Science at Cloudera, gave a talk dubbed “The Life of a Data Scientist”. In the talk, he defined data scientist as:

Data Scientist (n.): Person who is better at statistics than any software engineer and better at software engineering than any statistician. [1]

This definition gets to the heart of why it is so hard to hire a good data scientist. How many software engineers do you know that understand what Student’s t-test means? How many statistician do you know who has heard of Dependency Injection? To be honest, I know a couple, but that’s a couple out of 100+ software engineers and statisticians that I know. [2] The intersection of two small groups, statisticians and qualified software engineers, ends up being tiny.

The rest of the world is catching onto this supply-demand gap of data scientists. Research published by McKinsey Global Institute on Big Data reports:

Addressing the talent shortage will not happen overnight, and the search for deep analytical talent that has already begun can only intensify. [3][4]

If you are a data scientist, this is one great time to be one. Also, if you happen to be a great software engineer or a stastistician, you know what you should be learning next =)

P.S. If you happen to be a good statistician, software engineer or both (i.e. data scientist), we are hiring. Here at Treasure Data, we are building a platform to bring the power of Hadoop to the masses. If this sounds like your cup of tea, please drop us a line.