Big Data/BI Zone is brought to you in partnership with:

Treasure Data's Big Data as-a-Service cloud platform enables data-driven businesses to focus their precious development resources on their applications, not on mundane, time-consuming integration and operational tasks. Our pre-built, multi-tenancy cloud platform is already in use by over 50 customers worldwide and is managing more than 200 billion rows of data and processing 130,000 jobs per day. Discover how Treasure Data can help you focus on your core business and benefit from the fastest time-to-answer service available. Sadayuki is a DZone MVB and is not an employee of DZone and has posted 27 posts at DZone. You can read more from them at their website. View Full User Profile

Data Scientist: A Unicorn?

08.18.2012
| 4848 views |
  • submit to reddit

Finding a good engineer is hard. Finding a good data scientist doubly so.

A couple of months ago, Josh Wills, Director of Data Science at Cloudera, gave a talk dubbed “The Life of a Data Scientist”. In the talk, he defined data scientist as:

Data Scientist (n.): Person who is better at statistics than any software engineer and better at software engineering than any statistician. [1]

This definition gets to the heart of why it is so hard to hire a good data scientist. How many software engineers do you know that understand what Student’s t-test means? How many statistician do you know who has heard of Dependency Injection? To be honest, I know a couple, but that’s a couple out of 100+ software engineers and statisticians that I know. [2] The intersection of two small groups, statisticians and qualified software engineers, ends up being tiny.

The rest of the world is catching onto this supply-demand gap of data scientists. Research published by McKinsey Global Institute on Big Data reports:

Addressing the talent shortage will not happen overnight, and the search for deep analytical talent that has already begun can only intensify. [3][4]

If you are a data scientist, this is one great time to be one. Also, if you happen to be a great software engineer or a stastistician, you know what you should be learning next =)

P.S. If you happen to be a good statistician, software engineer or both (i.e. data scientist), we are hiring. Here at Treasure Data, we are building a platform to bring the power of Hadoop to the masses. If this sounds like your cup of tea, please drop us a line.

Published at DZone with permission of Sadayuki Furuhashi, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Mark Unknown replied on Mon, 2012/08/20 - 9:41am

I'd have to agree. 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.