Data scientist: Most 'in demand' job of the century?

Thomas Davenport, Harvard Business School professor, says demand for data scientists is 'just insatiable'.

Comments

“Pretty short-term tenures are common, so if you want to hang onto them, you need to do all the usual things: pay them well, give them stock options, and interesting work. They do not like to be micromanaged — give them a ‘long leash’.” Thomas Davenport, Harvard Business School professor, is talking about what he calls the “most in-demand job” of the century: the data scientist.

Two months ago, a Harvard Business Review article he co-wrote with DJ Patil grabbed a lot of attention for stating that data scientist is “the sexiest job of the 21st century”.

He credits the HBR editors for the title, but one thing undisputed is that the hiring curve for quantitative analysts and data scientists “is almost totally vertical.”

“The demand is just insatiable,” says Davenport. “I suppose that makes them sexy.”

He admits he was initially sceptical about “big data” and thought it was just analytics under another name. “When big data came along everybody there was a lot of what I call Hadoopaloosa,” he says alluding to the open source technology Hadoop. “Big data technologies are open source and pretty widely available, but what I think is really hard to find and unique are the data scientists.”

Davenport says one of the most interesting aspect of his research on analytics was the people who do the work and how to organise and motivate them. “It is even harder with data scientists. They have quite esoteric education backgrounds. Many of them are scientists, physicists, and biologists. They need to be able to manage data, do quantitative analysis, and explain it all to executives and decision makers.”

Today, the roles are taken by people with science-oriented PhDs. They have also done lot of computation and quantitative analysis in graduate school. Network-oriented big data environments like Facebook and LinkedIn, meanwhile, are also interested in sociologists, anthropologists, and psychologists.

“You need that combination of analytical skills and data management skills, the ability to pull data out of unconventional sources, and then convert unstructured data and then to analyse it and interface it,” he explains.

He elucidates that when he asked several companies who recruited data scientists about the one absolute trait they should have, it is the ability to write code.

One thing data scientists and traditional analysts have in common is the need to interact effectively with decision makers or as Davenport puts it, “be able to tell a story with data.”

They also have to be effective communicators, notes Davenport. “The relationship orientation is important in both traditional analysts and data scientists, but it is with different types of people.”

Data scientists are much more likely to work in product and service development groups, with CTOs, and with customers. Traditional analysts tend to work with internal decisions makers and executives within the company who are trying to use the analytics to make better decisions.

Davenport believes the shortage for data scientists will be a “relatively short-term,” because universities will have programmes in data science in place in the next few years.

“You don’t necessarily have to aim for a PhD in experimental physics, because I suspect within four or five years we will have data science programs within universities; and master’s degrees, rather than undergraduate degrees,” he says.

So those who would like to pursue this field can have an undergraduate degree in a data oriented computationally oriented mathematically oriented field and then get a master’s degree in data science or big data.

“Then you will be all set for a great career.”

The data scientists will not necessarily be in IT, but will have a relationship with the data management and the business intelligence/analytics people in the industry.

He says it would be tough to retain them, because of the range of job opportunities for them. “They are paid very well, they are very impatient, and they want to make a big impact quickly. And if they can’t, they move on.”

Davenport had earlier interviewed one such data scientist, Jonathan Goldman, for the HBR article. The article came out in October and he says Goldman, who had created the ‘people you may know’ application in LinkedIn, has already moved to another company.

Goldman had initially shown the application to the LinkedIn product team, who said they already have an address book look up capability. So Goldman told Reid Hoffman, LinkedIn co-founder, who had hired him, that he was getting a push back from the product people.

Hoffman let him have a little ad space on the homepage and the app became “fantastically successful.”

Davenport stresses that had Goldman not have had that relationship with Hoffman, it would not have happened.

“I hope a number of universities around the world can develop programmes that churn these people out at a high rate, because we need them if big data is going to be successful.”

Divina Paredes interviewed Thomas Davenport at the Premier Business Leadership Series in Las Vegas which she attended as a guest of SAS.