In the early days of computing, developers were often jacks of all trades, handling virtually any task needed for software to get made. As the field matured, jobs grew more specialized. Today, we’re seeing a similar pattern in a brand-new domain: Big data.
But it’s not just us saying this. According to P.K. Agarwal, regional dean and CEO of Northeastern University’s recently formed Silicon Valley campus, says that so far, big-data professionals have commonly handled everything from data cleaning to analytics, and from Hadoop to Apache Spark; they’ve handled it all.
But studying big data is becoming more and more like studying medicine…you start to develop specialties.
This brings us to today’s data-scientist shortage. Highly trained data scientists are now in acute demand as organizations awash in data look for insight in all those petabytes of data. As a response to this, other professionals are learning the skills, attempting to answer at least some of those questions for themselves, earning the informal title of “citizen data scientist.”
This has resulted in a multipronged approach, with both tools and education.
“It used to be that anybody who was in the business world needed to learn PowerPoint and Excel,” Agarwal said. “Probably five years from now, Microsoft Office will have something combining Excel, R and Tableau. It’s just the natural progression. If there’s this new class of citizen data analysts, they’re going to need new tools.”
Northeastern offers eight-week “bootcamps” in data analytics aimed at a broad spectrum of business people, as well as more intensive certificate programs for professional data scientists. Specialized master’s degree programs are also in the works.
We are guessing it will take at least a few more years to even out the shortage, meanwhile tools will need to become much more sophisticated. Another thing that will have to improve? Automation.
Read the full article here.