The shortage of qualified data scientists is often highlighted as one of the major handbrakes on the adoption of big data and AI. But a growing number of tools are putting these capabilities in the hands of non-experts, for better and for worse.
There’s been an explosion in the breadth and quality of self-service analytics platforms in recent years, which let non-technical employees tap the huge amounts of data businesses are sitting on. They typically let users carry out simple, day-to-day analytic tasks—like creating reports or building data visualizations—rather than having to rely on the company’s data specialists.
Gartner recently predicted that workers using self-service analytics will output more analysis than professional data scientists. Given the perennial shortage of data specialists and the huge salaries they command these days, that’s probably music to the ears of most C-suite executives.
And increasingly, it’s not just simple analytic tasks that are being made more accessible. Driven in particular by large cloud computing providers like Amazon, Google, and Microsoft, there are a growing number of tools to help beginners start to build their own machine learning models.
These tools provide pre-built algorithms and intuitive interfaces that make it easy for someone with little experience to get started. They are aimed at developers rather than the everyday business users who use simpler self-service analytics platforms, but they mean it’s no longer necessary to have a PhD in advanced statistics to get started.
Most recently, Google released a service called Cloud AutoML that actually uses machine learning itself to automate the complex process of building and tweaking a deep neural network for image recognition.
They aren’t the only ones automating machine learning. Boston-based DataRobot lets users upload their data, highlight their target variables, and the system then automatically builds hundreds of models based on the platform’s collection of hundreds of open-source machine learning algorithms. The user can then choose from the best performing models and use it to analyze future data.
For the more adventurous developers, there are a growing number of open-source machine learning libraries that provide the basic sub-components needed to craft custom algorithms. (Read More...)