Your First Machine Learning Project

Written by Kopis 04/13/18


Machine Learning has eclipsed Big Data as the hot buzzword in Business Intelligence as of late. There are a number of tools available now that make it easier than ever to train, test, and deploy machine learning models for business tasks. These platforms offer an exciting step forward for businesses not yet at the machine learning level of BI maturity.

My own experience both teaching analytics and working with companies at varying levels of BI maturity requires me to offer a word of caution here: don’t fall into the trap of believing a technology package will be a silver bullet. These tools are amazing, to be sure, but they won’t work for you unless you’ve defined their purpose first. Don’t invest in an enterprise-grade machine learning solution without knowing ahead of time what you expect to feed into it and get out of it.

If you’re wondering exactly what you should expect to get out of it, think of two practical examples.

Predicting critical system errors
At Kopis, we have amassed a lot of data with our proprietary systems monitoring platform, Vigilix. The Vigilix agent running on client machines logs critical system events and errors, and over time that equals a lot of data points generated. What good are these terabytes worth of data? By applying machine learning algorithms to the available data, we are able to model what particular system configurations or string of events may precipitate a critical system error. That model can be applied to future events and generate predictive alerts for critical errors before they happen. More data means better training and testing data for model optimization.

Retaining your employees
Imagine this: a multinational manufacturing firm with thousands of employees worldwide faces an unusually high employee attrition rate every year, but their HR department can’t make sense of it. They have terabytes of data from different internal systems at their disposal…wouldn’t it be great if they could leverage all that internal data to figure out what factors run common among those employees who have left the company? I was part of a team earlier in my career that did just that. The end result was a regular weekly report that listed the employees most likely to leave the company in the next 6 months based on the models we developed, trained, and tested.

Ultimately, the role of machine learning in your organization depends on your business goals and company strategy. It’s not something you do just because other companies are doing it. Of course, with any BI initiative, there are plenty of “unknown unknowns” that are uncovered as projects progress—but you should have clear initial goals for what you want machine learning to improve in your business.

So, where to start? There are two recommended routes here: Amazon AWS and Microsoft Azure. While we won’t examine setup for these platforms, it’s important to know there are multiple options in both these cases, and which one you go with largely depends on what you already have in place in your enterprise environment. Whichever platform you choose, Kopis can help you navigate the decision and get the most out of your machine learning project.