There is a huge amount of excitement around Artificial Intelligence (AI) right now and understandably Machine Learning is getting a lot of attention. The last thing anyone wants is for failed projects to trigger a new AI Winter so it's really important that we get this right. I therefore thought it would be helpful to publish a series of articles on the practical application of Machine Learning.
This blog is pitched at a non technical audience, however I'm more than happy to spin off deep techie conversations if required. As a Master Inventor and senior technical leader in IBM, I must stress that this is my personal blog and that the views and opinions expressed are entirely my own. They do not necessarily represent IBM's positions, strategies or opinions.
Machine Learning is all about computers learning from real world experience. For example, if you want to build a system that answers questions on your corporate web site, you start by giving the computer a set of example questions and the answers you would like in response. The Machine Learning system takes this training data and learns how to answer Questions. Similarly, if you want to train a system to recognise people by the sound of their voices you collect examples of their voice recordings and label each recording to build a training set.
At a high level, this is an attractive message as it leaves the audience with the impression that everything is easy. Grab some data, run it through a Machine Learning algorithm and you have a run-time system that you can apply easily to your live, operational data.
The devil though is in the detail and there are some important points that are worth noting:
- Machine Learning can only be effective when provided with good quality training data that is representative of the live operational data. It's not a short cut! Ensuring you have the right data is an important and time consuming aspect of the project. I estimate that 80-90% of the time I spend on analytics projects is spent getting the data right.
- Machine Learning systems come in many different forms. There are neural networks, probabilistic classifiers, Markov models, fuzzy networks and rules based systems. All of these different AI algorithms can be trained using Machine Learning. Machine Learning simply describes how the algorithm is trained. For example, I have a long history with rules based systems in defence applications. Defence customers liked rules based systems as they could understand why a system took a certain action in response to an input. However, I often hear rules based systems criticised because the rules are hard to define. In my Defence systems, the rules were derived using Machine Learning.
- be realistic about what you expect of a Machine Learning system. Quite often there is a belief that if you throw a Machine Learning, or any AI, tool at a large pot of data it will discover something important. For example, I have worked with law enforcement agencies where we were analysing huge amounts of data as part of criminal investigations. Sometimes, the answer just didn't exist in the data! In a Question Answering system, sometimes the Questions are so ambiguous that a human expert would struggle to understand. We shouldn't expect AI systems to solve unsolvable problems.
- think hard about how you assess the accuracy of your system. As with training data, it's important that any test data is representative of the live, operational data. However, remember that these are statistical systems that can be skewed by the data. Consider a Counter Fraud solution where the system has to identify cases of fraud. If 75% of the test data cases are not fraudulent, then I can achieve an accuracy of 75% just by never declaring a case to be fraudulent. Conversely, if I declare all cases to be fraudulent I achieve a perfect performance in predicting fraud but generate a huge number of false alarms in the process. There's a whole raft of work on the science of measuring accuracy that I will discuss in a later blog, however the key point to understand is that the test data can be skewed to alter the performance metrics.
- also remember to factor in the cost impact of any decisions. Consider a Customer Relationship Management (CRM) solution. You may develop a Machine Learning system to predict the likelihood of a customer leaving and going to an alternative supplier. A system may be 100% accurate in predicting that some customers are going to stay whilst failing to predict that other more valuable customers are going to leave. In tuning the system it's important to consider the cost impact of a wrong decision.
- don't assume that Machine Learning will easily out perform other analysis techniques. One of my earliest experiences was in applying Machine Learning in Formula 1 where many of the applications were in control systems. Control theory is a huge discipline with massive amounts of research and a whole academic and professional discipline behind it. Many of the systems we were looking at had been thoroughly analysed and modeled using control theory. These existing engineering approaches generally performed better. However, within control theory there were (are) specific problems that may benefit from a Machine Learning approach. Engineers working in control systems may use Machine Learning as one of the tools in their toolkit.
In my next blog I will talk more about training and test data and how to ensure you get the data right.
No comments:
Post a Comment