This post is adapted from a short introductory talk on Machine Learning which I delivered in September 2020 in an accounting hackathon https://accountinghack.sg/ held in Singapore. It provides a short example of how one can train a machine learning model and the many caveats that may come along the way.
After more than 10 years in the industry, I now teach undergraduate courses on Machine Learning and Software Engineering. During the course of the instruction, I do find that some common mistakes and trends often emerge as my students train their machine learning models. I’ll try to speak about some of these mistakes in this blog post and will do so by using a simple task on object recognition. Let’s assume we have the following task:
Build a model that differentiates between a black wallet and ball
Building our first model
Firstly, let’s try to determine how we can train a model that differentiates between a black wallet and a ball. Probably we will require some images of a black wallet, as well as some images of a ball. We will hence require two classes and we will train the model as below.
Notice that after we have completed training the model, we encounter our first issue. When I am not holding up any object, the model defaults to Class 1, which is that of the black wallet. Although it properly detects a ball when I hold up a soccer ball, this behavior might have unintended consequences further down the road.
Adding a Background Class
To solve this issue, one solution is to add a background class. In the background class which is chosen to be added, I am seen in the image standing nonchalantly and holding nothing up. You can see the entire training process in the image below.
After the addition of this background class, we are now able to properly classify the wallet class, the ball class and the background class. However at this point, we are still far from completing the task. What may be some other issues might the model suffer from?
Generalizing the Model
In the next video segment, you would note that I am holding up a significantly smaller ball for the model to test. At this stage, the model incorrectly classifies the small ball as either a wallet, or as the background. As such the model is unable to generalize. This means that when I test the model with data that we are expected to see during production, it will probably fail.
Trending AI Articles:
1. Microsoft Azure Machine Learning x Udacity — Lesson 4 Notes
2. Fundamentals of AI, ML and Deep Learning for Product Managers
In fact the generalization of the model is probably one of the most important steps you would need to carry out. This ensures that the data used to train the model is representative of the data that the model will be seeing after it is deployed. In the video segment below, you would notice that I added photos of me holding a smaller ball which is sent to be trained in the model.
Thereafter the model that was trained was able to identify balls of different shapes, colors and sizes.
Other potential pitfalls
We show below two other potential pitfalls of the model, one whereby I have switched to wearing a different colored jacket and the other where I had intentionally shown the machine an empty hand.
In both scenarios, the performance of the model was still acceptable but in reality, I would recommend that these different conditions be added into the training data as well.
Finally, you might consider the final bias , which is me! Recall that the original task was to be able to differentiate the photo of a black wallet and a ball. Hence the huge bias that we see in the data is myself, and I should not have been part of the machine learning model.
This simple example illustrates the issue where it is all too easy for us to introduce external influences (many times unknowingly) into the model. In the final example, notice that I used my hand to hold up the object for inference. One might still argue that the hand, or the color of my hand might be an external bias influence as well.
I hope this short post illustrates the steps one might need to take to ensure that the data used for training is sufficiently comprehensive in order to build a good model.
You can find the software demo used in this blog post at: https://teachablemachine.withgoogle.com/
Don’t forget to give us your ? !
How to build a good A.I. Model (Simple Pictorial Example) was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.
365 Data Science 