Posted by NOTE: This post is published on behalf of our summer intern, Javier Prieto. As Team BigML, we thank him for his valuable contributions to our mission of making Machine Learning easy and beautiful for everyone!
This summer, I had the chance to do an internship at BigML. I came across BigML thanks to a friend, and it stood out among the rest of the internships I wanted to apply to. My initial goal was to acquire some experience working while building my CV. Along the way, I learned a lot about the lifecycle of Machine Learning models and how they are eventually put to use.
After completing the BigML Engineer certification, I started to work with the team in the creation of models and data analysis for predictive applications. I mainly worked on the Smart Safe Stadiums project, which aims to fight violence and racism in football (aka soccer in the U.S.) stadiums in the Netherlands. Given that I am a first-year engineering student, I didn’t yet have a good perspective on how a Machine Learning project is constructed end-to-end, but thanks to my colleagues, I got to see how my contributions fit into the bigger picture of the project, and the value that the models that we created were delivering to final users of the solution.
During my internship, I had the pleasure to have Alvaro Clemente as my mentor. He has been incredibly helpful all the way through, from assigning me work that we thought was fitting my interests, to working side by side with me to solve problems that appeared along the way.
At the beginning of June, I started to work on an Image Classification task. The task at hand can best be described as training an image classification model for detecting flares and smoke using live video feeds from cameras inside the stadiums. Ultimately, we had to assign the images to one of three classes:
- Flares
- Smoke
- Nothing
Training Image Classification models was amazingly easy with BigML and its powerful algorithm, even without much previous experience. With just a few clicks, I was able to test and compare different architectures, and even customized architectures using OptiML, achieving remarkable results quickly.
As I started creating the models, I encountered various problems, which very often related to the data that I was provided with. The limited size of the dataset can be a limiting factor in the model performance, and we suffered from this issue in this project, as acquiring more relevant data wasn’t necessarily easy. We had to think of ways around that problem and we decided to try Data Augmentation techniques to artificially increase the size of the dataset by adding modified versions of images from the original training dataset. This is, indeed, a proven trick that can work handsomely on many image classification use cases.
By default, when you train a Deepnet model, the BigML platform performs some Data Augmentation behind the scenes for your image data such as height shift, width shift, and cutouts. These approaches were chosen because they are deemed to be the less likely to change the underlying class when applied whereas other methods can be problematic. However, I chose to perform my incremental augmentations fit for the Smart Safe Stadium project as a pre-processing step before using BigML to create the model. With the help of some python libraries and the easy-to-use BigML python bindings, I created a script that applied this pipeline to a folder with images and created two separate folders: the training set, with augmented images; and the test set. Subsequently, we had the option to connect to the BigML API to use these datasets to train a Deepnet directly.
We concluded that after every match, we were going to have a new set of images that we could work with and leverage to improve the models to make a better classification in the future. Thus, I created another script that automated the training process of a new neural network that includes both the old and the new dataset.
I was excited both because I was learning something new and the combination of these steps presented a true potential for real-life impact in the final project.
The last task I completed was related to Object Detection. Soon before the aforementioned feature was released to the platform, I started working on another part of the Smart Safe Stadiums project: the detection of flags and banners. This is a more complicated Machine Learning task that requires more complex models. But again, with BigML I was able to label the dataset and train models with relative ease.
When we trained a Deepnet with all the images we had and the manually labeled regions with the location of the flags and banners, the resulting evaluation was not good enough. It all circled back to the problem we had identified before; the dataset was not big and diverse enough, so the model did not have enough information to predict accurate regions. Once again, Data Augmentation is an obvious solution to mitigate this issue, but it’s not as straightforward to apply to Object Detection tasks. To boot, it’s very easy to introduce data leakage that will further invalidate evaluation results.
As a first test, I tried performing simple Data Augmentation with manual modifications of some images in the training dataset, modifying the regions accordingly. I used those images to create a bigger dataset and trained a new model and it showed some improvements, which shows that with some future work with this approach we can get better results.
During my internship at BigML, I’ve come to absorb much more than I had initially expected. It was an amazing introduction to the world of real-life Machine Learning. I am very grateful for the opportunity to work with such an innovative, fast-moving company and the very intelligent and kind BigMLers who have helped me take the first steps toward my career.
The Official Blog of BigML.com 