Transfer learning

The transfer learning, we use a model pre-trained on a very large dataset as the starting point for solving another related task. This related task is the one that you’re interested in implementing and is called the downstream task. For example, if we have a model trained on a large dataset to perform image classification, we can go ahead and use it to solve another image classification problem for which we don’t have the resource or data to do a full training to achieve high precision. When you consider the convolutional neural network architecture that we just saw, the goal of the convolutions were to extract features from images, and when those features were matched to labels, we could classify the image. But if we have a model that was trained on a large dataset that might contain similar low-level features to what we want in our model, then the filters that we’ll learn to extract those features might be a useful starting point for us and why should we reinvent the wheel? We reuse the architecture or layers of the pre-trained model and the weights obtained from training that model beforehand as the starting point for our problem. This significantly reduces the efforts involved in gathering data and the processing power and time required for training. We’re not just limited to the filters of the convolutional layers, the weights learned in the dense part of the model might also be useful to us. For example, it will be highly expensive to train a model with similar accuracy and precision for one of our own tasks, such as distinguishing between cats and dogs. But the popular MobileNetV2 model was trained on the ImageNet, which has 1,000 classes using over a million images. The filters that MobileNetV2 learned in its convolutional layers are highly generalized and could work for a new scenario like cats and dogs, and indeed it already has cats and dogs within its data set so we have a head start should we want to build a new cats versus dogs model. We can simply use MobileNetV2 and perform transfer learning. Let’s explore this visually, the architecture on the left is a convolutional neural network with n convolutional layers. 


These may have many filters that were learned on a large dataset like ImageNet. The architecture on the right is my unfinished architecture from model that I want to work on. I know that I need feature extraction in it, so how should I design those layers? I know that the filter is used in the existing model do an excellent job at extracting low-level features from a large range of images, so maybe I could just reuse them and copy them to my architecture so that my architecture makes the most of what the pre-trained model learned. 


Now an input could pass through those filters before going into my final classification for my classes and when I train them in my data, I don’t need to train the convolutional layers, and this saves me a lot of time and a lot of compute costs. We’ve touched on two of the reasons why you would want to use transfer learning, and it’s important to reiterate them here. First,

 it saved time and cost. 

Deep learning and in particular CNN’s, need a lot of data and need a lot of processing time to figure out and learn the correct filters for that data. By using pre-learned parts of the network, you can take advantage of this and stand on the shoulders of giants while saving yourself a lot of time, effort, and money. 

The second is improved performance with smaller datasets. 

The pre-trained networks might have been trained on millions of data items already, so not only are you saving time and effort, your architecture can take advantage of the features learned across such huge datasets, and few people have the time and resources to train at that kind of scale. Your particular task may also not have nearly as many training examples so it might be impossible to train a good model from randomly initialized weights, so why not take advantage of the pre-trained weights? There are some options into how you can transfer learning, and we’ll look at those next.