Deployment of Gender Classification Project With Tensorflow Serving

Initially, I knew nothing about Tensorflow Serving, probably because it isn’t so popular as the other tools used in deploying like Heroku, Flask e.t.c, I haven’t even heard of it or seen it before in the first six months I’ve gotten into machine learning. One night, my friend just hit me up asking if I’ve heard of it or have made use of it before when deploying, I was like “Noo..at all, I’ve not even heard of it” and told him he can always google search, make researches and all.

About a week later, he landed his first paid job as an intern DevOps Engineer through deploying one of his latest project using Tensorflow Serving and he also wrote an article on it. I was so thrilled and excited for him, which led to my interest in Tensorflow serving. Well, I suggest our minds should always be prepared for inspiration from a lot of ways, Now let’s explore what this Tensorflow Serving really does.

We successfully trained our model in the previous article and this time, we are bringing the model into production and making it available for real-life applications because a model without deployment is like keeping a portion of food on the table without eating it. The food solves your problem of being hungry same as the way the model solves real-life cases. There are several tools used in deploying, you can use Docker, Flask, Heroku e.t.c. which you can also explore and apply in various use cases. Each of them has its uniqueness and usefulness as well.

Serving is how you apply a machine learning model after you’ve trained.

Tensorflow Serving

Tensorflow serving makes the process of taking the model into production easier and faster. It allows you to safely deploy the models and run experiments while keeping the same architecture and APIs. Out of the box, it provides integration with Tensorflow and can serve other types of models also. Tensorflow Serving is also part of TFX Extended, Simply, TFX Extended is an API designed to make production-ready and build machine learning systems.

This is a sample preview of all the processes required for a successful machine learning system, so let’s say we want to run our models on a mobile phone, embedded systems, or on the web, we could simply use TF lite for the mobile or create javascript version of the model and run it on the web, It is actually a better idea to have a centralized model where you can simply send a response on a device, the server will then execute and return back to the device. The diagram is as shown below :

Tensorflow serving also provides an easy and friendly customer experience to the systems/apps because once the model is updated, the server and the device get instant access to it efficiently without crashing or damaging the server. This creates an automated process between the model, serving device and most importantly avoids an absurd customer experience. For instance, sending app updates when the user is using the outdated version of the app compared to when the app automatically updates and gets immediate access to the server. So let’s see how we successfully deployed this project with tensorflow serving below: In order to load a trained model into the tensorflow server, we need to save our model to a “savedmodel” format which will create a “protubuf” file in a well-defined directory heierachy, and will include a version number. Tensorflow Serving allows us to select a version of a model “servable”, which we’ll make inference requests. Each version of the model will later be saved into a different sub-directory under the given directory. The code is as follows:

We can now check the MetaGraphDefs i.e the models and the SignatureDefs(the method you can call in our saved model) by executing this command code, it is like showing the entire history of the saved model :

Serve The Model With Tensorflow Serving :

First we start by adding the tensorflow serving distribution source as a package URL : We’re preparing to install TensorFlow Serving using Aptitude since this Colab runs in a Debian environment. We’ll add the tensorflow-model-server package to the list of packages that Aptitude knows about. Note that we’re running as root.

Then we install the tensorflow server :

Now let’s load the tensorflow serving below, we’ll break down the lines of code and the parameters used here :

This is where we’ll load the model, run our TensorFlow serving, make inference requests using REST. So the third line of the codes above tells it to use tensorflow serving, and I included the bash magic cell at line 2 because I made use of google colab during the project N.B if you’re using a different IDE e.g jupyter notebook, spyder e.t.c. you don’t need to include line 2 because colab doesn’t provide a direct terminal while these IDE’S does.

The third line tells it to use the tensorflow model server, and the required paremeters are stated in the fourth, fifth, sixth line respectively as explained below:

1. rest_api_port: This is the port we’ll use in making our inference requests to run the tensorflow model server. Pretty direct yeah?
2. model_name: This the URL we’ll set at the inference requests, you can choose any name for it. A thing to note is that the — model_name will also appear in the URL on which you will be serving your models, so if you have multiple models at action managing your serving model URLs also becomes a lot easier.
3. model_path: This is the path to the directory/location where the model is saved. The last line here specifies that you want to enable logging and sometimes logs are just so helpful while debugging. I have personally used them quite a lot to figure out errors easily.

Now we’ve successfully loaded the TensorFlow serving and we’re getting to a very interesting part which is performing inference requests. Now we make a request to our Model In Tensorflow Serving: First, let’s quickly get a sample of an image in the test data :

Output Image :

He is so cute yeah *winks*? , Let’s move on to the next step where we’ll create JSON object for a batch of three inference requests and let’s see how our model recognizes things.

Now we have gotten to the final part of the deployment, which is invariably the testing part of our model i.e how well our model will perform and recognize the objects. We’ll send a predict request as a POST to our server’s REST endpoint, and pass it three examples. We’ll ask our server to give us the latest version of our servable by not specifying the version :

Now let’s specify a particular version of our servable. Since we only have one, let’s select version 1. We’ll also look at all three results below:

I guess the model didn’t classify Baby’s image correctly because,when we even consider real-life events, it’s not straightforward to tell whether a baby is a male or female from his/her facial look.But since it classified the remaining images correctly we’re good to go .

So we successfully built a model and deployed it using TensorFlow serving, You can still try out other techniques e.g vgg16, resnet50 or build your model from scratch as we did and I suggest you also try out various methods of improving the model for either binary or multi-label classification using CNN. We can also integrate this deployed model on a website, an application that will be user-friendly of course one will need to have a vast knowledge about the tools used in building the website or an application.

CONCLUSION

This article covers the deployment of the project on my previous article, Trust me it wasn’t easy coming up with this article, I hope you enjoyed this and could also gain some information. All the codes and files for this project are available here on my github repo: Github . If you found this article helpful, please do share it with your friends and leave a clap :-). If you have any queries, feedback or suggestions do let me know in the comments. Also, you can connect with me on Twitter, Linkedin.There are also available resources online you can always read, watch and learn for you to understand better based on concepts like this. There is so much to share with all of you and I’m just getting started. Stay tuned for more!