Aryan Singh's World

Data Science, Statistics, ML, Deep Learning

Tensorflow 2.0 in 2 minutes

Tensorflow 2.0-alpha was released a couple of days ago with a bunch of exciting features. It can be installed by following command:

pip install -U –pre tensorflow

In this post I explore the 17 most key features among them. The purpose is to make this short, crisp but touch on all the major pointers.

  1. Improvement in tf.keras high level API: Tensorflow 2.0 takes the compatibility between imperative Keras and DAG driven tensorflow to next level by adopting tf.keras to its core. This will make prototyping and productionizing deep learning models fast. Also, this will engage and bring more developers towards deep learning, keras being more intuitive.
  2. Eager execution by default: No need to create an interactive session to execute the graph. TF 2.0 introduces the eager execution by default moving all the session related boilerplate under the hood.
  3. Compact and improved documentation: TF 2.0 offers better organized documentation. Most of it is available here:
  4. Clarity: 2.0 takes clarity to the next level by removing various duplicate functionalities like multiple versions of GRU and LSTM cells available. 2.0 takes care of choosing the optimum node according to hardware giving developer a single unified library to choose from for instance one implementation of LSTM and one for GRU.
  5. Low Level API: Full low level API in tf.raw_ops with inheritable interfaces for variables, checkpoints and layers to define your own components.
  6. Easy Up-gradation: Conversion script in tf_upgrade_v2 to convert TF 1.0 code into TF 2.0 code automatically. Just write: !tf_upgrade_v2 –infile <input_file> –outfile <output_file>
  7. Backward compatibility: Comes with a separate backward compatibility module tf.compat.v1 for getting the older components.
  8. One optimizer, one losses module, one layers module: Unified optimizers module in tf.keras.optimizer.*. Similarly one losses and one layers module under tf.keras.losses.* and tf.keras.layers.*.
  9. Better graphical visualization: 2.0 gives better graph visualizations in Tensorboard even for keras models.
  10. Easy to distribute: Provides more options for scaling and multi GPU training via tf.distribute.Strategy module. strategy = tf.distribute.MirroredStrategy()
    with strategy.scope():
    <define the model here>
  11. Save and Import keras model: Easy to save and load the keras models by using tf.keras.experimental.*.
  12. Run Keras On TPUs: 2.0 comes with tf.distribute.experimental.TPUStrategy() that will allow the keras code to run on TPUs.
  13. New datasets available: 2.0 comes with new datasets to test the models on in vision, audio and text domain.
  14. More pre-trained models at TF Hub: More pre-trained models from the world of NLP and Vision available at Tensorflow Hub.
  15. Improved error reporting: Improved error reporting with exact line number and full call stack.
  16. TFFederated: TF federated to support federated learning on edge devices.
  17. Swift support and Fast AI: 2.0 to come with a Swift library. Jeremy Howard will be delivering a course on the same.

Source: Tensorflow Dev Summit 2019

How to choose the Machine Learning algorithm to use for a problem?

The choice of machine learning algorithm to solve a particular problem is very hard to determine before trying a bunch of algorithms along with hyperparameter optimisation. But there are some pointers that can be kept in mind while figuring out the right algorithm:

1. Time Series Data: For data having one dependent variable in the form of time sequence algorithms like ARIMA or sequence models like LSTM can be benchmarked to find the optimum solution.

2. Speech/ Text Analytics: Probably a deep learning based approach along with sequence to sequence models like RNN and LSTM can be a good start.

3. Text Classification: Sequence models like HMM, CRF and LSTM can be tried for this solution.

4. Structured Data(Regression): Linear regression can be used as baseline, followed by SVM regression followed by using non linear kernels like rbf. Tree based ensemble models like Random Forrest and XGBoost should be tried for a more intuitive solution.

5. Structured Data(Classification): We can start with Logistic Regression for baseline. It also explains the importance of each of the dependent variables in terms of the coefficient. Furthermore, SVM, Random Forrest and XGBoost can be tried. If we have a large number of training examples and better hardware then solution based on deep learning along with appropriate activation functions like softmax(Multiclass) or sigmoid(binary) can be used.

6. Image/Video based Data: For image/video based data a pre trained DL based network is a good starting point. Particularly tried and tested architectures like VGG and Resnet 50 trained on Image net dataset can be used and the final few layers can be retrained to tail it to our particular problem. For real time object detection YOLO is a very elegant solution and can be given a try.

What do NaMo’s speeches convey?

This weekend while wandering around the labyrinths of internet, I stumbled upon the corpus of Indian prime minister Mr. Narendra Modi’s speeches. I thought it would be interesting to analyse the speeches to see what are the main issues he speaks about and what is the overall connotation of the speeches. In this blog, I present my analysis of the speeches along with the visualizations in the form of graphs and plots.

Unigram and Bigram Frequency

I used count vectoriser from sklearn feature extraction to vectorise the text into frequency vectors and then summed it over the rows to find the frequency of each word in the overall corpus. Later I plotted the top 30 words by frequency on a barplot to analyse them. Following is the result I got:

Since mann ki baat is a program aimed at listening to and addressing problems of people PMs main focus is on issues relating to poverty and water. Also he talks about taking actions by using phrases like time, make, great.

Most Frequent Nouns, Adjective and Verbs

Next up I thought it would be interesting to do POS tagging of each speech to see what are the major issues PM lays stress upon and how positive/willing he is to solve them. For this I pos tagged the whole corpus using nltk and then found out the most common nouns, verbs and adjectives out of it. I plotted 16 most common nouns, adjectives and verbs in the form a word cloud to visualise and draw inferences. Here is what I got:

Nouns Cloud:


It is clear from the word cloud that the main issues that are being highlighted are related to basic amenities like water, villages, farmers. Interestingly enough, yoga is repeatedly a frequent part of the conversation. PM has also addressed issues about black money but the frequency is on the lower side.

Verbs Cloud:


The verbs mostly have a positive connotation. Words like think, make, started and done indicate the action oriented approach.

Adjectives Cloud:


Adjectives do reveal a basic essence of the major fields/issues that PM is targetting. Youth, poor and digital India initiatives are some of the most frequent areas touched upon.

Sentiment Analysis Of The Speeches

Next up I analysed each speech for it’s sentiment score to understand whether the connotation is positive/negative and how it has changed overtime. I use TextBlob library to sentiment score each speech. Here is how the time series sentiment score looks like:


Looking at the overall analysis, the speeches don’t seem to be that positive. This might be because they are aimed at addressing the issues people face on daily basis. Overall years 2015 and 2016 are more positive as compared to the other years.




Training Machine Learning Models in Cloud: FloydHub – Part 2

In the last tutorial we saw how to train a Deep Learning model on Floydhub GPU via Jupyter notebooks. In this week’s blog I am going to demonstrate how to run a local python script on the Floydhub GPU.

Till step 4 the process remains same as the previous post where we create a Floydhub project and initialize the same locally. Our directory contains the following files:

Once initialized we need to issue the following commands to run the script remotely on floydhub GPUs:

floyd run --gpu --env tensorflow-1.3 "python"

Here  is the file that trains and runs a Feed Forward net on the dataset. Here is what this command does internally:

  • Syncs the local code to FloydHub’s servers.
  • Provisions a GPU instance on the cloud with TensorFlow 1.3 installed.
  • Executes the command python on the GPU server.
  • Stores the output logs and generated output data.
  • Terminates the GPU instance once the command finishes execution

Here is the project state after the execution of the command:



Following is the output page with metrics:



Following is the command line output:



With that we come to end of this tutorial series. In next series I explore Google Colab as a way to procure GPU for training Deep Learning models. Till Then:

Happy Deep Learning !!

Training Machine Learning Models In Cloud: FloydHub

Training machine learning models can be a time-consuming task and can take several hours to days to train a model especially if the model is a dense deep neural network. Recently I faced a similar issue while trying to run a feedforward net with GridSearch for a classification problem. Since my laptop lacks a GPU it took over 8 hours to train the model and considerable consumption of memory and CPU rendered my laptop useless for the time duration.

This experience along with several other experiences motivated me to look for an alternative way to train models that can be done remotely without compromising time and resources of my laptop. After doing some research I stumbled upon Floydhub a cloud-based solution for training our Machine Learning Models.

Following are the advantages of Floydhub:

  • Easy to use and intuitive UI
  • Great documentation
  • Interactive environment through Jupyter notebook on cloud
  • Seemless integration with python
  • Pre setup GPU along with libraries like Keras and Tensorflow pre-installed

Here I will display a binary classification problem of customer churning/not churning from a bank on the basis of the customer characteristics on the floydhub platform. We will be training afeed-forwardd neural net with back propagation and stochastic gradient descent using the Keras library to achieve this task. I will be displaying two methods of doing this. One with Jupyter notebook and another using remote execution of a python script. The dataset for this exercise can be downloaded from here:

Let’s go over both the methods one by one.

Part 1 : Training ANN using Jupyter Notebook in Floydhub


Account on Floydhub.

Step 1: Install the floyd-cli for python.

First step is to install floyd-cli for python. This can be achieved by issuing the following command:

pip install -U floyd-cli

Step 2: Login into the Floydhub account

Once we have the floyd-cli installed we need to login to floydhub account from the command line to enable our floyd interface to run our job. This can be achieved as follows:

floyd login
Login with your FloydHub username and password to run jobs.
Username [aryan]: aryan
Login Successful as aryan

Step 3: Create a Floydhub project on cloud

Now we need to create a Floydhub project on the Floydhub portal. We can do so by hovering on the + sign on the top right corner.


Once we click on create project we need to add details for the project:


Now the project is ready and we can see the following project page:


Step 4: Initialize the Floydhub project locally

As highlighted in the previous image we need to issue the following command in the command prompt to initialize the project locally:

floyd init aryancodify/bank-churn

Let’s issue the command and look at the output. Before issuing the command we can go to the working directory where we want this project to be initialized:


Step 5: Sync the Jupyter notebook and datasets from local to floydhub and running it on cloud

Once the project is initialized locally we can sync our jupyter notebooks and dataset to the floydhub project on cloud to run the notebook interactively using the gpu. We need to issue the following command for this:

floyd run --gpu --mode jupyter

Once we run the command we will see the following output:


We had the following files which will be synced to cloud:


This can take some time as the floydhub runtime executes following task in the backend:

  • Sync your local code to FloydHub’s server
  • Provision a GPU instance on the cloud (if you want CPU, drop the --gpu flag)
  • Set up an deep learning environment with Tensorflow and keras installed
  • Start a Jupyter server on the cloud, and open the url in your browser.

Once this command is executed a new window will open up in the browser with the view of the Jupyter notebook:


Here we can see our notebook that we synced from local file system to floydhub. Then we can open the Jupyter notebook and start interacting with it.

Step 6: Looking at the Job status and metrics:

We can check the status of our job by going into the jobs page of the web dashboard. This will list all our jobs:


We can click on a particular project to check on its jobs.

For eg here clicking on the bank-churn project we see the following dashboard:


Here we can see that our GPU utilisation was almost 95.2% i.e. 11 gb.

Looking at the notebook:

We use the Keras library to build a feedforward net in order to perform our binary classification the project and notebook can be accessed here:


We can clearly see that we trained the model for 100 epochs and it took roughly 1-2 mins to train the model.

Since building the network is out of scope for this post will not be discussing the details here, though appropriate Markdowns and comments have been added in the notebook for a better understanding.

After training the model we get an accuracy of around 86% on the test dataset which is pretty good.

To check whether Keras is running with the GPU we use the following commands in the Jupyter notebook:

from keras import backend as K

We will look at achieving the same using the remote python script in part 2 of this tutorial.