Posts in TENSORFLOW

This post continues describing a presentation I gave for my development team on Neural Networks and TensorFlow. Please read Part 1 first.

The second use-case I created was a model for simple plain-language search prediction. The idea I had was that a user could use a search input for the sales allowance reporting tool my team had built. The tool shows users the changes in sales allowances on ad campaigns, and it outputs those reports in a variety of formats depending on what the user needs. A language model might help users find the report type they want.

I had the team send me many different terms and phrases that users might use to find reports in the tool. I compiled all of these into about a hundred different text prompts and assigned each to a corresponding report type as its "intent". This array of prompts and intents became a rudimentary training data set for the model.

For this NN, I set up three nodes in TensorFlow, each with 4 inputs to represent 4 different report types that could be predicted. Once again I used a linear regression loss function and the Adam optimizer to compile the model. The unique part about this model compared to my first example was that I used TensorFlow's Universal Sentence Encoder to create the embeddings for the language model. I was surprised by how easy this was to implement. I just loaded the USE and used the models `embed` method to feed it all of the test prompt data.

With the NN set up and the model trained, I could now feed the endpoint a prompt and TensorFlow would return a set of 4 float values, 0 to 1. Each number corresponded to a report type, and its value was the percent likelihood that the prompt was intended for that report. With some simple TypeScript logic, the endpoint would return a URL for the report type with its value closest to 100%.

My hope was that the team understood how TensorFlow streamlines the setup of a NN and model training for language models, and how we could apply this to real-world use-cases going forward.

To end my presentation, I wanted to help them see visually what TensorFlow was doing, and what some bigger possibilities could be. I found a really cool tool called TensorFlow's Embedding Projector. It uses a 3D visualizer to display data points and correlation in several examples of ML models. A couple examples I used for my presentation were Word2Vec All (showing correlation between words), and Mnist with images (showing how handwritten numbers correlate to actual number values).

I set up a repo for the presentation materials I used here:

https://github.com/jdillman1989/tfjs-express-demo

I recently gave a presentation and demo for my current application development team about Neural Networks. It was an opportunity to show the team in a very practical, hands-on way the basics of NN and how they might be used. I chose to use real-world data that the team regularly works with, and built a fullstack demo of what it could look like from our development perspective, and from a user perspective.

It's uncommon to see TensorFlow implementations outside of Python, but my team has more experience in TypeScript/JavaScript. I thought it would help them grasp the basic concepts seeing it within their wheelhouse. I got surprisingly good results with a simple ExpressJS and React stack.

I set up two endpoints in the ExpressJS server to demonstrate two different use-cases: a price calculator and plain-language search predictions. The price calculator is a very bare-bones TensorFlow implementation just to introduce some of the high-level concepts to the team. It's a single node that takes a 1-dimensional input, with X values being regular pricing of a shipper, and Y values being historic sale allowances on that shipper. I used the Adam optimizer in TensorFlow to compile the model because of the relatively few training iterations I was doing for the presentation. It uses a simple linear regression loss function to predict what the future sale allowance will be for a given shipper based on the annual ad campaign it is part of.

I used data from two different annual ad campaigns, one with high correlation and one low correlation, to demonstrate how different loss values can affect the resulting model. My hope was that this would show the team a very practical example of some TensorFlow basics.

The second NN I created was to demonstrate the basics of a language model. I will go over that one in a following blog post, including some other helpful materials I used in my TensorFlow presentation.

I set up a repo for the presentation materials I used here:

https://github.com/jdillman1989/tfjs-express-demo