Posts in EXPRESSJS

This post continues describing a presentation I gave for my development team on Neural Networks and TensorFlow. Please read Part 1 first.

The second use-case I created was a model for simple plain-language search prediction. The idea I had was that a user could use a search input for the sales allowance reporting tool my team had built. The tool shows users the changes in sales allowances on ad campaigns, and it outputs those reports in a variety of formats depending on what the user needs. A language model might help users find the report type they want.

I had the team send me many different terms and phrases that users might use to find reports in the tool. I compiled all of these into about a hundred different text prompts and assigned each to a corresponding report type as its "intent". This array of prompts and intents became a rudimentary training data set for the model.

For this NN, I set up three nodes in TensorFlow, each with 4 inputs to represent 4 different report types that could be predicted. Once again I used a linear regression loss function and the Adam optimizer to compile the model. The unique part about this model compared to my first example was that I used TensorFlow's Universal Sentence Encoder to create the embeddings for the language model. I was surprised by how easy this was to implement. I just loaded the USE and used the models `embed` method to feed it all of the test prompt data.

With the NN set up and the model trained, I could now feed the endpoint a prompt and TensorFlow would return a set of 4 float values, 0 to 1. Each number corresponded to a report type, and its value was the percent likelihood that the prompt was intended for that report. With some simple TypeScript logic, the endpoint would return a URL for the report type with its value closest to 100%.

My hope was that the team understood how TensorFlow streamlines the setup of a NN and model training for language models, and how we could apply this to real-world use-cases going forward.

To end my presentation, I wanted to help them see visually what TensorFlow was doing, and what some bigger possibilities could be. I found a really cool tool called TensorFlow's Embedding Projector. It uses a 3D visualizer to display data points and correlation in several examples of ML models. A couple examples I used for my presentation were Word2Vec All (showing correlation between words), and Mnist with images (showing how handwritten numbers correlate to actual number values).

I set up a repo for the presentation materials I used here:

https://github.com/jdillman1989/tfjs-express-demo

I recently gave a presentation and demo for my current application development team about Neural Networks. It was an opportunity to show the team in a very practical, hands-on way the basics of NN and how they might be used. I chose to use real-world data that the team regularly works with, and built a fullstack demo of what it could look like from our development perspective, and from a user perspective.

It's uncommon to see TensorFlow implementations outside of Python, but my team has more experience in TypeScript/JavaScript. I thought it would help them grasp the basic concepts seeing it within their wheelhouse. I got surprisingly good results with a simple ExpressJS and React stack.

I set up two endpoints in the ExpressJS server to demonstrate two different use-cases: a price calculator and plain-language search predictions. The price calculator is a very bare-bones TensorFlow implementation just to introduce some of the high-level concepts to the team. It's a single node that takes a 1-dimensional input, with X values being regular pricing of a shipper, and Y values being historic sale allowances on that shipper. I used the Adam optimizer in TensorFlow to compile the model because of the relatively few training iterations I was doing for the presentation. It uses a simple linear regression loss function to predict what the future sale allowance will be for a given shipper based on the annual ad campaign it is part of.

I used data from two different annual ad campaigns, one with high correlation and one low correlation, to demonstrate how different loss values can affect the resulting model. My hope was that this would show the team a very practical example of some TensorFlow basics.

The second NN I created was to demonstrate the basics of a language model. I will go over that one in a following blog post, including some other helpful materials I used in my TensorFlow presentation.

I set up a repo for the presentation materials I used here:

https://github.com/jdillman1989/tfjs-express-demo

As a for-fun side project, I've been working on a simple top-down pixel art game engine in vanilla TypeScript on-and-off over the past year. It's been a nice break to work on TypeScript code that is completely decoupled from any framework or library, and I've always been interested in game development. Some features I want to highlight in the game engine:

1. Function- rather than Class-based code

   I see this as the trajectory that coding is moving towards, especially in web development. It takes some getting used to, and I could do a bit more structuring of the code. But I like how accessible all of my logic is.

2. Fully animated using HTML5 Canvas and requestAnimationFrame

   I learned that I needed to couple game object states with realtime Date methods rather than frame renders. A variety of factors can affect the browser window's framerate, like the device's battery level and GPU usage. I don't want the game to feel like Neo's bullet-time in those conditions. So objects can change their animations in real time, but the actual frame of the animation that's drawn is tied to framerate through `requestAnimationFrame`.

3. Color values that can be manipulated based on material and lighting data

   All the visuals of the game are stored as arrays of RGB values, rather than something like PNGs. Because of this I have full and immediate access to manipulate any color values of any visual element. If a character equips a helmet, or upgrades their weapon, I can modulate RGB values to display that change on the character. If a character is in a dark area and moves under a light source, I can change how shadows look on the character, and even adjust the colors based on the color of the light source. This lets the game display an enormous variety of visual states without having to manage a huge number of assets.

4. Visual effects for wind and weather

   I can generate sine waves that help calculate the position and animation of certain objects like cloaks and capes. It's an easy and convincing way to animate something blowing in the wind. Wind values can adjust the amplitude, frequency, and radius of the sine waves. Wind values change based on randomly calculated large circles that move across the map. Objects within those circles will change their animation frames or sine wave parameters based on the strength of the circle's wind value. There is also a day/night cycle that adjusts global ambient light values. Global ambient light values can also trigger events at certain thresholds, like lanterns turning on or fireflys lighting up.

5. Reusable menu systems for settings and inventory

   I can calculate how big I should draw a menu window based on the contents it will have. Menu and inventory options can be set up in various grids that the player can navigate using a keyboard or controller. I use the same code to render text for objects like interactive sign posts.

6. Controller support

   I use the native browser gamepadconnected event, and the navigator.getGamepads method to access connected controllers. I can read the input state of a controller and map those inputs to various functions within the game.

The basic architecture of the project is vanilla TypeScript for the front-end game engine, ExpressJS backend for basic CRUD operations on game data, and a MongoDB database to store game data. I have not built anything that could be considered a real game with this yet. There's a lot that should be done still, like save data for players, drawing larger maps, and implementing web sockets. The work has been fun, and it's enjoyable to see it play out on screen. Someday I'd like to take all of this work and turn it into a real web-based game.

Here's the repo for the game engine:

https://github.com/jdillman1989/garden-quest

Here's an example of what the game can do currently:

https://garden-quest.jdillman.com