Neural Networks with JavaScript Succinctly

The best way to see what this e-book is all about is to take a look at the screenshot in Figure 1- 1. The image shows an example of using a neural network written using the JavaScript language without any significant external libraries. The network predicts the size of the company (small, medium, or large) that a college student will work at after graduation, based on sex (male or female), GPA, major (such as finance), and parents’ annual income.

If you look at the shell carefully, you’ll see that the program is hosted by the Node.js JavaScript environment. The code in this e-book was run on Node.js version 10.14.1, but any relatively recent version will work. The key programming language dependency is JavaScript ES6, also known as ECMAScript 6, ECMAScript 2015, and JavaScript 6. You can run the code examples in a web browser environment if the browser supports ES6, and if you refactor Node.js console.log() statements and use the HTML5 File API or the XMLHttpRequest for file reading and writing.

This e-book assumes you have intermediate or better programming skills, but does not assume you know anything about neural networks.

This e-book presents complete example programs for the three major types of neural network problems. A multiclass classification problem predicts a discrete value where there are three or more possible values, for example, predicting the size of company (small, medium, large). A regression problem predicts a single numeric value, for example, predicting a student’s GPA. A binary classification problem predicts a discrete value where there are only two possible values, for example, predicting the sex of a student.

Everyone I know learns a new programming technology in the same way: you get an example program up and running and then experiment with the code. All the program code is presented in this e-book so you can copy and paste it, and run the demo programs. The data files used by the programs are in the appendix of this e-book. You can find the program code and data on GitHub. To prepare, all you have to do is install Node.js and use the code and data presented here. No other software is needed.

If you look at the screenshot in Figure 1-1 carefully, you’ll see that there is a lot going on. Some of the fundamental concepts shown include neural network architecture, neural network inputoutput, tanh and softmax activation, back-propagation, error and accuracy, normalization and encoding, and model interpretation. This illustrates the major challenge for learning about neural networks: although most concepts are relatively simple, there are many concepts, and they interact with each other in nonobvious ways. But you can learn all important neural network concepts by running and examining the code in this e-book.

All the neural network examples in this e-book use a single hidden layer of nodes. Neural networks that have two or more hidden layers are called deep neural networks. In theory (the Universal Approximation Theorem, sometimes called the Cybenko Theorem), a neural network with a single hidden layer can compute anything that a neural network with multiple hidden layers can. But in practice, deep neural networks can solve some problems that simple neural networks cannot.

Other forms of deep neural networks use complex architectures. Examples include convolutional neural networks (often used to classify images) and recurrent neural networks (often used for natural language processing). Implementing a neural network from scratch gives you complete control over your system and enables you to have a complete understanding of your system.

The techniques used in this e-book can be extended to neural networks with two hidden layers. But for three or more hidden layers, or for deep neural networks with complex architectures, implementing from scratch is not practical in most situations. For these scenarios, you should use a deep neural network code library, such as TensorFlow, Keras, or PyTorch.

However, at my company, where we have hundreds of engineers working on deep neural systems, it’s considered essential to have a solid foundation that can be best gained by understanding single, hidden-layer neural networks and implementing them from scratch. Much of the content presented in this e-book was developed for engineers who are new to the field of machine learning and neural networks.