What is a Neural Network?

Neural networks, so named because they simulate in software the activities of neurons in the brain, are particularly suited to certain types of machine learning applications, including image processing, speech recognition, and natural language processing, and have become very popular for these tasks. However, they require large amounts of computational power both for training and inferencing, which can be time consuming or even impractical on typical computers. Specialized hardware, such as GPUs or custom devices, can speed up these calculations.

Inferencing

In the simplified example above, in which software is used to determine a probability for whether the image is a dog or a cat, the neural network consists of an input layer, a hidden layer, and an output layer. Each circle represents a software simulation of a neuron, and sophisticated neural networks may have many hidden intermediate layers, which are additional steps to help refine the result. At each step, inputs are multiplied by weight values, which are created during the training phase. Weights are expressed as two-dimensional matrices.