24.1. The Inspiration

Recall that the goal of machine learning, and artificial intelligence more broadly, is to build intelligent machines that perform tasks typically requiring human intelligence such as reasoning, problem-solving, understanding language, or making decisions. Many innovative products and systems over the years have been inspired by nature. This is called Biomimicry, the practice of learning from nature’s designs and strategies to solve human design challenges. Birds inspired flight, the kingfisher inspired the aerodynamic nose of the Japanese bullet train, and streamlined shapes found in nature inspired Speedo’s LZR Racer suit. This suit helped swimmers win 98% of all swimming medals at the 2008 Olympics before being banned.

Similarly, when seeking to build intelligent machines, it makes sense to look at nature’s most sophisticated computer: the human brain. Artificial Neural Networks (ANNs) are machine learning models inspired by the networks of biological neurons in our brains. Just like our brain uses neurons to process data and make decisions, ANNs use artificial neurons to analyze data, identify patterns, and make predictions.

Figure: Biological neurons connected via synapses, Image Source

A biological neuron is composed of a cell body, many branching extensions called dendrites, a long extension called the axon which splits off into many branches, and at the tip of these branches are minuscule structures called synaptic terminals or synapses which are connected to the dendrites and cell bodies of other neurons. They are not actually attached but are so close that they can very quickly exchange chemical signals. Biological neurons produce short electrical impulses which travel along axons and make the synapses release signals called neurotransmitters. When a neuron receives a sufficient amount of these signals within a few milliseconds, it fires its own electrical impulses.

Thus, individual biological neurons behave in a rather simple way, but they are organized in a vast network of billions of neurons, with each neuron connected to thousands of others. Together, they are capable of performing highly complex computations. As you will see in the next section, a neuron in an Artificial Neural Network called a perceptron behaves in a simple way, but when several of these perceptrons are layered together, they are capable of performing highly complex tasks!

Just like airplanes are inspired by birds but do not flap their wings to fly, ANNs have gradually become quite different from their biological cousins. Some researchers prefer to drop the biological analogy altogether, for example by using the term “unit” instead of “neuron,” but we will continue to use “neuron” to emphasize its biological inspiration.