Introduction to Neural Networks

Humans can learn a lot from nature. In fact, many inventions and technological advances are inspired by biology. One idea inspired by biological nervous systems is the artificial neural network (ANN), often referred to as just a neural network (NN). A neural network consists of many processing nodes, or neurons, interconnected with varying connection strength. To understand what makes up a neural network, let’s look at a biological neuron.



The soma, often referred to as the cell body, receives information from other cells via many dendrites. The electrical impulses are then summed together upon arrival. If the summed electrochemical signals entering the neuron are of great enough magnitude, the neuron becomes active and fires signals down the axon. The axon is the transmission in which signals travel. The electrical impulses then reach the axon terminals where neurotransmitters and chemicals are release across the synapse, a small gap between the sending (presynaptic) and receiving (postsynaptic) membrane.

Neural networks attempt to communicate and process information in the same manner that biological networks do. An artificial neuron, or node, mimics the processing of a biological neuron. Information is received from many input sources and summed. If the summed total of the input is at or beyond the active threshold, then the neuron becomes active. Otherwise, the neuron is inactive. This artificial neuron behaves similar to biological neurons when processing information.



Neural networks are comprised of many neurons formed into interconnected layers. In a simple feed-forward network, you will typically see 3 layers of neurons. The first layer is the input layer. As the name suggests, this starting layer receives the initial input data to be fed into the network for processing. The next layer is the hidden layer. There can be one or more hidden layers in a neural network. The hidden layer gets its name based on the fact that it is only receiving and sending to and from other nodes, and its input and output are not read directly. Quite simply, the hidden layer is “sandwiched” between the input layer, and our last layer – the output layer. This last layer receives the data from the hidden layer and the state of each neuron acts as the result set from our input data fed into the input layer. Let’s have a look at what a simple three-layer neural network could look like.



Notice how each node in the input layer connects to every node in the hidden layer, and the same for each node in the hidden layer connecting to the every node in the output layer. You can see the node sets i, j, and y, but what about the w’s? Those represent the strength of the connection between each node. Every connection between two nodes has a weight. The weight determines how strong or weak the transporting signal becomes. So how does information process through a neural network? It’s not so difficult. The value of j₁ can be determined by first, summing up its weighted inputs. In this case, the value weighted inputs are the value of i₁ multiplied by w₁, plus the value of i₂ multiplied by w₂, plus the value of i₃ multiplied by w₃.

This can be expressed mathematically as: i₁w₁ + i₂w₂ + i₃w₃

The same is concept can apply to any node in the network.
Given n number of inputs, we can define the weighted sum to be:

Once the weighted sum is determined, it is passed to the transfer function which determines the active or inactive status of the neuron (Typically a value of 1 for active, and 0 or -1 for inactive). The threshold function is usually a nonlinear function such as sigmoid, or step function.
A sigmoid function is characterized by its “S” shape and can be expressed as:

A step function is defined simply by: f(x) = 1 when x >= 0 and f(x) = 0 when x < 0

At this point, you should be familiar with what a neural network is, and how information is processed through from the input to the output (forward calculation). Please feel free to send me your questions or comments.

De-clutter your desktop with Launchy

Do you have too many icons on your desktop? Do you have a hard time finding files, documents or pictures? Launchy can help. Launchy is an open source keystroke launcher for Windows. That means that with just a few keystrokes, you can quickly and easily locate anything on your computer, reducing the need for space-consuming icons on your desktop. To get started, visit http://www.launchy.net. Click “Download (Launchy, Skins and Plugins)”. Then click the “here” link.



Once you download and install Launchy, double click the icon to run the application. If you don’t see anything, try pressing Alt+Space to open the Launchy window.



Although optional, the first thing you’ll want to do to get the most from Launchy is to update the catalog. To do so, click the gear icon in the top right corner, or right click the Launchy window and click “Options”. Then click the catalog tab and add extra folders that you would like Launchy to index. More information on the catalog can be found in Launchy’s official documentation at http://www.launchy.net/Readme.pdf. (Note: You must have Adobe Reader to view PDF files.) Now you can bring up the Launchy window by pressing Alt+Space. Type in the name of a folder, application, or file, and Launchy will quickly display a list of likely items you’re trying to open.



Press Enter or click the item you want to open and it will launch. Now you can remove space-consuming icons from your desktop and let Launchy do the work for you!