I have epilepsy, so you can imagine how fascinated I am about the inner workings of our brains. I have also, for a long time, been wanting to do an article about Artificial Intelligence! The first step in creating a decent A. I. (artificially intelligent) app is to set up a neural network. This enables us to teach it teach itself. Let’s explore this topic more.
Neuron
A neuron, also called a nerve cell, is the basic unit of the nervous system. Neurons are the unit which the brain uses to process information. Each neuron is made of a soma (cell body), axon, and dendrites. Dendrites and axons are nerve fibres.
Synapses
Neurons do not touch; instead, they form tiny little gaps called synapses. These gaps can be electrical synapses or chemical synapses, but ultimately these gaps pass the signal from one neuron to the next. You get excitatory synapses and inhibitory synapses. Signals arriving at an excitatory synapse cause the receiving neuron to fire. Signals arriving at an inhibitory synapse inhibit the receiving neuron from firing. Here is more information on the subject: Spatial and Temporal Summation.
Axon
An axon typically conducts electrical impulses away from the neuron’s cell body (soma). Axons, or nerve fibres, transmit information to different neurons, muscles, and glands. In pseudounipolar neurons (sensory neurons), such as those for warmth and touch, the electrical impulse travels along an axon from the periphery to the cell body, and from the cell body to the spinal cord along another branch of the same axon. Nerve fibres are classed into three types: A delta fibres, B fibres, and C fibres.
Dendrites
Dendrites are the branched projections of a neuron that act to propagate the electrochemical stimulation received from other neural cells to the cell body, or soma, of the neuron from which the dendrites project. Electrical stimulation is transmitted onto dendrites by upstream neurons (usually their axons) via synapses which are located at various points throughout the dendritic tree.
Artificial Neural Network
An artificial neural network (ANN) is an interconnected group of nodes, similar to the vast network of neurons in a human brain. Neural networks consist of multiple layers and the signal path traverses from the first (input), to the last (output) layer of neural units.
The purpose of a neural network is to solve problems similar to the way a human brain would. Neural networks are based on real numbers, with the value of the core and of the axon typically being a representation between 0.0 and 1.
Perceptrons
A Perceptron is an algorithm for supervised learning of binary classifiers which are functions that can decide whether or not input, represented by a vector of numbers, belongs to some specific class.
Layers
Layers are made up of a number of interconnected nodes which contain a sigmoid (activation function). Information is presented to the neural network via an input layer that communicates to one or more hidden layers. The actual processing in hidden layers is done with the use of weighted connections. The hidden layers then link to an output layer.
Single-layer Neural Networks
A Single-layer neural network is a network in which the output unit is independent of the other layers—each weight effects only one output.
Multi-layer Neural Networks
A multi-layer network is a feedforward artificial neural network model that maps sets of input data onto a set of appropriate outputs.
Forward Propagation
In a feedforward neural network the information moves in only one direction, forward—obviously, from the input nodes, through the hidden nodes (if any), and to the output nodes.
Back Propagation
Back propagation is a training algorithm consisting of feeding forward values and calculating errors and propagating it back to the earlier layers.
Reinforcement Learning vs. Supervised Learning
Reinforcement learning differs from supervised learning in that correct input and output pairs are never presented, and sub-optimal actions are not explicitly corrected. There is a focus on on-line performance which involves finding a balance between exploration of uncharted territory and exploitation of current knowledge.
Creating a Simple Artificial Neural Network
Let’s create a neural network, based on what we have discussed above.
Open Visual Studio and create a new Visual Basic Windows Forms project. There is no need to add controls to your form yet; we are only creating the neural network. In Part 2 of this article series, you will see how to implement the network you’ll be creating today.
Create a new class, name it Neuron, and add the following code:
Private lstDendrites As List(Of Dendrite) Private intDendrites As Integer Private dblBias As Double Private dblDelta As Double Private dblValue As Double
These members will represent their associated Properties that will be exposed to other classes. Add the Properties now:
Public Property DendriteCount() As Integer
Get
Return intDendrites
End Get
Set(value As Integer)
intDendrites = value
End Set
End Property
Public Property Dendrites() As List(Of Dendrite)
Get
Return lstDendrites
End Get
Set(value As List(Of Dendrite))
lstDendrites = value
End Set
End Property
Public Property Value() As Double
Get
Return dblValue
End Get
Set(value As Double)
dblValue = value
End Set
End Property
Public Property Bias() As Double
Get
Return dblBias
End Get
Set(value As Double)
dblBias = value
End Set
End Property
Public Property Delta() As Double
Get
Return dblDelta
End Get
Set(value As Double)
dblDelta = value
End Set
End Property
The DendriteCount property establishes the number of Dendrites present for this Neuron. Other properties include the Bias, Value, and Delta, which aid in identifying the correct output for our supplied input. In Part 2, you will train these values to produce the correct output. Now, add the Constructor for the Neuron class:
Public Sub New()
Dim n As New Random(Environment.TickCount)
Me.Bias = n.NextDouble()
Me.Dendrites = New List(Of Dendrite)()
End Sub
The Bias gets set to a random value. Add the Dendrite class now:
Public Class Dendrite
Private dblWeight As Double
Public Property Weight() As Double
Get
Return dblWeight
End Get
Set(value As Double)
dblWeight = value
End Set
End Property
Public Sub New()
Dim rndRandom As New Random
Dim dblNumber As Double = (rndRandom.NextDouble() * _
(1.0 - 0.00000001)) + 0.00000001
Me.Weight = dblNumber
End Sub
End Class
The Dendrite class contains only a Weight property that gets set to a randomly generated number. Based on this value, we will be able to train our Neurons to distinguish between what is acceptable and what is not.
Add the Layers class:
Public Class Layer
Private lstNeurons As List(Of Neuron)
Private intNeurons As Integer
Public Property CountNeurons() As Integer
Get
Return intNeurons
End Get
Set(value As Integer)
intNeurons = value
End Set
End Property
Public Property Neurons() As List(Of Neuron)
Get
Return lstNeurons
End Get
Set(value As List(Of Neuron))
lstNeurons = value
End Set
End Property
Public Sub New(numNeurons As Integer)
Neurons = New List(Of Neuron)(numNeurons)
End Sub
End Class
The Layers class contains a counter to keep track of the number of Neurons as well as a Neurons list that will contain the list of all the Neurons.
Add the Network class:
Public Class Network
Private lstLayers As List(Of Layer)
Private intLayers As Integer
Private dblLearningRate As Double
Public Property CountLayers() As Integer
Get
Return intLayers
End Get
Set(value As Integer)
intLayers = value
End Set
End Property
Public Property Layers() As List(Of Layer)
Get
Return lstLayers
End Get
Set(value As List(Of Layer))
lstLayers = value
End Set
End Property
Public Property LearningRate() As Double
Get
Return dblLearningRate
End Get
Set(value As Double)
dblLearningRate = value
End Set
End Property
Public Sub New(intLayers As Integer(), _
dblRate As Double)
If intLayers.Length >= 2 Then
Me.LearningRate = dblRate
Me.Layers = New List(Of Layer)()
For l As Integer = 0 To intLayers.Length - 1
Dim iLayer As New Layer(intLayers(l))
Me.Layers.Add(iLayer)
For n As Integer = 0 To _
intLayers(l) - 1
iLayer.Neurons.Add(New Neuron())
Next
iLayer.Neurons.ForEach(Function(neu)
If l = 0 Then
neu.Bias = 0
Else
For d As Integer = 0 To intLayers(l - 1) - 1
neu.Dendrites.Add(New Dendrite())
Next
End If
End Function)
Next
Else
Return
End If
End Sub
End Class
This class contains a list of Layers as well as a counter that counts the number of Layers present in the network. The LearningRate property determines how long the Network should take to learn the correct output from the correct input.
The Constructor adds the Dendrites to the Neurons and the Neurons to the Layers, with all their respective properties. In Part 2, you will explore and extend the Network class further.
Conclusion
A Neural Network’s main purpose is to replicate a human’s brain as closely as possible. In Part 2, you will learn more about Genetic Algorithms. See you then.
