It is all in the genes. In Part 1 of the AI and Visual basic article series, “Setting Up a Neural Network Using Visual Basic and AI,” I have shown you how to create a basic Neuron Network. This article will delve into Genetic Algorithms and create a way to train our already created Neuron Network to distinguish right from wrong.
Let’s start with biology first, again
Genes
A gene is a region (or locus) of DNA which is made up of nucleotides and is the molecular unit of heredity. The transmission of genes to an organism’s offspring is the basis of the inheritance of phenotypic traits. Genotypes, along with environmental and developmental factors, determine what the phenotypes will be.
Genotypes
A genotype is the DNA sequence of the genetic makeup of a cell of an organism or individual. It determines specific phenotypes (characteristics) of that cell or organism or individual.
Genetic Recombination
Genetic recombination is the production of offspring with combinations of traits that may differ from those found in either parent.
Mutation
A mutation is the permanent alteration of the nucleotide sequence of the genome of an organism or virus. Mutations result from errors during DNA replication or other types of damage to DNA.
Chromosomes
A chromosome is a DNA molecule with part or all of the genetic material of an organism.
You might say: Enough biology! Fact is: You need to understand how cells and neurons are put together to create a decent Neural network.
Genetic Algorithm
A Genetic Algorithm is inspired by the process of natural selection that belongs to the larger class of evolutionary algorithms. Genetic algorithms are commonly used to generate solutions to optimization and search problems by relying on operators such as mutation, crossover and selection.
Crossover
A Crossover is a genetic operator used to vary the programming of a chromosome or chromosomes from one generation to the next.
Selection
Selection is the stage of a genetic algorithm in which individual genomes are chosen from a population for later breeding.
Roulette Wheel Selection
Roulette Wheel Selection (fitness proportionate selection), is a genetic operator used in genetic algorithms for selecting potentially useful solutions for recombination. In Roulette Wheel Selection, the fitness function assigns a fitness to possible solutions or chromosomes. This fitness level is used to associate a probability of selection with each individual chromosome. This selection is similar to a Roulette wheel in a casino where a proportion of the wheel is assigned to each of the possible selections based on their fitness value.
Rank Selection
Rank Selection is similar to Roulette Wheel Selection except that selection probability is proportional to relative fitness rather than absolute fitness. This means that it doesn’t make any difference if the fittest candidate is hundred times fitter than the next fittest or 0.1% fitter.
Elitism
Elitism involves copying a small proportion of the fittest candidates, unchanged, into the next generation.
Our Project
You will pick up today where you left off in Part 1, “Setting Up a Neural Network Using Visual Basic and AI.” If you haven’t completed Part 1 yet, I suggest you have a read through it and download the project so that you can follow along.
Open the Network class and add the Sigmoid Function:
Private Function Sigmoid(dblSignal As Double) As Double
Return 1 / (1 + Math.Exp(-dblSignal))
End Function
The Sigmoid function above works out value to set for the current Neuron.
Still in the Network class, add the Configure Function:
Public Function Configure(lstInput As List(Of Double), _
lstOutput As List(Of Double)) As Boolean
If (lstInput.Count <> Me.Layers(0).Neurons.Count) _
OrElse (lstOutput.Count <> _
Me.Layers(Me.Layers.Count - 1).Neurons.Count) Then
Return False
End If
Fire(lstInput)
For i As Integer = 0 To _
Me.Layers(Me.Layers.Count - 1).Neurons.Count - 1
Dim neur As Neuron = _
Me.Layers(Me.Layers.Count - 1).Neurons(i)
neur.Delta = neur.Value * (1 - neur.Value) * _
(lstOutput(i) - neur.Value)
For j As Integer = Me.Layers.Count - 2 To 3 Step -1
For k As Integer = 0 To Me.Layers(j).Neurons.Count - 1
Dim neu As Neuron = Me.Layers(j).Neurons(k)
neu.Delta = neu.Value * (1 - neu.Value) * _
Me.Layers(j + 1).Neurons(i).Dendrites(k).Weight * _
Me.Layers(j + 1).Neurons(i).Delta
Next
Next
Next
For i As Integer = Me.Layers.Count - 1 To 2 Step -1
For j As Integer = 0 To Me.Layers(i).Neurons.Count - 1
Dim n As Neuron = Me.Layers(i).Neurons(j)
n.Bias = n.Bias + (Me.LearningRate * n.Delta)
For k As Integer = 0 To n.Dendrites.Count - 1
n.Dendrites(k).Weight = n.Dendrites(k).Weight + _
(Me.LearningRate * _
Me.Layers(i - 1).Neurons(k).Value * n.Delta)
Next
Next
Next
Return True
End Function
First, we check how many layers there are and how many Neurons objects. We loop through the layers and Neurons inside the Layers and set the Delta values of the Neurons. Lastly, we loop through the Neurons again and set the Dendrite values.
Add the next Function:
Public Function Fire(lstInput As List(Of Double)) As Double()
If lstInput.Count <> Me.Layers(0).CountNeurons Then
Return Nothing
End If
For lay As Integer = 0 To Layers.Count - 1
Dim lLayer As Layer = Layers(lay)
For neu As Integer = 0 To lLayer.Neurons.Count - 1
Dim nNeuro As Neuron = lLayer.Neurons(neu)
If lay = 0 Then
nNeuro.Value = lstInput(neu)
Else
nNeuro.Value = 0
For np As Integer = 0 To _
Me.Layers(lay - 1).Neurons.Count - 1
nNeuro.Value = nNeuro.Value + _
Me.Layers(lay - 1).Neurons(np).Value * _
nNeuro.Dendrites(np).Weight
Next
nNeuro.Value = Sigmoid(nNeuro.Value + nNeuro.Bias)
End If
Next
Next
Dim lLastLayer As Layer = Me.Layers(Me.Layers.Count - 1)
Dim intLayerCount As Integer = lLastLayer.Neurons.Count
Dim dblOutput As Double() = New Double(intLayerCount - 1) {}
For i As Integer = 0 To lLastLayer.Neurons.Count - 1
dblOutput(i) = lLastLayer.Neurons(i).Value
Next
Return dblOutput
End Function
This Function produces the output based on our input.
Design your Form to resemble Figure 1:
Figure 1: Our Design
Add the following code to train our Neurons:
Private Sub Button2_Click(sender As Object, _
e As EventArgs) Handles Button2.Click
Dim lstInputs As New List(Of Double)()
lstInputs.Add(0)
lstInputs.Add(1)
Dim lstOutputs As New List(Of Double)()
lstOutputs.Add(0)
lstOutputs.Add(0)
For i As Integer = 0 To 999999
nNeuralNet.Configure(lstInputs, lstOutputs)
Next
CreateTree(nNeuralNet, TreeView1)
End Sub
We supply numbers via the input received from the Textboxes, and then output the results into a Treeview.
Add the following code to execute its own output, which it has calculated itself:
Private Sub Button1_Click(sender As Object, e As EventArgs) _
Handles Button1.Click
Dim lstInputs As New List(Of Double)()
lstInputs.Add(0)
lstInputs.Add(1)
nNeuralNet.Fire(lstInputs)
CreateTree(nNeuralNet, TreeView1)
End Sub
Add the CreateTree method:
Private nNeuralNet As New Network(New Integer() {2, 4, 2}, 21.5)
Private Sub CreateTree(nNet As Network, t As TreeView)
t.Nodes.Clear()
Dim root As New TreeNode("Artificial Neural Network")
nNet.Layers.ForEach(Function(NetworkLayer)
Dim nLayer As New TreeNode("Network Layer")
NetworkLayer.Neurons.ForEach(Function(Neuron)
Dim nNeuron As New TreeNode("Neuron")
nNeuron.Nodes.Add("Bias: " + Neuron.Bias.ToString())
nNeuron.Nodes.Add("Delta: " + Neuron.Delta.ToString())
nNeuron.Nodes.Add("Value: " + Neuron.Value.ToString())
Neuron.Dendrites.ForEach(Function(Dendrite)
Dim nDendrite As New TreeNode("Dendrite")
nDendrite.Nodes.Add("Weight: " + Dendrite.Weight.ToString())
nNeuron.Nodes.Add(nDendrite)
End Function)
nLayer.Nodes.Add(nNeuron)
End Function)
root.Nodes.Add(nLayer)
End Function)
root.ExpandAll()
t.Nodes.Add(root)
End Sub
This outputs a Treeview based on the results, shown in Figure 2 underneath
Figure 2: Running
Conclusion
Neural Networks and genetic Algorithms are only the tip of the iceberg in the world of A.I. I hope I have introduced AI to you in this series, so that you can take it further.
