NEURAL NETWORKING

Neural Network

This is a 9 it's sloppily written and rendered at an extremely low resolution of 28 by 28 px but your brain has no trouble in recognizing it as a 9 and I want you to take a moment to applause yourself.

How crazy it is that brain can do this so effortlessly? I mean all the below are recognizable as 9 even though the specific value of each px is very different from one another. The particular light-sensitive cells in your eye namely pros and cons on the inner surface of your eyeballs that are firing when you see this 9 are very different from the one firing when you see this 9 but something is the crazy smart visual cortex of yours resolves these as representing the same idea at the time recognizing other images as their own distinct idea.

But if we told you to sit down and write for us a program that takes in a grid of 28 by 28 like this and outputs a single number between 0 and 1o telling you what it thinks the digits are. Well the task goes from comically trivial to dauntingly difficult and here it comes the machine learning and neural networking but here we want to show you how neural network works.

Assuming no background and to help visualize what It is and which any fumbling let us say that is part of math. This blog is moly based on giving you briefly about the learning and networking on the bases of a real-life example. Without getting into the math part now by the same blog.

What we are going to do is put together a neural network that can learn to recognize handwritten digits. This is somewhat classic and easy to learn example. By the end of this blog, you should find yourself more resources able. There are many many variants of neural networks and in recent years there’s been sort of huge progress in research towards there variants but as this is an introductory blog and we still stick to the simplest and crystal-clear form with no adding toppings. And It is of utmost necessity to learn the basics before getting into the complex programs and note it, for instance, it does have some complexity to wrap our mind around but even in these simples form it can recognize handwritten digits.

The neural network is inspired by brains and let take it down for a moment. Simple think of neurons and how they are connected. For now, we want you to think of neuron having number 1-0 as a neuron. As disc sided earlier there are 28 x 28 px or neurons with a number ranging from 0.0 to 1.0 and further, they give a list of 784 neurons. Whereas the number rang is known as gray scaling numbers with 0 to be black and 1 to be white. And that is known as activation, which means the higher the activation more the brighter it is. And this all 784 make a layer and furthermore continues followed with 9 more layers.

Now talking about the last layer is termed as the output layer and which is helped by the inner two layers in such a way by calculation mathematically that activation number 1 should lighten up in only one of that neuron namely that neuron. And this actually how it represents how much the system thinks that a given image corresponds with a given digit. And the layer between input and output is termed as hidden layers

By here, we meant to say that the activation of the 1st layer enhances the 2nd layer which further enhances another layer and lastly, it enhances the output layer. And with the highest activation number output layer is the system choice so to speak for what digit this image represents

Before getting into the math part, just be with us to derive logic behind the importance of layer. Well, we want to get back to your preschool classes when you started to recognize we piece together various components such as 8 with 2 loops on one another 9 with half lop face same side on one another 6 as such curve above the loop. and further. Let suppose the 2nd a layer does the same and for all other loops and structures. Why the 2nd layer? as such, It is closer to the final answer and more accurate.

By here you would have a question how do we define a structure? the same way we tried for loops and structure we furthermore divide the loop too small components. And similarly, we have a huge array of such structured components.

Now we wrap up a bit for further understanding. The input layer with 784 spots, 2nd layer determines little edges, the 9th layer determines bigger loops and structures and the final layer output layer determines the final result.

By here you would have a certain curiosity that is there really 2 layers!? NO, actually not, honestly it boosts up the thinking and grasping knowledge.

And this is how image reorganization comes into play and more as speech remaking or auto-tuning are similar exam

Now getting toward the math part, just for now try to understand for only one specific neuron out of all those 2nd layers. That for picking up for that little edges that which further help in the 9th layer to take up bigger loops and components. But here the 2nd layer should be light up with the previous layer in such a manner to recognize the 1st little edge. And here the complexity starts.

Well, the paraments and formulas will help us getting from 1st to 2nd, and respectively, well what we’ll do is assign a weight (activation number of 2nd layer neurons) to each of the connections between our 2nd layer neuron and the neurons from the 1st layer. Take all those activations from the 1st layer and compute their weighted sum according to these weights. And we use green px for positive weights and red for negative weights.

Talking about our respective neuron to get the specific litter edge we think of that regime to be green and other to be in red 

The only problem in the above system is that their sum could be range from any negative to the positive number but what we want is to final up activation number between 0 and 1 and for that, we want to have to add up a certain specific function that squishes the real number line to range between 0 and 1 and a common function that does this called sigmoid function also known as the logistic curve.

So the activation of the neuron here is basically a measure of how positive relevant weighted sum is but maybe it's not that you want the neuron to light up when the weighted sum is bigger than 0 and to lighten up you want the same number say 5 and therefore you have to add some bias for inactivity and for that what if we need to do is just add up some specific number for such 5 number before adding up to sigmoid Squishification function. So, the weights tell you what px pattern this neuron in the 2nd layer is picking up on and the bias tells you how high the weighted sum needs to be before the neurons start getting meaningful active and this all stuff is for only one neuron. And each one neuron is connected to each other neuron to 2nd layer and further up.

On a rough calculation, we have all 784 pc neurons from 1st layer and each one of that 784 connection has its own weights associated with it also each one has some other number that you add on the weighted sum before squishing it with the sigmoid and that a lot to think about with this a hidden layer of 16 neurons(let suppose) that’s a total of 784*16(weights)*16(biases) and all this is for only one layer to another one layer. After all, set in done we have 784*16+16*16+16*16 weights and 16+16+10 biases give around 19002 total weights and biases, 19000 knobs and dials that can be tweaked and turned to make this network behave in different ways 

A lot of heck to do more by how the further is more technical and math part which is will be discussions in further blogs so stay tuned.

Hope this found you interesting.

~Aaditya Rasala