Artificial intelligence is a field that has been around for many years; mostly in the realms of mathematics and computer science. It was a curio to be studied, but rarely used in the field. In recent years, however, the prevalence of AI in computing has been continually increasing. It turns out that most things a human can do in terms of analysis and categorisation, a machine can do better. Isn’t there a song in a musical which goes something like that?

While Artificial Intelligence is a term that has been around and widely understood for many years, it’s usually the domain of Science Fiction with characters such as Johnny Five, the Terminator, and Data to name a few. This aspect of artificial intelligence focuses on emulating the human consciousness or behaviour, but doesn’t touch on the problem solving abilities humans inherently possess. This branch of artificial intelligence is what is being applied to real and ground breaking projects in the modern world; these focus on emulating the human brain’s ability to categorise data and harness it for data analysis.

Neural networks allow you to analyze data and draw meaningful conclusions much more efficiently, and enable a process that would once consist of scores of humans reading through spreadsheets to be automated. This automation can be accomplished through in-depth knowledge of your data, an idea of what conclusions may come of this data, and degree level knowledge of mathematics.

If you’ve gotten this far, don’t worry; there’s no maths in this article. None. Implementing Artificial Intelligence is a topic of far greater scope (and maybe something for a film or three).

Thanks to the Internet, there is no need to build your own; there are already cloud based solutions available such as IBM’s Watson and MonkeyLearn which can take most of the work out of using neural networks. These services enable the end user to interact with them using a provided API, and also provide graphical interfaces to their technologies which enable them to be tailored to the task at hand.

What follows is a primer about Artificial Intelligence in the context of Machine Learning. Sadly, this means you won’t actually be able to construct your own T-800 or Glados by the end of this article.

Neural Networks 101

Neural networks, as their namesake implies, aim to mimic the human brain in the way they’re constructed and how they process data. They will usually take the form of having three distinct layers; the input layer, which is where the data is fed into the network, the processing or hidden layer, where the data is processed, and an output layer, where the neural network’s conclusion is spat out.

These three layers take the forms of nodes and weighted connections. The nodes are effectively black boxes; there’s some clever maths inside them which will do the same thing every time data is passed to it. The weighted connections are where the magic happens; the connections have weights attached to them, which can either increase or decrease the relevance of that specific node. Through learning, the neural network can decide which weights need adjusting (and how much by) in order to come to what it thinks are the correct conclusions.

To give a very basic example of the kind of thing that neural networks are capable of, let’s start by thinking of decision trees. Decision trees allow you to derive a conclusion from a data set by using choices as classifications. Each choice branches off into further choices; filtering data into the correct classifications as set by the creator, until you end up with a conclusion.

Tree Diagrams

Tree diagrams aren’t neural networks in the sense of computing, because the human brain does all of the legwork involved in designing the tree itself, and a neural net is usually capable of teaching itself correct analysis of data. Despite this, we can draw parallels between the two classification methods.

Deriving the questions to ask at each step of the tree is akin to teaching the neural network; for this step you already have data and know enough about what the data means to think of the right questions to ask just like you would teach a human to analyze data. When you then start running data through this binary tree, it is able to work in the ways that the creator has predicated. As such, it will produce reasonable conclusions about your data; assuming the data you are supplying is close enough to the data you used to design the tree.

Neural networks tend to have two different uses; classification, and functional approximation; also known as clustering. Neural networks that classify are starting to become more and more popular in applications because of their ability to (correctly) classify data that only a human would otherwise be able to analyze or interpret.

One such example of this includes Google’s usage of neural networks in order to recognise house numbers and faces from Street View imagery; it would take an unimaginably long time for an analyst to go through the images and manually pick out house numbers or people’s faces.

Supervised vs. Unsupervised

Before we begin talking about supervised versus unsupervised learning it’s worth mentioning that the difference isn’t the ability to leave the net to learn for itself versus having to supervise it or correct the learning process; as the names might otherwise imply.

The difference is in the functionality of the nets. Unsupervised nets are those which have ‘learnt’ in an unsupervised fashion; they haven’t actually processed any data to inform their decision making process because they have a sheer mathematical base. This ends up clustering the data naively into groups based on the formulae you choose. Now, I promised no maths in this article, so we’re just going to leave that be.

Supervised networks, however, tend to require quite a lot of data in order to work properly. You’re teaching the network what you’re looking for, after all, and as such you need to also produce a data set which includes both example data and the conclusions the network is expected to reach from that data.

Teaching a robot

Teaching a neural network what’s expected of it is, thankfully, much easier than teaching a robot to love.

In the case of supervised networks, it’s a necessity to have a good base data set. Unsupervised networks, however, do not require such a data set, but do require a compromise. The results that you receive from them will not be as definitive as if you had trained the network, and will require interpretation in order to be useful.

Data sets

The only problem with this base data set is generating it; this is where most of the effort in creating a neural network will be spent. You need to make sure that the data that you have is as representative of what you wish to determine as possible. That is to say, you need to make sure the information is well rounded and contains roughly equal examples of the final categories you wish to determine, as well as all of the parameters required to reach that category. These two groups of data are vital so the neural net can learn about the data and teach itself how to reach the same conclusions.

How much data you will need depends solely on how complex your neural network is. If you have multiple categories, each with nuances, you will need a lot more data than a binary yes/no decision with a few deciding factors. A binary network like this can be constructed reliably with under a dozen teaching entries! In addition to this, all of your data needs to be normalized; that is to say, representing all of your data as numbers which have a fixed range. This means that your pet neural network knows what to expect from its training data (and otherwise might not produce valid results).

Training and its effects

Once you’re happy with the data that has been produced, the next step is to run this data through the neural network. From this point on, the neural network is learning. What this entails is the neural network changing its own parameters in the nodes and synapses in order to produce the expected results. This might mean that some nodes have a greatly lowered effect on the final result, or that other nodes are weighted to influence the categorisation to a greater degree. This process is iterative, which means that iteration is key in order to achieve higher quality results.

Watch the robots carefully!

During training the number of incorrect conclusions the network generates should constantly decrease. However, it is also possible that a neural network’s amount of error can wildly fluctuate and even rise over the course of training.

Just like humans, neural networks can learn badly, or come to incorrect conclusions when presented with information. Thankfully in most cases, supplying more good training information to the network will overcome these issues.

What would I use Artificial Intelligence for, if not building robots?

Well, I’m glad you asked! Artificial Intelligence in the forms discussed here would most commonly be used in cases where it’s prohibitively expensive (or outright impractical) to get people to do the work for you. Replacing people with machines, if you will. This takes many forms; from data analysis, to grouping participants by any number of arbitrary criteria. Next time your bank turns you down for that new shiny credit card, don’t blame a person; it’s entirely likely the machines are already controlling your life.

As a direct consequence of neural networks’ ability to learn and adapt, they can be programmed with vast amounts of information stemming from human experiences, and in turn can be used to analyze much more data than any one human could ever read.

One (rather mundane) example of how AI is being used is in determining the most important parts of sentences. This is useful in text analysis; if you are trying to analyse a large body of text, or periodical publications (like tweets) at something approximating real time. This enables you to find out the most important (or most popular) words being used. That’s just a basic use though; IBM’s Watson is capable of achieving much more using similar approaches. It’s helping doctors with medical diagnoses, as well as winning Jeopardy on its days off!

Any last words?

As computers become more powerful and more able to handle vast amounts of data, artificial intelligence will become a much more standard way of dealing with large sets of data.

I for one, welcome our new machine overlords.