Neural Network failure

I have deleted the posts regarding the neural network class as the class didn't work - I couldn't solve the XOR problem.

Adaptive Protection

One thing that has fascinated me for the last few years has been the thought that instead of these centralised, hub like antivirus (and anti-malware, anti-spyware, etc.) systems and firewalls that work on a per computer/network basis, we could have an adaptive peer to peer system.

At this point in time I think there are three major uses for the internet. Number one is business, number two is socialising and number three is sharing data. Although I suppose three encapsulates both one and two.

The only downsides to the internet are crime and government. I'm not about to try and fix governance of the internet except to say that peer to peer trumps centralised control, and I'll hopefully convince you of that in this blog, if not this post.

There are two types of crime on the internet; mostly on the web and in email, but they also feature in other applications. The first is digital crime like trojans, spyware, etc. The second type is confidence tricks, like when on ebay someone sells you a DVD allegedly signed by Ron Jeremy and it's just a blank DVD with a squiggle on it, and there's nothing to prove it is what it is. The latter type of crime is rife in a walks of life. Digital crime, however, can only happen on computers.

So now we get to the point of my post: adaptive protection from digital crime.

Our current model for protection is one where we install some software, be it a firewall, script blocker, antivirus programme or whatever and allow it to run. Every so often the software will call home, either automatically, or because we tell it too, and it will update how it works. This is what we do and it means that we are always one step behind the attackers.

Let me throw some crazy, metaphorical idealism at you.

The internet is like fertiliser. It's not a living thing its self, it doesn't change as such. Its purpose is always to be the place where ideas can grow and evolve. The applications of the internet: bit torrent, web, ftp, usenet, gopher, finger, etc., are all lifeforms that evolve, or become endangered or extinct (to over extend the metaphor). Essentially we (users) behave like seed carriers; we increase the population of an application by spreading it around, getting others to use it, a lot like corn or carrots.

The other applications, the ones that are less than favourable for most users, are also able to flourish in this fertiliser. Like weeds, I suppose, or maybe bacteria or fungi. So like sensible farmers we deploy pesticides to kill them off, which work fine until they evolve and we have to make a better pesticide, and so on and so on.

We have one advantage over bio-technologists, though. If we start doing mad scientist type experiments to try and create programmes that can behave in a way that is adaptive and can help wipe out the undesirable programmes, we can't cause ecological disaster, like DDT or engineering resistant plants might.

My basic premise is that the best way to beat the viruses et al. is through co-operation and on the fly adaptation, via implementing a new internet application. Rather than an application that considers the internet as a place to guard against, create an internet application that wants to defend its turf.

It would have to be an application that runs in the background of your computer, and is able to understand what should be happening and what shouldn't, and how to prevent these things. Once recognition of a problem takes place and action is taken then the programme would need to propagate that information to neighbours to help them identify similar problems.

You cannot define all the rules up front. You can, however, define how to define rules, and let evolution take its course. This is one area I think that evolutionary computing will excel in. It is perfectly suited to coming up with solutions that involve iterative design over many generations.

I haven't done enough reading around operating systems to be able to implement such a system, but the idea is most intriguing. I also haven't read around in the adaptive protection literature, so I don't know how far along this research is. It is definitely something I will be getting involved in.

It seems I'm not very good at keeping on track.

I keep getting distracted by things and don't do enough work on my AI stuff.

It's not that I haven't done any work, but I've started a new project and I haven't progressed at all on the mini project. This is partly due to not having unpacked my Linux box after moving house, but also because my social life has improved after moving.

The new project isn't getting much work done on it either, but I don't want to discuss it yet, as it in the really really early stages. It's currently being developed on Windows XP, but it should be easy enough to port it to Linux when the time comes.

Merry Christmas to anyone reading this!

Minor Update

It's been a lot longer than I had planed since my last post.

I have made steps towards a neural network for solving sudoku, but I hit a snag when I came to generating test sudoku on the fly, then I got distracted with other AI research I'm doing and updating my website and stuff going on in my non-digital life.

I haven't got beyond my snag, so I have no real update, other than to say I have created a simple class for single layer neural networks

My other AI research has lead me to investigate Information Theory. I'm finding it very complicated so far, not having a degree in mathematics, but I'm sure I'll figure it out in time.

For now, you'll have to wait, while I decide if I should use canned sudoku grids or work to finish my generator.

New Mini Project

It's been a long while since I've been motivated to do any development.

I've finally come up with a small enough project that I should be able to do over a few weeks.

My main project: finishing the robot football AI, will be put on hold while I get back into the swing of C++ and Linux (slackware).

This mini project is a very simple neural network that will be limited to two layers (no need for complex learning algorithms). I will train the network to recognise sudoku puzzles, and hopefully I'll be able to get it to fill in novel puzzles.

The network code will be a lot simpler than the code I used for my Masters' thesis, with two one dimensional arrays of doubles for the input and output, and a single two dimensional array of doubles for the connections.

I'll post the code as bits get completed.

3D thing source

OK! the source can be found at http://www.matthewellen.co.uk/index.php?page_name=portfolioitem&item=dynamicmotion

Unveiling

Ok! It's done.

It's not quite how I said it would be. There are no collisions, but everything else is there. All the nodes have the same mass, so there was no need to include that in the model.

To download the finished product goto http://www.geocities.com/griffle_the_waffle/ppi and click on the products link in the left hand menu. The first product is the one you want, although you might want to try out the others too.

I'll post links to the source in a bit.

Warm up exercises

Right, today I'm going to start a simpler project:

I'm going to try to make an l-system style programme, but without any branches. Also each node will be of a certain type, that will be attracted to one type and repel its self from another. The attraction/repulsion will cause acceleration based on the mass of the nodes.

The nodes will be able to collide with eachother, and will also be enclosed within a cube, so they will bounce of the sides of that too.

The l-system part will be how I control the number of nodes - each type will beget two of two other types. You will be able to increase and decrease the number of nodes as whole layers, in a binary way i.e. 1, +2, +4, +8, etc.

I know I'm meant to be making AI software, but this idea excites me more than thoroughly designing a neural network designer.

The idea struck me after conversing with Baz of Elitebastards fame. He introduced me to an l-system that used repelling to form its shape. It looked pretty cool, so much so I wanted to do something like it.

Starting afresh

Post 1: the least important post.

This is the start of my blog. I'm creating a blog about developing AI software. If you have no interest in AI software, or me, turn back now. This will only make you turn blind.

Since this is the first post, I will tell you what I shall be developing first: An Artificial Neural Network Designer.

I have already taken the first steps, I have decided what it will be able to do:

• Load a network
  
• Save/save as
• Create a new network
• Add a neuron
• Remove a neuron
• Edit neuron properties
• Add a neuron group
• Remove a neuron group
• Edit group properties
• Add a neuron to a group
• Remove a neuron from a group
• Add connections: N->N, N->G, G->N, G->G
• Removing connections
• Create activation functions (AFs)
• Edit AF properties
• Remove AFs
• Set network bias

And since I want to get things right this time, I'm going full force with UML.

Anyway, wish me luck.