So you lost a needle in a haystack. That is no problem at all, technically. Just pick up every straw, carefully, examine it thoroughly and continue until you found the needle. Of course this is boring – so delegate it to somebody who gets paid for work like this. He will hate you.
Better use one of the new deep learning machines. First you will describe what is meant to be done. “Needle in haystack” is probably too generic. Say “examine every single object in this pile of things, put dry grass on a new pile and then stop at every metallic object with a pointed tip.”
Given that the basic concepts of “take” and “examine” are already available, the AI will start its operation and probably stops several times to ask “is this grass” or “is it pointed” but eventually it renders the lost needle. Within a certain probability. If not, you can always repeat this with different parameters – like maximum length or an additional test for flexibility.
After that you could find millions of needles in millions of haystacks.
But what if you want to find the front light of a Lego car in a box of Lego bricks? You could use the same AI, but you will have to train it again. Then you could probably use it to find Lego parts in haystacks.
This is AI. It is basically “just” very sophisticated pattern matching. Your language contains a patterns that have to be detected. The resulting task is: comparing patterns. The physical action consists of transforming the visual input patterns into coordinated movements.
And the nice thing here: Deep learning avoids having to provide all those patterns explicitly, the AI has to sort them out by itself. This is serious math, computing and a big portion of magic – but well covered under the hood. I wouldn’t dare trying to understand it.
So much for background. As always I am looking more at the practical side of new technology.
As a start I did a quick scan on deep learning APIs, from Google and others. Will it be easy to integrate AI into our frameworks as a plugin, just like the usual SQL or non-SQL databases and search engines? I believe it will, at least structurally.
This is what I expect: Seen from the high level of an application the AI integration boils down to quite simple and common tasks:
- Batching copies of incoming data into the AI Service – be it local or cloud-based. This phase probably needs some attention and well-thought parameters. Hopefully very little transformation. And probably some “external” backtracking capabilities so that changes in parameter values will re-feed the training process starting at given points. That will become interesting – from the aspect of workflow, traditional databases don’t need that.
- Then there will be some queries that will have different languages, but generally follow the usual concept: Feed the engine a text and get a result. Also common practice, but the query languages will be highly individual.
- Finally the answer: Consists mostly of references to previously fed input and query-specific metadata, like probabilities or subquery suggestions. All in all quite straightforward – from a user’s perspective.
Well, with a little luck I will find some projects that justify the effort of building these plugins and using all of this. Which will – as always – show some unexpected aspects, but that is what makes new technologies interesting.
So after all there seems to be no big mystery in AI today, as seen from the outside. I’d say: Also nothing to be afraid of. Unless the only job you are able to do is finding needles in haystacks.
Or if somebody decides to train the AI to kill every terrorist in a crowd of people. Even then I wouldn’t blame the AI.