Thursday, March 3, 2016

Why Deep AI will always be deeply dumb

I have been trying to put my finger on what is the difference between a superb correlation engine and what I call "intelligence". I gather that "Deep AI" is a re-branding of multi-layer neural nets with back propagation [not sure what the means] and that, as such, it provides a correlation engine that is more or less useful. But it feels like something is missing. It is a matter of philosophy [I plan to oversimplify]. You see, there are realists and idealists.
Realists believe that facts exist in the universe and that our intelligence lies on our ability to discover the pattern of those facts. Most people tend to be realists: Scientists, computer scientists and, particularly, AI researchers trained in Cambridge MA.
Idealists, by contrast, believe that we can only discover those facts or pattern of fact in the universe that we are prepared to observe and that, through the limitation of how we perceive, we only perceive what our organic design is capable of we are designed to. An idealists might believe, for example, that "facts" are purely human facts and that, given another intelligent life form, facts could be entirely different. Where a realists might think about it and conclude that facts were finite - due to the finite number of particles and laws of the universe; at the same time an idealist would conclude that facts are infinite because, bending the universe to our perception is an ongoing process.
I (as an idealists) never quite understood why the notion that "what you see is seen through your eyes" is such a controversial idea and so repugnant to the masses. It is something which I think AI researchers must ignore with a slight pang of guilt  - because it is so obviously true.
As far as I know, the Greeks did not really think in these terms. For Plato (I guess) the ideal was better than the real and the role of the person within the universe was never considered. If I had to say, Plato would be the idealists and Aristotle would be a realists. The position and definition of idealism was clearly given by George Berkeley (my man) at a time when Newton was - presumably - a significant proponent of realism. It would be interesting to know what Leibnitz thought about this subject. The other deep thinkers of Germany, Kant and Hegel, had views which I am sure I do not understand. (At this point, I would be surprised if anyone understood what Hegel was thinking.) More recently Descarte's "I think therefore I am" and "mind versus body" suggest he might have been a realist, treating the body like a machine. At least at one time there were idealists and realists making their ideas known in England. Meanwhile, on the continent, it seems like mixed bag. Lately, the French Philosopher Bergson would need to be considered an idealist because he routinely railed against the supposed facts of our perception and tried to point out that there was more dynamic, fluid, and pre-linguistic flow of time which could never be understood through our perceptions - or at least not through what we say about our perceptions. In America, we had our own schools of thought that were a bit different. The only philosophers who I have tried to read, Pierce and James, must have been the equivalent of idealists. At least I want to think so because I like what they wrote about.
The reason for talking about this history is to put into perspective the events following World War I that brought English and continental ideas to America [and perhaps stamped out what was left of American philosophy - or drove it out of Massachusetts towards California]. I want to say that the realists and British way of thinking (as well as German bomb making) came to Cambridge MA and the prominent universities of Harvard and MIT. This limited the idealist and phenomenologists to migrating south of the Charles River into Boston and university backwaters there, like Boston University (my college), and I don't know where else. Boston University got the Hegel scholars and the Bergson experts. In Cambridge, they picked up from where the dons of Oxford and Cambridge (England) left off. I am sure that over in Boston we were given a somewhat different picture of these philosophical ideas.
The point I want to make is that because of this history, Cambridge MA became the home of the most advanced thinking in realistic philosophy. They got all the money and all the best students, while a weaker flame of idealistic philosophy flickered, not dead, across the river in Boston. So realism is the basic philosophy behind AI research. And that is what is wrong with it. I'll explain.
Leaving philosophy aside for a moment let's look at more concrete topic in engineering and mathematics: fitting model objects to data. The classic example is the "regression line" in statistics but engineers also occasionally need to fit circles or hyperbolas to data and know that each different sort of ideal object ("model") has its own methods for fitting and its own value to the user. A good engineer will tell you that the data itself is not inherently "linear" or "circular" but the question can come up as to which model is a better fit to the data - which is a more useful way to think about the data. From my own work I can tell you that you can also use piece-wise linear fitting and get better fits by allowing more and more pieces. Ultimately, you can go all the way and have a different 'piece' for every consecutive pair of data points - so that you get a meaningless zig-zagged mess that passes through each data point and tells you nothing useful. These practitioners know very well that the data, by itself, does not have any spine and that you have to choose one to use if you want to interpret the data usefully. I doubt they would all agree to the statement: that data fitting is basically an idealistic process but, with a little thought they might agree that correlations in the data, by themselves, are not particularly useful. What is useful is the correlations that exist in the idealized model, after it has been fit to the data.
It is exactly that spine which is missing from neural networks and other "deep" technologies. Leave it to the realists to go about looking to discover new patterns in the data, rather than finding pre-existant ones there. So in the end the problem with Deep AI is that it springs from a naif realism that the data contains the patterns and the untested incorrect hypothesis that knowledge is the pursuit of facts that exist in the data. I suspect smart people like Marvin Minsky started thinking about machines before they were done with childhood, and long before they had acheived any maturity in understanding the word "intelligence". With the guilty shrug of the shoulders they must have begun the grand adventure of making a better correlation machine - and have been busy with implementation details and teraflops ever since. It is too bad because if they had better philosophy they would build better machines.
I watched an episode of Chronicle (a Boston TV show highlighting one or another aspect of the New England community)  about Deep AI and they mentioned IBM's "Watson" machine. An interviewee, perhaps the director, mentioned how that machine uses human language and studies correlations there. That is potentially a far more viable approach as it looks to build up an understanding of existing human patterns to devise its engine. Good luck to them and a pox on the other "exponential growth of technology" which is frequently alluded to as bringing us to the edge of intelligent machines...soon but just not yet.

No comments:

Post a Comment