On Kolmogorov Complexity

This semester I assisted to a course on Kolmogorov Complexity and Minimum Description Length, given at the Institute for Logic, Language and Computation of the University of Amsterdam.
The main lecturer, Paul Vitanyi, is the c0-author of THE book on Kolmogorov Complexity. Peter Gr├╝nwald, invited professor in two lectures, is the author of THE book on the Miniumum Description Length principle. The exercise sessions were given by a talented PhD student of them, Wouter Koolen-Wijkstra.

But what is this all about?
In simple words, the Kolmogorov Complexity of a sequence of characters is the length of the shortest program that outputs that sequence. It is the theoretical limit of compression.

On Intelligence by Jeff Hawkins

I would like to recommend this book by Jeff Hawkins, in which the author tries to create a theory about the neocortex.

He claims that the neocortex is basically a hierarchical memory system able to detect temporal and spatial patterns. Jeff Hawkins, and his company Numenta, are now trying to move forward and implementing this “neocortical algorithm” as software running on a computer.

I enjoyed a lot reading it and I am trying now to read the technical papers. So far it looks like a good model, specially for computer vision systems, but it’s not yet clear to me how to solve problems from other cognitive areas such as language processing or planning.
More posts on that for the coming weeks!
Update [2013]: 5 years later, nothing very impressive has come out of Numenta. Even though the ideas on this book are appealing, in practice, all solid Machine Learning results require: 1) a very clear loss function, 2) an efficient optimization algorithm and 3) preferably, lots of data.

Artificial General Intelligence

Back in 1956, the founders of the new AI research field (John McCarthy, Marvin Minsky, Allen Newell and Hebert Simon) were deeply convinced that in a period of one generation we would have human-level intelligent computers.
However, after more than 50 years, we are still not able to solve some tasks that humans do without any apparent effort (such as distinguishing a dog from a cat or a horse in any kind of picture). Many frustrating results mark the history of AI: low quality of (early) machine translation systems, lack of robustness of speech recognition and computer vision systems, etc.
The so called “AI winter” is generally perceived to be finished by now, since many researchers have new hopes on building Artificial General Intelligence. Recent contributions from both neuroscience and theoretical computer science were decisive to create this optimism.
Here is a book edited by Ben Goertzel and Cassio Pennachin putting together several of the different renewed ideas.

As I read it, I will post comments on individual chapters concerning different approaches to AGI.