I moved to London to work in the Autosuggest Core Relevance team for www.bing.com. Many new interesting challenges lie ahead, as I am going to work with text and information retrieval, instead of images and computer vision. My background in CS algorithms and Machine Learning will be very useful, though.
Yesterday I attended to this workshop at EPFL:
It was a good opportunity to see old friends and colleagues, and listen about their latest research. In general, the quality of the talks was quite good, ranging from very theoretical machine learning (sparse coding, optimization, etc.) to commercial applications of computer vision (www.faceshift.com).
Somewhere in the middle of that spectrum, I also quite liked the talk about learning image local descriptors (BRIEF and LBGM) as a compact and efficient alternative to SIFT or SURF, which are hand-designed, slower and use more bits. There were also applications to speech, face analysis and even remote sensing.
Have a look at the program and keep an eye on it in the coming days, as the slides will probably become available. You will find several other interesting talks:
Lately, I have been working with Deformable Models and I am surprised by how well they can work.
In the video above I am using an Inverse Compositional Active Appearance Model, which was trained with images of myself. It’s specially tuned for my face, but I still find it quite impressive how well it can track my face in realtime!
On the other hand, this model is quite sensitive to lighting conditions and partial occlusions. Training it, is also somehow of an art, because, as opposed to discriminative models, increasing the amount of training data might actually decrease performance. This happens because we use PCA to learn the linear models of shape and texture, which will degrade if data has too much variation or noise.
Still, it’s quite impressive what one can achieve by annotating a few images (about 50, in this case). In addition, as one annotates images, one can start training models that will help us landmark the next ones (in a process of “bootstrapping”, similar to the one in compilers).
I recently read an article from Y. Bengio and Y. LeCun named “Scaling Learning Algorithms to AI”. You can also find it as a book chapter in “Large-Scale Kernel Machines”L. Bottou, O. Chapelle, D. DeCoste, J. Weston (eds) MIT Press, 2007.
In some aspects it is an “opinion paper” where the authors advocate for deep learning architectures and their vision of the Machine Learning. However, I think the main message is extremely relevant. I was actually surprised to see how much it agrees with my own opinions.
Here is how I would summarize it:
- no learning algorithm can be completely universal, due to the “No free lunch theorem”
- that’s not such a big problem: we don’t care about the set of all possible functions
- we care about the “AI set”, which contains the functions useful for vision, language, reasoning, etc.
- we need to create learning algorithms with an inductive bias towards the AI set
- the models should “efficiently” represent the functions of interest, in terms of having low Kolmogorov complexity
- researchers have exploited the “smoothness” prior extensively with non-parametric methods. However many manifolds of interest have strong local variations.
- we need to explore other types of priors, more appropriate to the AI set.
The authors then give examples of two “broad” priors, such as the sharing of weights in convolutional networks (inspired by translation invariance in vision) and the use of multi-layer architectures (which can be seen as levels of increasing abstraction).
Of course here is where many alternatives are open! Many other useful inductive-bias could be found. That’s where I think we should focus our research efforts!
The names of the participants in the room was no less impressive, (by no special order): Corinna Cortes (Google), Rob Schapire and David Blei (Princeton University), John Langford and Alex Smola (Yahoo), Yann LeCun (NYU), Sanjoy Dasgupta (Univ. California), Michael Collins (MIT), Patrick Haffner (AT&T), among many others.
In the last few months (in which I haven’t posted in this blog) there were a few interesting web platforms related to Machine Learning poping-up, most notably:
Just a short link to an article of the New York Times about AI.
In 1978, Dr. McCarthy wrote, “human-level A.I. might require 1.7 Einsteins, 2 Maxwells, 5 Faradays and .3 Manhattan Projects.”
I think we probably have the genius scientists around, but not so sure about the 0.3 Manhattan Projects!
Update: You might also want to read latest Shane Legg’s predictions about human level artificial intelligence.
Today I assisted to the first edition of TEDx Geneva. This was a locally-organized event following the same spirit of the original TED talks: “ideas worth spreading”.
I think the program was really good, because in this region there are some many incredible organizations. He could listen to people from CERN, EPFL, the United Nations, the Red Cross and some independent Swiss adventurers and entrepreneurs. We also had the opportunity to (re)watch some videos of the most popular TED talks recorded in the US.
Update: In addition to C++ (and C which I really love), I also started using some scripting languages. First I learnt Lua, so that I could use the Torch Machine Learning Library. Then, I started using python, which I really love due to the wide availability of (easily installable) libraries. Ah, I look forward to learn the new C++11 standard, which seems to be quite neat.
cmake - it’s cross platform and simple enough to start using it. I don’t know the advanced features, but it’s pretty easy to create a project that generates libraries and executables and links properly with other dependencies (like OpenCV).
Cloud Computing (new):
Amazon EC2 - I also used the IBM Smart Cloud, but Amazon has more features and better APIs. Recently, with the introduction of the spot instances, things also got a lot cheaper when you need to process large amounts of data.
NoSQL Databases (new):
redis - redis is what we can call a “data structure server” and it’s probably the nicest piece of software I started using recently. It is just beautiful. Simple. Intuitive. Fast. I can not recommend it enough.
Computer Vision Library: