I recently made public more of my programming projects from the past. My github profile now includes:
- luacnn - Convolutional Neural Network for hand digit recognition using Torch7 and Lua.
- metrics - a Torch7 package to compute some metrics, such as area under ROC.
- lakeml - some Machine Learning algorithms implemented in C++, e.g. AdaBoost, K-means, EM for (diagonal) Gaussian Mixture Models.
- partitracker - a simple Particle Filter implemented in C++ and OpenCV.
- sudoku - a Sudoku puzzle solver in pure C.
- kdtree - a K-d tree C++ implementation, for Nearest Neighbours search.
- likewise - a simple Python webserver for image similarity search, using the embeddings from a pre-trained convnet.
- fractals - Zoomable Mandelbrot fractal renderings using PyGame.
- acm_utils - Data structures and algorithms useful for programming competitions (trie, balanced tree, big ints, suffix arrays, etc.) and other small C or C++ pearls.
It turns out that there are several very high profile Portuguese researchers in the field of Machine Learning, even though they all seem to live and work abroad. Here are, to the best of my knowledge, the most cited ones, including their current affiliation and the total citations according to Google Scholar (as of Sept. 2015):
- Fernando Pereira (Google, USA) citations: 32496
- Pedro Domingos (University of Washington, USA) citations: 27248
- Manuela Veloso (Carnegie Mellon University, USA) citations: 22405
- Nando de Freitas (Oxford University and Google DeepMind, UK) citations: 18439
- José Príncipe (University of Florida, USA) citations: 16823
- Nuno Vasconcelos (University of California San Diego, USA) citations: 8123
I have recently started working as a Research Engineer at Google DeepMind, in London. It’s very exciting to be surrounded by some of the best researchers in Machine Learning in the world, with such an ambitious mission statement and the right conditions to pursue it. I will be working in Deep Learning and Reinforcement Learning real world applications with positive impact in society.
Go ahead and read some of the coolest publications from my colleagues.
Yesterday I attended a talk organized by the London Machine Learning meetup group, where Yoshua Bengio was the invited speaker. Not surprisingly, there were about 200 people attending.
Yoshua reinforced the idea that a lot of the success of learning algorithms for AI tasks comes from incorporating meaningful priors. These should be general enough to hold true in a wide range of applications, but also specific enough to vastly reduce the amount of training data needed to achieve good generalization. This reminded me of a previous post I wrote in this blog, almost 5 years ago!
In the meanwhile, deep learning became main-stream, and Yohua slides high-lighted several theoretical progresses that were made, e.g:
- Expressiveness of deep networks with piecewise linear activation functions: exponential advantage for depth (Montufar et al NIPS 2014)
- Theoretical and empirical evidence against bad local minima (Dauphin et al NIPS 2014)
- Manifold and probabilistic interpretations of auto-encoders:
I recently started playing with the torch library again. Torch7 is now a growing set of packages, managed by luarocks. I really like this approach because it forces torch contributors to make their code more modular and re-usable.
So far, I have done a few very simple contributions to the torch ecosystem:
1) Extended the matio package, which reads MAT files, to support structs, cell arrays and strings, in addition to loading tensors, which was already implemented:
2) Started working on a metrics package for torch, which will compute things like ROC (Receiver Operator Curve) and Confusion Matrices. Probably dozens of other people have written similar code for torch, but I couldn’t find an authoritative package doing just that. So, I started one at:
3) Finally, I did a small refactoring on my example project that uses convolutional neural networks for handwritten digit recognition. The code is now split into two files: one to load the USPS dataset and another one to create the network, train it and evaluate it. This makes it easier for new people to understand the example.
Ten years after my Summer internship at Microsoft as an undergraduate student, I had the opportunity to be back to the Redmond campus. This time the goal was to attend Microsoft Research’s techfest and participate in the Practice of Machine Learning conference. I can’t share much about what I saw there, but I must say I was impressed by the amount of great talks and cool demos. MSR is really a leading research institution and lots of theory and cutting-edge applications keep on emerging, most notably in many Bing products. It’s great to be part of this community.
It was also a good occasion to see some old friends, which made my stay even richer.
Looking forward to be back!
Nassim Nicholas Taleb (author, scholar, statistician, trader, philosopher, risk manager)
Books: The Black Swan, Antifragile: Things That Gain from Disorder
Bret Victor (engineer, designer, inventor, dreamer):
Talks: Inventing on Principle, The Future of Programming
Essay: Learnable Programming
Salvatore Sanfilippo (programmer, I mean rock-star programmer):
Software: redis (source code)
Soon I will be in Dublin, Ireland, attending one of the best conferences in Information Retrieval. Looking forward to hearing about the latest research and probably meeting some old friends.
A couple of years ago, I was doing research involving Convolutional Neural Networks for object classification tasks (see here). At that point, deep learning was an (re)emergent field, but it hadn’t achieved mainstream yet.
Today, several factors are contributing to the increase of visibility:
- Announcements of great results in Kaggle competitions and commercial applications, such as Google’s improved photo search and Microsoft’s fast and robust speech recognition system.
- Online courses, such as Geoffrey Hinton’s “Neural Networks for Machine Learning” in Coursera.
- Greater variety of software implementations, including many that use the power of GPU parallelism.
What will be the next milestones?
I decided to move from blogger to a self-hosted wordpress installation, which gives me much more control. I had some issues while automatically importing all posts and images, but after some manual steps, I got it to work and the blog looks much better now!
Now you can read me at http://blog.hpenedones.org