- luacnn - Convolutional Neural Network for hand digit recognition using Torch7 and Lua.
- metrics - a Torch7 package to compute some metrics, such as area under ROC.
- lakeml - some Machine Learning algorithms implemented in C++, e.g. AdaBoost, K-means, EM for (diagonal) Gaussian Mixture Models.
- partitracker - a simple Particle Filter implemented in C++ and OpenCV.
- sudoku - a Sudoku puzzle solver in pure C.
- kdtree - a K-d tree C++ implementation, for Nearest Neighbours search.
- likewise - a simple Python webserver for image similarity search, using the embeddings from a pre-trained convnet.
- fractals - Zoomable Mandelbrot fractal renderings using PyGame.
- acm_utils - Data structures and algorithms useful for programming competitions (trie, balanced tree, big ints, suffix arrays, etc.) and other small C or C++ pearls.

- Fernando Pereira (Google, USA) citations: 32496
- Pedro Domingos (University of Washington, USA) citations: 27248
- Manuela Veloso (Carnegie Mellon University, USA) citations: 22405
- Nando de Freitas (Oxford University and Google DeepMind, UK) citations: 18439
- José Príncipe (University of Florida, USA) citations: 16823
- Nuno Vasconcelos (University of California San Diego, USA) citations: 8123

]]>

Go ahead and read some of the coolest publications from my colleagues.

]]>Yoshua reinforced the idea that a lot of the success of learning algorithms for AI tasks comes from incorporating meaningful priors. These should be general enough to hold true in a wide range of applications, but also specific enough to vastly reduce the amount of training data needed to achieve good generalization. This reminded me of a previous post I wrote in this blog, almost 5 years ago!

In the meanwhile, deep learning became main-stream, and Yohua slides high-lighted several theoretical progresses that were made, e.g:

- Expressiveness of deep networks with piecewise linear activation functions: exponential advantage for depth (Montufar et al NIPS 2014)
- Theoretical and empirical evidence against bad local minima (Dauphin et al NIPS 2014)
- Manifold and probabilistic interpretations of auto-encoders:

- Estimating the gradient of the energy function (Alain and Bengio ICLR 2013)
- Sampling via Markov chain (Bengio et al NIPS 2013)
- Variational auto-encoder breakthrough (Gregor et al arXiv 2015)

Enjoy reading!

]]>So far, I have done a few very simple contributions to the torch ecosystem:

1) Extended the matio package, which reads MAT files, to support structs, cell arrays and strings, in addition to loading tensors, which was already implemented:

https://github.com/hpenedones/matio-ffi.torch

2) Started working on a metrics package for torch, which will compute things like ROC (Receiver Operator Curve) and Confusion Matrices. Probably dozens of other people have written similar code for torch, but I couldn’t find an authoritative package doing just that. So, I started one at:

https://github.com/hpenedones/metrics

3) Finally, I did a small refactoring on my example project that uses convolutional neural networks for handwritten digit recognition. The code is now split into two files: one to load the USPS dataset and another one to create the network, train it and evaluate it. This makes it easier for new people to understand the example.

https://github.com/hpenedones/luacnn

Enjoy it!

]]>It was also a good occasion to see some old friends, which made my stay even richer.

Looking forward to be back! ]]>

Books: The Black Swan, Antifragile: Things That Gain from Disorder

**Bret Victor** (engineer, designer, inventor, dreamer):

Talks: Inventing on Principle, The Future of Programming

Essay: Learnable Programming

**Salvatore Sanfilippo** (programmer, I mean rock-star programmer):

Software: redis (source code)

Today, several factors are contributing to the increase of visibility:

- Announcements of great results in Kaggle competitions and commercial applications, such as Google’s improved photo search and Microsoft’s fast and robust speech recognition system.
- Online courses, such as Geoffrey Hinton’s “Neural Networks for Machine Learning” in Coursera.
- Greater variety of software implementations, including many that use the power of GPU parallelism.

What will be the next milestones?

]]>Now you can read me at http://blog.hpenedones.org

]]>