The statistical learning theory is a probabilistic theory. It considers systems capable of
learning, i.e. to adapt to the environment based on samples of environmental data. The paradigm of
learning theory is data classification or pattern recognition (like medical diagnosis, manuscript
characters recognition, etc). If the examples are previously classified (by an expert), learning is
called "supervised". If the classifiaction is based only on regularities of the data detected by
the learner itself, learning is "unsupervised". The aim is to give correct answers -to classify
correctly- new incoming signals not previously used for learning. This property is called
generalization. The theory allows to determine under which conditions generalization is possible.
Statistical physics considers the same framework, and predicts the typical generalization
performance on well defined tasks in the limit of a large number of high dimensional data.
Depending on the the type of problem and on the learning algorithm, the existence of phase
transitions of second and first order between low-performance and high-performance learning phases
have been discovered. We review some recent results obtained with the statistical mechanics
approach, and discuss some open questions.
|