August 30, 2012
The task of parameter/function estimation has been at the center of scientific attention for a long time and it comes under different names such as filtering, prediction, beamforming, classification, regression. Conventionally, the task has been treated as an optimization task of an appropriately adopted loss function. However, in most of the cases, the choice of the loss function is mainly dictated by its mathematically tractability and not by a physical reasoning related to the specific problem at hand. The task is further complicated when a-priori information, in the form of constraints, becomes available. The presence of constraints in estimation tasks is recently gaining in importance, due to the revival of interest in robust learning schemes.
In this talk, the estimation task is treated in the context of set theoretic estimation arguments. Instead of a single optimal point, we are searching for a set of solutions that are in agreement with the available information, which is provided to us in the form of a set of training points and a set of constraints.
The goal of this talk is to present a general tool for parameter/function estimation, both for classification as well as regression tasks, in a time adaptive setting in (infinite dimensional) Reproducing Kernel Hilbert spaces (RKHS). The general framework is that of convex set theory via the powerful and elegant tool of projections.
The structure of this talk evolves along the following directions:
It presents in simple geometric arguments the basic principles behind the convex set theoretic approach via projections in the generalized online setting. In contrast to the classical POCS theory, developed for a fixed number of convex sets, we consider convex sets which are built “around” the training data, and thus their number increases as time evolves and new data samples are received.
It presents two case studies of particular interest in the adaptive learning community:
On line classification
Adaptive constrained regression in the context of robust beamforming
The resulting algorithms are of linear complexity with respect to the number of unknown parameters and are supported by strong convergence results.
The work has been carried out in cooperation with Isao Yamada and Kostas Slavakis.
Talk slides in pdf [~3,3MB]http://www.iit.demokritos.gr/docs/seminars/theodoridis-slides.pdf