Online Independent Component Analysis With Local Learning Rate Adaptation

N. N. Schraudolph and X. Giannakopoulos. Online Independent Component Analysis With Local Learning Rate Adaptation. In Advances in Neural Information Processing Systems (NIPS), pp. 789–795, The MIT Press, Cambridge, MA, 2000.

Download

143.5kB	64.6kB	65.9kB

Abstract

Stochastic meta-descent (SMD) is a new technique for online adaptation of local learning rates in arbitrary twice-differentiable systems. Like matrix momentum it uses full second-order information while retaining O(n) computational complexity by exploiting the efficient computation of Hessian-vector products. Here we apply SMD to independent component analysis, and employ the resulting algorithm for the blind separation of time-varying mixtures. By matching individual learning rates to the rate of change in each source signal's mixture coefficients, our technique is capable of simultaneously tracking sources that move at very different, a priori unknown speeds.

BibTeX Entry

@inproceedings{SchGia00,
     author = {Nicol N. Schraudolph and Xavier Giannakopoulos},
      title = {\href{http://nic.schraudolph.org/pubs/SchGia00.pdf}{
               Online Independent Component Analysis With Local
               Learning Rate Adaptation}},
      pages = {789--795},
     editor = {Sara A. Solla and Todd K. Leen and Klaus-Robert M\"uller},
  booktitle =  nips,
  publisher = {The {MIT} Press, Cambridge, MA},
     volume =  12,
       year =  2000,
   b2h_type = {Top Conferences},
  b2h_topic = {>Stochastic Meta-Descent, >Entropy Optimization},
   abstract = {
    Stochastic meta-descent (SMD) is a new technique for online adaptation 
    of local learning rates in arbitrary twice-differentiable systems.  Like
    matrix momentum it uses full second-order information while retaining 
    O(n) computational complexity by exploiting the efficient computation
    of Hessian-vector products.  Here we apply SMD to independent component
    analysis, and employ the resulting algorithm for the blind separation of 
    time-varying mixtures.  By matching individual learning rates to the rate
    of change in each source signal's mixture coefficients, our technique is
    capable of simultaneously tracking sources that move at very different,  
    {\em a priori}\/ unknown speeds.
}}