Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain

N. N. Schraudolph and T. J. Sejnowski. Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain. In Advances in Neural Information Processing Systems (NIPS), pp. 499–506, Morgan Kaufmann, San Mateo, CA, 1993.
In Ph.D. thesis

Download

122.4kB	101.2kB	85.5kB

Abstract

We present the information-theoretic derivation of a learning algorithm that clusters unlabelled data with linear discriminants. In contrast to methods that try to preserve information about the input patterns, we maximize the information gained from observing the output of robust binary discriminators implemented with sigmoid nodes. We derive a local weight adaptation rule via gradient ascent in this objective, demonstrate its dynamics on some simple data sets, relate our approach to previous work and suggest directions in which it may be extended.

BibTeX Entry

@inproceedings{SchSej93,
     author = {Nicol N. Schraudolph and Terrence J. Sejnowski},
      title = {\href{http://nic.schraudolph.org/pubs/SchSej93.pdf}{
               Unsupervised Discrimination of Clustered Data
               via Optimization of Binary Information Gain}},
      pages = {499--506},
     editor = {Stephen Jos{\'e} Hanson and Jack D. Cowan and C. Lee Giles},
  booktitle =  nips,
  publisher = {Morgan Kaufmann, San Mateo, CA},
     volume =  5,
       year =  1993,
   b2h_type = {Top Conferences},
  b2h_topic = {>Entropy Optimization},
   b2h_note = {In <a href="b2hd-Schraudolph95">Ph.D. thesis</a>},
   abstract = {
    We present the information-theoretic derivation of a learning algorithm
    that clusters unlabelled data with linear discriminants.  In contrast to
    methods that try to preserve information about the input patterns, we
    maximize the information gained from observing the output of robust binary
    discriminators implemented with sigmoid nodes.  We derive a local weight
    adaptation rule via gradient ascent in this objective, demonstrate its
    dynamics on some simple data sets, relate our approach to previous work
    and suggest directions in which it may be extended.
}}