Accelerated Training of Conditional Random Fields with Stochastic Gradient Methods

S. Vishwanathan, N. N. Schraudolph, M. W. Schmidt, and K. Murphy. Accelerated Training of Conditional Random Fields with Stochastic Gradient Methods. In Proc. 23rd Intl. Conf. Machine Learning (ICML), pp. 969–976, ACM Press, 2006.

Download

pdf djvu ps.gz
1.1MB   185.3kB   1.4MB  

Abstract

We apply Stochastic Meta-Descent (SMD), a stochastic gradient optimization method with gain vector adaptation, to the training of Conditional Random Fields (CRFs). On several large data sets, the resulting optimizer converges to the same quality of solution over an order of magnitude faster than limited-memory BFGS, the leading method reported to date. We report results for both exact and inexact inference techniques.

BibTeX Entry

@inproceedings{VisSchSchMur06,
     author = {S.~V.~N. Vishwanathan and Nicol N. Schraudolph
               and Mark W. Schmidt and Kevin Murphy},
      title = {\href{http://nic.schraudolph.org/pubs/VisSchSchMur06.pdf}{
               Accelerated Training of Conditional Random Fields
               with Stochastic Gradient Methods}},
      pages = {969--976},
     editor = {William Cohen and Andrew W. Moore},
  booktitle = {Proc.\ 23$^{rd}$ Intl.\ Conf.\ Machine Learning (ICML)},
  publisher = {ACM Press},
       year =  2006,
   b2h_type = {Top Conferences},
  b2h_topic = {>Stochastic Meta-Descent},
   abstract = {
    We apply Stochastic Meta-Descent (SMD), a stochastic gradient
    optimization method with gain vector adaptation, to the training
    of Conditional Random Fields (CRFs). On several large data sets,
    the resulting optimizer converges to the same quality of solution
    over an order of magnitude faster than limited-memory BFGS, the
    leading method reported to date. We report results for both
    exact and inexact inference techniques.
}}

Generated by bib2html.pl (written by Patrick Riley) on Thu Sep 25, 2014 12:00:33