Speech and Natural Language Processing

A curated list of speech and natural language processing resources. Other lists can be found in this list(link is external). If you want to contribute to this list (please do), send me a pull request. All Sub-caterogires are listed in alphabetical order

Finite State Toolkits and Regular Expressions

• AT&T FSM Library(link is external) The AT&T FSM libraryTM is a set of general-purpose software tools available for Unix, for building, combining, optimizing, and searching weighted finite-state acceptors and transducers.
• Carmel(link is external) Finite-state toolkit, EM and Bayesian (Gibbs sampling) training for FST and context-free derivation forests/
• Categorial semiring(link is external) Categorial semiring as described in Sproat et al. 2014(link is external)
• dk.brics.automaton(link is external) Java toolkit for FSAs and regular expression.
• Fare(link is external) Fare is a finite state and regular expression libary for the .NET framework written in C#.
  am is a JavaScript library for working with automata and formal grammars for
  regular and context-free languages
• Foma(link is external) Finite-state compiler and C library
• fsa(link is external) Toolkit used in RWTH ASR engine
• fsm2.0(link is external) Thomas Hanneforths fsm 2.0 library written C++ has a few nice operations
  such as three-way composition
• fstrain(link is external) A toolkit for training finite-state models
• jopenfst(link is external) Java port of the C++ OpenFst library; originally forked from the CMU Sphinx project
• Kleene programming language(link is external) High level finite state programming language built on top of OpenFst.
• MIT FST Toolkit(link is external) WFST toolkit no
  maintained anymore but feature a few commands not found in other toolkits
• MoMs-for-StochasticLanguages(link is external) Spectral and other training algorithms for WFSAs.
• n Shortest Path for PDT(link is external) n Shortest Path for PDT
• Noam(link is external)
  "Noam is a JavaScript library for working with automata and formal grammars for regular and context-free
  languages". Also has pretty cool examples using viz.js(link is external)
• OpenFst(link is external) OpenFst is a library for constructing, combining, optimizing, and searching weighted finite-state transducers (FSTs).
• openfst-utils(link is external) Nice set of
  utilities for OpenFst includes implementation of Categorial
  semirings.openfst-utils.
• openlat(link is external) Toolkit for manipulating word
  lattice built on top of OpenFst. Includes support for reading and writing HTK
  compatible lattices.
• PyFst(link is external) Python interface to OpenFst
• SFST - Stuttgart Finite State Transducer Tools(link is external)
  "SFST is a toolbox for the implementation of morphological analysers and other
  tools which are based on finite state transducer technology."
• Treba(link is external) "Treba is a basic command-line tool for training, decoding, and calculating with weighted (probabilistic) finite state automata (PFSA) and Hidden Markov Models (HMMs)."

Many of the toools in the machine translation section also implement interesting graph and semiring operations.

Language Modelling Toolkits

• Bayesian Recurrent Neural Network for Language Modeling(link is external) This is a C/C++ implementation for Bayesian recurrent neural network for language modeling (BRNNLM)
• Berkeley LM(link is external)
• Bigfatlm(link is external)
  Provides Hadoop training of Kneser-ney language models, written in Java.
• CSLM(link is external) "Continuous Space Language
  Model toolkit. CSLM toolkit is open-source software which implements the so-called continuous
  space language model.
• DALM(link is external) Double array language model.
• KenLM(link is external)
  Kenneth Heafield's language model toolkit, uses a very fast and low memory
  representation.
• lwlm(link is external) lwlm is an exact, full Bayesian implementation of the Latent Words Language Model (Deschacht and Moens, 2009).
• Maximum Entropy Modeling(link is external)
  Le Zhang has a comprehensive set of links related MaxEnt models.
• Maximum entropy language models: SRILM extension(link is external)
  "This patch adds the functionality to train and apply maximum entropy (MaxEnt)
  language models to the SRILM toolkit. Currently, only N-gram features are
  supported"
• mitlm(link is external)
  My personal favourite LM toolkit, super fast and seems to get slightly higher
  accuracy.
• MSRLM(link is external)
  "This scalable language-model tool is used to build language models from large
  amounts of data. It supports modified absolute discounting and Kneser-Ney
  smoothing."
• OpenGrm(link is external)
  Language modelling toolkit for use with OpenFst.
• cpyp(link is external) C++ library for modeling with Pitman-Yor processes
• RandLM(link is external) Bloom filter based random language models
• RNNLM(link is external)
  Recurrent neural network language model toolkit.
• Refr(link is external)
  Re-ranking framework from the Johns-Hopkins workshop on confusion language
  modelling.
• rwthlm(link is external) A toolkit for training neural network language models (feedforward, recurrent, and long short-term memory neural networks). The software was written by Martin Sundermeyer.
• SRILM(link is external) Very popular toolkit,
  source code avaliable but only non-free for commerical use.

Speech Recognition

• AaltoASR(link is external) Aalto Automatic Speech Recognition tools
• Barista(link is external) Barista is an open-source framework for concurrent speech processing.
• Bavieca(link is external) New open source toolkit
  featuring static and dynamic decoders.
• kaldi-nnet-dur-model(link is external) Neural network phone duration model on top of the Kaldi speech recognition framework, (Interspeech paper)(link is external)
• CMU Sphinx(link is external) Open Source Toolkit For Speech
  Recognition Project by Carnegie Mellon University
• HTK(link is external) "The Hidden Markov Model Toolkit (HTK) is a
  portable toolkit for building and manipulating hidden Markov models."
• Juicer(link is external) Juicer is a Weighted Finite State Transducer (WFST) based decoder for Automatic Speech Recognition (ASR).
• Julius(link is external) "Julius is a high-performance, two-pass large vocabulary continuous speech recognition (LVCSR) decoder software for speech-related researchers and developers."
• Kaldi(link is external) Modern open source toolkit lead by
  Dan Povey featuring many state-of-the-art techniques.
• OpenDcd(link is external) An Open Source WFST based Speech Recognition Decoder.
• Phonetisaurus(link is external)
  Josef Novak's super fast WFST based Phoneticizer, site also
  has some really nice tutorials slides.
• Sail Align(link is external) SailAlign is an open-source software toolkit for robust long speech-text alignment implementing an adaptive, iterative speech recognition and text alignment scheme that allows for the processing of very long (and possibly noisy) audio and is robust to transcription errors. It is mainly written as a perl library but its functionality also depends…
• SCARF: A Segmental CRF Toolkit for Speech Recognition(link is external)
  "SCARF is a toolkit for doing speech recognition with segmental conditional
  random fields."
• trainc(link is external)
  David Rybach and Michael Riley's tool for direct construction of
  context-dependency transducers (Interspeech best paper).
• RASR(link is external) RWTH ASR - The
  RWTH Aachen University Speech Recognition System

Signal Processing

• An Interactive Source Separation Editor(link is external) "ISSE is an open-source, freely available, cross-platform audio editing tool that allows a user to perform source separation by painting on time-frequency visualisations of sound."
• Bob(link is external) Bob is a free signal-processing and machine learning toolbox originally developed by the Biometrics group at Idiap Research Institute, in Switzerland.
• Matlab Audio Processing Examples(link is external)
• SAcC - Subband Autocorrelation Classification Pitch Tracker(link is external) "SAcC is a (compiled) Matlab script that performs noise- robust pitch tracking by classifying the autocorrelations of a set of subbands using an MLP neural network."

Text-to-Speech

• HTS(link is external) HMM-based speech synthesis
• RusPhonetizer(link is external) Grammar rules and dictionaries for the phonetic transcription of Russian sentences

Speech Data

• cmudict(link is external) CMUdict (the Carnegie Mellon Pronouncing Dictionary) is a free pronouncing dictionary of English.
• LibriSpeech ASR corpus(link is external) LibriSpeech is a corpus of approximately 1000 hours of 16kHz read English speech, prepared by Vassil Panayotov with the assistance of Daniel Povey. The data is derived from read audiobooks from the LibriVox project, and has been carefully segmented and aligned.
• TED-LIUM Corpus(link is external) The TED-LIUM corpus was made from audio talks and their transcriptions available on the TED website.

Machine Translation

• Berkeley Aligner(link is external)
  "...a word alignment software package that implements recent innovations in
  unsupervised word alignment."
• cdec(link is external)
  "Decoder, aligner, and model optimizer for statistical machine translation and
  other structured prediction models based on (mostly) context-free formalisms"
• Jane(link is external)
  "Jane is RWTH's open source statistical machine translation toolkit. Jane
  supports state-of-the-art techniques for phrase-based and hierarchical
  phrase-based machine translation."
• Joshua(link is external)
  Hierarchical and syntax based machine translation decoder written in Java.
• Moses(link is external)
  Standard open source machine translation toolkit.
• alignment-with-openfst(link is external)
• zmert(link is external)
  Nice Java Mert implementation by Omar F. Zaidan(link is external)

Machine Learning

• BIDData(link is external) BIDMat is a matrix library intended to support large-scale exploratory data analysis. Its sister library BIDMach implements the machine learning layer.
• libFM: Factorization Machine Library(link is external)
• sofia-ml(link is external) Fast incremental learning
  algorithms for classification, regression, ranking from Google.
• Spearmint(link is external)
  Spearmint is a package to perform Bayesian optimization according to the
  algorithms outlined in the paper: Practical Bayesian Optimization of Machine
  Learning Algorithms Jasper Snoek, Hugo Larochelle and Ryan P. Adams Advances
  in Neural Information Processing Systems, 2012

Deep Learning

• Benchmarks(link is external) - Comparison of different convolution network implementations.
• Cafee(link is external) - Really active deep learning toolkit with support for cuDNN and lots of other backends.
• cuDNN(link is external) - Deep neural network from Nvidia with paper here(link is external). Torch 7 has support for cuDnn(link is external) and here(link is external) are some Python wrappers.
• CURRENNT(link is external) - Munich Open-Source CUDA RecurREnt Neural Network Toolkit described in this paper(link is external)
• gensim(link is external) - Python topic modeling toolkit with word2vec implementation. Extremly easy to use and to install.
• Glove(link is external) Global vectors for word representation.
• GroundHog(link is external) Neural network based machine translation toolkit.
• KALDI LSTM(link is external) C++ implementation of LSTM (Long Short Term Memory), in Kaldi's nnet1 framework. Used for automatic speech recognition, possibly language modeling etc.
• OxLM: Oxford Neural Language Modelling Toolkit(link is external) Neural network toolkit for machine translation described in the paper here(link is external)
• Neural Probabilistic Language Model Toolkit(link is external) "NPLM is a toolkit for training and using feedforward neural language models (Bengio, 2003). It is fast even for large vocabularies (100k or more): a model can be trained on a billion words of data in about a week, and can be queried in about 40 μs, which is usable inside a decoder for machine translation."
• RNNLM2WFST(link is external) Tool to convert RNNLMs to WFSTs
• ViennaCL <http://viennacl.sourceforge.net/(link is external)> - ViennaCL is a free open-source linear algebra library for computations on many-core architectures (GPUs, MIC) and multi-core CPUs.

Natural Language Processing

• BLLIP reranking parser(link is external) "BLLIP Parser is a statistical natural language parser including a generative constituent parser (first-stage) and discriminative maximum entropy reranker (second-stage)."
• OpenNLP(link is external) The Apache OpenNLP library is a machine learning based toolkit for the processing of natural language text.
• SEAL(link is external) Set expander for any language described in this paper(link is external)
• Stanford CoreNLP(link is external) "Stanford CoreNLP provides a set of natural language analysis tools written in Java"

Applications

• Cloud ASR using PyKaldi(link is external) "CloudASR is a software platform and a public ASR webservice."

Other Tools

• GraphViz.sty(link is external)
  Really handy tool adding dot languge directly to a LaTex document, useful for
  tweaking the small colorized WFST figure in papers and presentations.

Blogs

• Between One and Zero(link is external) by William Hartmann
• cmusphinx(link is external) CMU Sphinx related blog
• Language Log(link is external)
• LingPipe Blog(link is external) Natural Language Processing and Text Analytics
• Natural Language Processing Blog(link is external) by Hal Daumé III
• Spoken Language Processing(link is external) "Some thoughts on Spoken Language Processing, with tangents on Natural Language Processing, Machine Learning, and Signal Processing thrown in for good measure."

Books

• DEEP LEARNING: Methods and Applications(link is external) By Li Deng and Dong Yu
• Foundations of Data Science(link is external) Draft by John Hopcroft and Ravindran Kannan
• Introduction to Matrix Methods and Applications(link is external) (Working Title) S. Boyd and L. Vandenberghe