Language and Computation Courses
Advanced Course
Neural Dependency Parsing of Morphologically-Rich Languages,
Erhard Hinrichs (University of Tübingen, Germany) and Daniël de Kok (University of Tübingen, Germany)
Week 1, 17:00 – 18:30, Room 243, Floor 2
This course will introduce parsing of morphologically-rich languages using neural transition-based dependency parsers. Until recently, most work on neural transition-based dependency parsing was conducted on English. However, with the recent introduction of the Universal Dependency annotation scheme and corresponding treebanks for 50 languages (De Marneffe et al., 2014; Nivre et al., 2016), it seems timely to explore neural transition-based dependency parsing for other languages. In this course, we will focus on morphologically-rich languages and will draw on our own dependency parsing research as one use case of this kind. More generally, we will discuss how information about the macrostructure of different clause types (de Kok and Hinrichs, 2016) and about word-level morphology (de Kok, 2015; Ballesteros et al., 2015) can substantially improve parsing accuracy for such languages. We will further show how recurrent neural networks can be used to model such information.
Course survey
If you are participating in our course, it would be great if you could fill out the following survey, to give us more insight into your background and what topics interest you the most:
https://goo.gl/forms/CdDdXXlDvVZakBHi2
Slides and literature
Course slides and literature will be uploaded after each lecture.
Day 1
Dependency grammar & Transition-based dependency parsers
- Sandra Kübler, Ryan McDonald, Joakim Nivre, Dependency Parsing, 2009
Day 2
Transition-based dependency parsers & linear classification
Day 3
- Danqi Chen & Christopher Manning, A Fast and Accurate Dependency Parser using Neural Networks, EMNLP 2014
Day 4
Morphology & word representations
- P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, Enriching Word Vectors with Subword Information
- Wang Ling, Chris Dyer, Alan Black, Isabel Trancoso, Two/Too Simple Adaptations of Word2Vec for Syntax Problems, NAACL 2015
- Omer Levy & Yoav Goldberg, Dependency-Based Word Embeddings, ACL 2014
- Omer Levy & Yoav Goldberg, Neural Word Embedding as Implicit Matrix Factorization, NIPS 2014
- Daniël de Kok, A Poor Man’s Morphology for German Transition-Based Dependency Parsing, TLT 14
Day 5
Recurrent Neural Networks for parsing
- Robbert Prins, Finite-state pre-processing for natural language, 2005
- Jeffrey Elman, Finding Structure in Time, Cognitive Science, Volume 14, 1990
- Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmity Bahdanau Fethi Bougares, Holger Schwenk, and Yoshua Bengio, Learning Phrase Represenations using RNN Encoder-Decoder for Statistical Machine Translation
- Daniël de Kok and Erhard Hinrichs, Transition-based Dependency Parsing with Topological Fields, ACL 2016
- Eliyahu Kiperwasser and Yoav Goldberg, Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations, TACL vol. 4
- Wang Ling, Tiago Luís, Luís Marujo, Ramón Fernandez Astudillo, Silvio Amir, Chris Dyer, Alan W. Black, and Isabel Trancoso, Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation, ACL 2015
- We will add the references from the discussion as soon as possible…