IN5550 – Neural Methods in Natural Language Processing

Course content

This course studies a selection of advanced techniques in Natural Language Processing (NLP), with particular emphasis on modern research findings. The focus of the course is on "deep learning", a type of machine learning techniques using artificial neural networks.?Recently, natural language understanding systems based on deep neural models?such as ChatGPT?have revolutionized many spheres of our society and IN5550 allows students to look "under the hood" of such systems and to learn how they are built.

Topics typically include representation learning for words and other linguistic units, document classification, sequence tagging, natural language generation and other NLP tasks. They are solved using techniques like Feed-Forward Neural Networks (FFNN), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and Transformers with self-attention. A special focus is put on using and designing large language models. In addition the course provides an introduction into biases and sustainability of modern deep learning methods in NLP.

The course includes strong practical components and puts emphasis on NLP problems and massive datasets of central importance in current research.?In the end of the course, the students are expected to? complete an experimental exam project and submit its summary in the form of a research paper. Thus, they will be prepared to further pursue an MSc project in deep learning based Natural Language Processing and/or Artificial Intelligence.