CS 4770 Natural Language Processing (Fall 2025)

Logistics

• Instructor: Yu Meng (yumeng5[at]virginia[dot]edu); Office Hour: After class every Monday and Wednesday
• Teaching Assistants:
  • Wei-Lin Chen (wlchen[at]virginia[dot]edu); Office Hour: Thursdays 5:00pm - 6:00pm (Zoom link)
  • Zhepei Wei (zhepei.wei[at]virginia[dot]edu); Office Hour: Wednesdays 4:00pm - 5:00pm (Zoom link)
  • Xinyu Zhu (xinyuzhu[at]virginia[dot]edu); Office Hour: Tuesdays 2:00pm - 3:00pm (Zoom link)
• Lecture Time: Mondays & Wednesdays 2:00pm - 3:15pm
• Location: Mechanical Engr Bldg 205

Course Overview

In this course, we will explore the foundational concepts and the latest advancements in Natural Language Processing (NLP). The course aims to provide a thorough understanding of both traditional and modern approaches to language modeling, from N-gram language models to large language models (LLMs). We’ll start by covering the fundamentals of language representation, including classic methods such as word embeddings and vector space models. As we progress, we will introduce the Transformer architecture, which underpins today’s most powerful LLMs, and explore how techniques like pretraining and fine-tuning have transformed the field. In the later part of the course, we will examine advanced topics including parametric knowledge, retrieval-augmented generation (RAG), Reinforcement Learning from Human Feedback (RLHF), and Reinforcement Learning with Verifiable Rewards (RLVR), which are essential for building and deploying NLP systems in practical, real-world scenarios.

Grading

• Assignments (60%): There will be 5 assignments (with different weights) for the entire semester.
  • All assignments must be completed individually
  • Assignments will be a combination of concept questions + coding questions
  • Submission via Canvas (as LaTeX report; handwritten submissions not accepted)
  • DO NOT procrastinate on assignments! The coding questions (especially in the latter part of this course) take time to implement and run!
• Project (35%): By the end of this course, you are required to complete a project related to NLP, present your results, and submit a project report. You are required to work in a team of 2 or 3 (any deviation from this team size requires prior approval from the instructor). The rule of thumb is to demonstrate that you are able to apply the knowledge learned from this course. The workload for the project is expected to be more extensive than individual assignments. Some example project choices (non-exclusive):
  • Use word embeddings to analyze sentence semantics (e.g., sentiment analysis)
  • Fine-tune BERT and evaluate its performance for any task you like (e.g., named entity recognition, relation extraction)
  • Benchmark LLMs (either open-weights or proprietary) for challenging tasks/applications (e.g., math reasoning, coding)
  • Use LLM APIs to create agents for an interesting application (e.g., personal assistants)
  • …

  The project grading breakdown (35%) is as follows:

  • Project proposal: 2% (Deadline: 09/24 11:59pm)
  • Mid-term report: 8% (Deadline: 10/20 11:59pm)
  • Final presentation (Deadline: 11/30 11:59pm) and final report (Deadline: 12/13 11:59pm): 25%

• Participation (5%+; points earned beyond 5% will become extra credit): We encourage everyone to actively participate in the class. There are several ways of earning the participation credit:

  • Guest lecture attendance on Zoom (3% each guest lecture):
    • We will have guest lectures delivered by leading researchers in this field
    • Each guest lecture can give you up to 3% participation credit (2% attendance + 1% asking questions – details shared before guest lectures)
  • End-of-semester teaching feedback (2%):
    • At the end of the semester, anyone who completes the teaching feedback survey will get 2%
  • Answering technical questions raised by classmates (5%):
    • We encourage and appreciate help from students to answer questions posted by classmates
    • Every helpful answer to technical questions will earn 1% (Slido and Piazza both count)
    • If you answer anonymously, we won’t be able to track your contributions!
    • The maximum credit you can get in this category is 5%

Late Day Policy (Only Applied to Assignments, Not Projects!):

• You have 7 free days total for all assignments
• After you’ve used up your 7 late days, a penalty of 20% will be deducted from the assignment grade for each additional day it is late
• You cannot use > 3 late days (72 hours) per assignment unless given permission in advance
• Late days cannot be applied to project deadlines! You will get a 0 for any project checkpoints that you missed

Policy on Using LLMs/GenAI:

Collaborative coding with LLMs is allowed, but if you directly copy the answers generated by LLMs (for either conceptual or coding questions), you’ll get a 0 for that entire assignment!

Schedule

Useful Materials

• For NLP background: Speech and Language Processing
• For deep learning background: Deep Learning

Students with Disabilities or Learning Needs

It is my goal to create a learning experience that is as accessible as possible. If you anticipate any issues related to the format, materials, or requirements of this course, please meet with me outside of class so we can explore potential options. Students with disabilities may also wish to work with the Student Disability Access Center (SDAC) to discuss a range of options to removing barriers in this course, including official accommodations. You may email an SDAC advisor at cmacmasters@virginia.edu to schedule an appointment. For general questions please visit the SDAC website: sdac.studenthealth.virginia.edu. If you have already been approved for accommodations through SDAC, please send me your accommodation letter and meet with me so we can develop an implementation plan together.

Religious Accommodations

It is the University’s long-standing policy and practice to reasonably accommodate students so that they do not experience an adverse academic consequence when sincerely held religious beliefs or observances conflict with academic requirements.

Students who wish to request academic accommodation for a religious observance should submit their request to me by email as far in advance as possible. Students who have questions or concerns about academic accommodations for religious observance or religious beliefs may contact the University’s Office for Equal Opportunity and Civil Rights (EOCR) at UVAEOCR@virginia.edu or 434-924-3200.

Harassment, Discrimination, and Interpersonal Violence

The University of Virginia is dedicated to providing a safe and equitable learning environment for all students. If you or someone you know has been affected by power-based personal violence, more information can be found on the UVA Sexual Violence website that describes reporting options and resources available - www.virginia.edu/sexualviolence.

The same resources and options for individuals who experience sexual misconduct are available for discrimination, harassment, and retaliation. UVA prohibits discrimination and harassment based on age, color, disability, family medical or genetic information, gender identity or expression, marital status, military status, national or ethnic origin, political affiliation, pregnancy (including childbirth and related conditions), race, religion, sex, sexual orientation, or veteran status. UVA policy also prohibits retaliation for reporting such behavior.

If you witness or are aware of someone who has experienced prohibited conduct, you are encouraged to submit a report to Just Report It (justreportit.virginia.edu) or contact EOCR, the office of Equal Opportunity and Civil Rights.

If you would prefer to disclose such conduct to a confidential resource where what you share is not reported to the University, you can turn to Counseling & Psychological Services (“CAPS”) and Women’s Center Counseling Staff and Confidential Advocates (for students of all genders).

As your professor and as a person, know that I care about you and your well-being and stand ready to provide support and resources as I can. As a faculty member, I am a responsible employee, which means that I am required by University policy and by federal law to report certain kinds of conduct that you report to me to the University’s Title IX Coordinator. The Title IX Coordinator’s job is to ensure that the reporting student receives the resources and support that they need, while also determining whether further action is necessary to ensure survivor safety and the safety of the University community.

Student Support Team

You have many resources available to you when you experience academic or personal stresses. In addition to the course staff, the School of Engineering and Applied Science has staff members located in Thornton Hall who you can contact to help manage academic or personal challenges. Please do not wait until the end of the semester to ask for help!

Community and Identity

The Center for Connection (The Connect) is a dedicated student space within UVA Engineering that fosters academic success and personal growth. Through its programs and initiatives, The Connect helps students strengthen their engineering identity while providing resources to help them thrive during their studies and beyond. Our work centers on three key areas: student belonging and development, academic support, and community programming grounded in intentional, data-driven strategies.

The Connect features an open study area, a flexible event space, and on-site staff who provide direct support and advising to students. It is part of the Office of Community, Opportunity, and Engagement.