Large Language Models and Knowledge-aware AI (Seminar)

• The number of participants is limited, with preference given to Master students
• To express interest, send an email to the lecturer (simon.razniewski@tu-dresden.de), including a short motivation statement and your transcript
• Places will be allocated based on background match (courses taken) and motivation

Topics

1. Hallucinations in LLMs
   Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models
   Javier Ferrando, Oscar Obeso, Senthooran Rajamanoharan, Neel Nanda
2. Physical world knowledge
   PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
   Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, Yue Wang
3. Knowledge representation in LLMs
   The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
   Lukas Berglund et al
4. Chain of thought multimodal
   Whiteboard-of-Thought: Thinking Step-by-Step Across Modalities
   Sachit Menon, Richard Zemel, Carl Vondrick
5. World model emergence in LLMs
   Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
   Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, Martin Wattenberg
6. Cultural knowledge
   Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking
   Yi Fung, Ruining Zhao, Jae Doo, Chenkai Sun, Heng Ji
7. Theory of mind
   How FaR Are Large Language Models From Agents with Theory-of-Mind?
   Pei Zhou et al.
8. Cognitive Psychology on LLMs
   Using cognitive psychology to understand GPT-3
   Marcel Binz, Eric Schulz
9. Historical perspective
   As We May Think
   Vannevar Bush

(own topic suggestions are welcome as well)

Deliverables

There are 5 deliverables. To pass the course, all have to be submitted on time. Percentages in brackets denote contribution to final grade.

1. Outline of report (5%)
2. Report 1st version (0%)*
3. Reviews on two other reports (15%)
4. Reports final version (40%)
5. Presentation (40%)

• 1st revision is not graded, but the prime chance to obtain feedback from advisor and peers.

Tentative timeline

• Mon 7.4.: Application deadline
• Wed 9.4. Notification of accepted participants
• Wed 16.4., 10am-12: "Introduction to KAAI" lecture, location TBD
• Wed 23.4., 10am-12: "Seminar survival skills" lecture + topic assignment, location TBD
• Wed 7.5.: 1st deliverable due
• 12.-20.5.: Meetings with advisors
• Wed 11.6.: 2nd deliverable due + submit bids for reviewing other papers
• Thu 19.6.: 3rd deliverable due
• Thu 3.7.: 4th deliverable due
• Mon, Tu 14./15.7.: Block seminar presentations

Material

• Slides 1st meeting
• Slides 2nd meetings
• Report template
• Easychair for reviewing