Large Language Models and Knowledge-aware AI (Seminar)

• The number of participants is limited, with preference given to Master students
• To express interest, send an email to the lecturer (simon.razniewski@tu-dresden.de), including a short motivation statement and your transcript
• Places will be allocated based on background match (courses taken) and motivation

Topics

1. Facts and hallucinations in LLMs
   Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models
   Javier Ferrando, Oscar Obeso, Senthooran Rajamanoharan, Neel Nanda
   Locating and editing factual associations in gpt
   K Meng, D Bau, A Andonian, Y Belinkov
2. Physical world knowledge
   PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
   Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, Yue Wang
3. Knowledge representation in LLMs
   The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
   Lukas Berglund et al
   

   Supposedly Equivalent Facts That Aren't? Entity Frequency in Pre-training Induces Asymmetry in LLMs

   Y He et al
4. Chain of thought reasoning
   Chain-of-thought prompting elicits reasoning in large language models
   J Wei et al
   Whiteboard-of-Thought: Thinking Step-by-Step Across Modalities
   Sachit Menon, Richard Zemel, Carl Vondrick
5. World model emergence in LLMs
   Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
   Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, Martin Wattenberg
   Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data
   Treutlein et al.
6. Cultural knowledge
   Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking
   Yi Fung, Ruining Zhao, Jae Doo, Chenkai Sun, Heng Ji
7. Theory of mind
   How FaR Are Large Language Models From Agents with Theory-of-Mind?
   Pei Zhou et al.
8. Cognitive Psychology on LLMs
   Using cognitive psychology to understand GPT-3
   Marcel Binz, Eric Schulz
9. Historical perspective
   As We May Think
   Vannevar Bush
10. LLM versus human language learning
    BabyLM challenge
    Charpentier et al.
11. Taxonomy construction and refinement with LLMs
    Refining wikidata taxonomy using large language models
    Y Peng, T Bonald, M Alam
    Towards ontology construction with language models

    M Funk, S Hosemann, JC Jung, C Lutz

     

    (own topic suggestions are welcome as well)

Deliverables

There are 5 deliverables. To pass the course, all have to be submitted on time. Percentages in brackets denote contribution to final grade.

1. Outline of report (5%)
2. Report 1st version (0%)*
3. Reviews on two other reports (15%)
4. Reports final version (40%)
5. Presentation (40%)

• 1st revision is not graded, but the prime chance to obtain feedback from advisor and peers.

Tentative timeline

• Mon 7.4.: Application deadline
• Wed 9.4. Notification of accepted participants
• Wed 16.4., 10am-12: "Introduction to KAAI" lecture, location TBD
• Fr 25.4., 10am-12: "Seminar survival skills" lecture + topic assignment, location TBD
• Wed 7.5.: 1st deliverable due
• 12.-20.5.: Meetings with advisors
• Wed 11.6.: 2nd deliverable due + submit bids for reviewing other papers
• Thu 19.6.: 3rd deliverable due
• Thu 3.7.: 4th deliverable due
• Mon, Tu 14./15.7.: Block seminar presentations

Material

• Slides 1st meeting
• Slides 2nd meetings
• Report template
• Easychair for reviewing