Research project course - Knowledge-aware AI (KAAI)

Module: INF-25-Ma-FP (Research project)

Credits: 13 CP (390 hours)

Overview: This research project course brings together up to 10 individual INF-25-Ma-FP research projects into a joint framework. The projects focus on building and extending epistemic twins from large language models (LLMs), with aspects such as explainability, consolidation, qualifiers, or RAG.

Schedule:

• Regular biweekly events on Tuesdays
• Open house format: 2-hour slots where instructors are present for Q&A
• Before every open house, provide a short status update in the forum: what was achieved, what is planned next, what are open questions/blockers (bullets suffice)

Deliverables:

1. Midterm presentation (15' + 10' discussion)
2. Final presentation (15' + 10' discussion)
3. Final report (6 pages ACL style)

Communication:

• Joint Slack or Matrix channel
• OPAL forum

Possible topics:

1. Extending twin construction with RAG (web search, corpus, proper ranking/diversity, attribution)
2. Twins with post-hoc referencing (retro-fitting attributions)
3. Annotating whether a triple was in the training data, or whether it is extrapolated
4. Taxonomies for relations (PATTY) with modern methods
5. Taxonomy for classes
6. Linking GPTKB to Wikidata
7. Researching whether epistemic twins can help in LLM explainability
8. Multilingual differences between twins, what they are, why they emerge?
9. Building a retro twin, and analyzing its content (e.g., based on Talkie-LLM)
10. Estimating LLM knowledge by random string sampling, and delineating memorization from inference
11. Twins with qualifiers: Extraction, distribution, correctness
12. Quantities in twins: Distribution, form, accuracy
13. Extreme homonymy: Distinction or conflation
14. Continuously updating a twin
15. Downstream effects of political biases in twins
16. Scaling images to millions of entities
17. Extracting and representing confidences in twins

Core reading:

1. Hu et al. Enabling LLM Knowledge Analysis via Extensive Materialization. ACL, 2025
2. Giordano and Razniewski. Foundations of LLM Knowledge Materialization: Termination, Reproducibility, Robustness. EACL 2026
3. Saeed and Razniewski. LLMpedia: A Transparent Framework to Materialize an LLM's Encyclopedic Knowledge at Scale. Arxiv 2026
4. https://gptkb.org

Application:

• The number of places is limited. To express interest, please send an email to TBD by TBD.