Applied AI
Two retrieval systems built to answer from a fixed set of documents and to say which one each answer came from. The first was my bachelor thesis. The second, Sourceling, moves the rules the thesis left in a prompt into the code around the model.
The thesis at Bergische Universität Wuppertal: concept, architecture, implementation and evaluation. Sourceling is my own build: product, architecture and code.
- Stage
- Thesis archived, prototype local
- Thesis
- Submitted February 2025, graded 1.0
- Sourceling
- Built in August 2026. Not deployed, no users
- Not yet
- Real users. No AI system I built is in production.
Thesis
A learning assistant for a first-year C course. It answers only from the lecture slides, exercise sheets and course website, and it is told to give hints instead of the finished solution. A supervisor agent picks one agent per indexed source by reading the agents’ own descriptions, so the routing settles which source an answer came from. Both rules that mattered lived in the system prompt. Asked directly for a program, the system logged that it must not write one and then wrote one. Citations in the text came out incomplete or wrong on several answers.
Sourceling
Multi-tenant retrieval: each tenant queries only its own uploaded documents. A question passes through 10 agent steps, and keyword search and vector search are fused in one PostgreSQL query. After drafting, a grounding and citation check decides whether the answer is sent. A failure goes back to retrieval or ends in a refusal. Through a gateway that drops the citation data, an answer ships without citations, and its audit row reads gate=not run. Tenants are separated by PostgreSQL row-level security. At first the application connected as the database owner, which bypasses those policies, and a test that connected the same way passed. It now connects under a role that cannot bypass them.
Trade-offs
One agent per source
One index over everything with metadata filters would have been less code and one model call. Separate agents cost more latency and more tokens per question, and the supervisor is a single point of failure. Every new document is another agent, and routing gets harder as the descriptions start to overlap. For a course with three sources I accepted that. For a large collection of documents I would not.
Rules held outside the model
In Sourceling the tenant filter is a database policy, and an answer is checked after it is written. The check costs a second model call, and a retry can double the time one question takes. In return, a checked answer that fails becomes a refusal: a draft whose citations do not hold up is withdrawn and replaced by the refusal.
The threshold is a design choice
The citation check passes at a citation precision of at least 0.80, the share of the quotes it checks that are found in the passages they cite. A second model also has to judge the claims in the answer supported. Nobody has scored the system against labelled answers, so I know what the threshold does and cannot say whether it is the right number.
Built with
- Thesis
- Python, LlamaIndex, Weaviate. FastAPI since submission; the thesis documents Flask with Socket.IO
- Sourceling
- Python, FastAPI, LangGraph, PostgreSQL with pgvector and pg_search, Next.js
- Records
- 56 decision records and 974 backend tests in 150 files, on 27 August 2026. CI has never finished a run, so every passing test ran on my machine.