What does structure buy?
Graph reasoning, keyword and vector retrieval over the same fictional company. Recorded measurements; no model generates the answers.
Loading recorded results…
Test macro F1 · all test questions
Higher F1 means closer agreement with the authored expected ids. Every recorded k is shown; system names and scores accompany each bar.
Read the limits
We wrote these 60 questions and their expected ids. Human review is pending. Test questions include related parameter and paraphrase variants; they are not independent companies. The graph knows the dependency structure by construction. This measures structured reasoning against passage retrieval, not general QA ability.
Both retrievers use the same ADR passages. Extra entity ids reduce precision. The local MiniLM checkpoint uses 384-dimensional embeddings in a real FalkorDB vector index.
Read all 60 questions and labelsAll test measurements · 21 rows
Test split only. Includes all, lexical and multi-hop subsets. Scroll horizontally on smaller screens.
| Test subset | System | k | Precision | Recall | F1 | Exact | p50 ms | p95 ms |
|---|