OpenRouter Q&A Service
SECTION 22B — OPENROUTER Q&A SERVICE
22B.1 Overview
The Q&A generation feature (§10.10) is powered by AI models accessed through the OpenRouter aggregator API. OpenRouter exposes a unified OpenAI-compatible REST surface across many model providers, including OpenAI’s Structured Outputs spec. The Notesglider backend:- Holds a single platform-wide OpenRouter API key.
- Routes every Q&A request through this key.
- Resolves the model to call per a Super-Admin-configurable resolution chain (§22B.2).
- Enforces spend guardrails per Organisation (§22B.4).
- Returns Structured Outputs validated against the JSON Schema for the requested question type.
Why Structured Outputs
“Structured Outputs is a feature that ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don’t need to worry about the model omitting a required key, or hallucinating an invalid enum value.”Benefits we rely on:
- Reliable type-safety — the response shape matches our schema; no defensive parsing of free-form prose.
- Explicit refusals — safety-based model refusals are programmatically detectable (handled via the §22B.3 refusal branch, not the validation chain).
- Simpler prompting — no need for strongly-worded “RESPOND IN JSON ONLY” instructions; the schema is the contract.
Note on the inspirational reference prompt: legacy markdown-driven prompt patterns exist in industry write-ups (which use#/##/###headings to encode document structure). Our project does NOT use markdown input for the prompt. Notesglider passes news items as a structured JSON payload extracted directly from the canonical Lexical tree (ephemeralrefhandle, headline, category, Brief text-flattened). atomic_uid values are stripped server-side before payload assembly and never reach the model — see §22B.3.A “Identity Boundary”.
22B.2 Model Resolution Chain
Only the Super Admin can configure models. No other role has any visibility into model configuration.
Resolution order at request time:
per_org_override_modelif set, elsedefault_model.- On failure → iterate the resolved fallback list (per-org override if set, else system-wide).
22B.3 Request Lifecycle
- Receive request from Q&A wizard with: number of questions
N, per-question type spec, selected news-item atomic_uids (resolved server-side from the wizard’s grid selection). - Prompt assembly:
- Backend builds the request-scoped
ref → atomic_uidmap (\{ "n1": <uid_1>, "n2": <uid_2>, … \}) held in memory for this single OpenRouter call. - Backend constructs the payload with only
refvalues, headlines, categories, and flattened Brief text (Lexical text-only walk; images and formatting stripped). atomic_uid values are stripped out before the payload leaves the backend boundary. - Backend appends the per-type prompt template (§22B.3.A) appropriate to the question type. Mixed-type requests dispatch one call per type group.
- Backend builds the request-scoped
- JSON Schema attachment: per the OpenAI Structured Outputs spec, the request includes the JSON Schema for the exact response shape. Schemas (one per question type variant) are defined in code as Pydantic models (server side) and JSON Schema (sent to OpenRouter).
- Model dispatch: call the resolved primary model.
- Validate response:
- Parse JSON (Structured Outputs guarantees JSON, but parse for safety).
- Run JSON Schema validation against the canonical shape for the requested type (per §11.1
qa.items[]):subjective.straightforward:\{ statement: string, answer: string \}.objective.direct:\{ statement: string, options: \{ A, B, C, D: string \}, correct_option: "A"|"B"|"C"|"D" \}. All four option keys required.objective.statement_analysis:\{ topic: string, statements: [string, string, string] (exactly 3), options: \{ A, B, C, D: string \}, correct_option: "A"|"B"|"C"|"D" \}.
- Run business-rule validation:
correct_optionmust reference a key actually present inoptions.- For
objective.statement_analysis: each of the 4optionsvalues must be drawn (without repetition) from the allowed set\{ "1", "2", "3", "1+2", "2+3", "1+3", "All of the above", "None" \}. statementsarray length must be exactly 3.- Exactly one
correct_optionper item — multi-correct or zero-correct payloads are rejected.
- Retry chain:
- Same model → up to 3 retries on validation failure or transport error.
- If still failing → advance to the next model in the fallback list, restart at retry 0.
- 4 fallback models × up to 3 retries each + 1 primary × 3 retries = 15 attempts maximum per request.
- Return to client: validated Q&A items.
- On total failure (all 15 attempts exhausted): return error response. Wizard surfaces “Q&A generation failed after retries. Please try again later, or skip Q&A for this document.”
22B.3.A Per-Type Prompt Templates
Each question type uses its own dedicated prompt. The selected news items are serialised as a structured JSON payload (not markdown) and appended after the prompt. Inspirational interrogative styles are listed inline so the model adopts the right voice; strict rules eliminate prose padding.Identity Boundary — atomic_uid Never Reaches the Model
This is the non-negotiable rule for identity handling in Q&A generation:
Why: atomic_uids are immutable identity primitives that must never be fabricated. Allowing the model to echo an atomic_uid back exposes us to hallucinated or malformed UIDs that would either pollute the audit chain (§11A) or fail the
atomic_uid_log lookup. The model is scoped strictly to content generation for the three question types. Identity is always backend-assigned and backend-resolved.
The model speaks only in ref handles — short opaque tokens (n1, n2, …) scoped to a single request. After validation, the backend swaps source_refs[] → source_atomic_uids[] via the request-scoped lookup map (§22B.3.D), then assigns the QA item’s own qa_id.
Common payload structure sent to the model alongside every prompt:
Note:Template T1 —refvalues are simple positional handles (n1,n2,n3, …) assigned by the backend at request assembly time. They have no meaning outside this single OpenRouter call and are discarded after post-processing. The backend keeps a private in-memory map\{ "n1": <atomic_uid>, "n2": <atomic_uid>, … \}for the duration of the request.
subjective.straightforward
objective.direct
objective.statement_analysis
22B.3.B Sample Structured Outputs
These are the exact response shapes the model must return for a single-item generation. Multi-item requests return an array of these. Sample forsubjective.straightforward (Template T1):
objective.direct (Template T2):
objective.statement_analysis (Template T3):
tests/fixtures/qa/ and consumed by the §32.6 Phase 8 snapshot tests.
22B.3.C Validation, Rejection & Regeneration Logic (Model Live)
The model is treated as a live, occasionally-drifting collaborator. Even with Structured Outputs the response can deviate in subtle ways (semantic-rule violations, allowed-set drift on Template T3). The backend handles every deviation deterministically without manual intervention. Validation pipeline (runs for every response, every attempt):openrouter_call_log, §22B.4 telemetry):
Why all-or-nothing: partial accept means the saved Q&A set mixes high-quality and degraded items; users cannot tell which is which. Cost of regeneration is bounded by the 15-attempt cap.
22B.3.D Backend Post-Processing & Identity Assignment
After the validation pipeline (§22B.3.C) returns a green response, the backend performs mandatory identity translation before the items are presentable to the wizard or persisted to the database. The model’s output never touches the database unmodified. Step-by-step:
What this gives us:
- Zero risk of model-fabricated atomic_uids ever entering the audit chain.
qa_idordering is deterministic and survives multiple wizard runs (new acceptances append after the existing max sequence).- Aggregation linkage (
source_atomic_uidsused by Compilation/Magazine Q&A merging in §10/Phase 10) is provably correct — every uid was sourced from the backend’s own resolved map, not from the model’s text. - Late-add regen (§13.6) reuses the same pipeline — the only difference is the actor role on step 3 and the resulting
qa-late-added = trueflag.
Implementation note for the developer: place this post-processing in a singleqa/post_process.pymodule. Any code path that writes toqa_itemsmust go through it. Direct INSERTs from model output are an architectural defect.
22B.3.E Assembly, Merge & Atomicity (Multi-Call → Single qa Node)
A single wizard run can request a mix of types (e.g. 3 × T1 + 2 × T2 + 5 × T3 = 10 items). Per §22B.3.A, the backend dispatches one OpenRouter call per type group — never a mixed-schema call. That produces multiple parallel responses that must be merged into a single, slot-ordered, persistable qa node.
This subsection is the full backend walkthrough from “Generate clicked” to “row in qa_items + node in content_json.qa.”
1. Wizard slot model
The wizard collects N questions as an ordered list of slots (1-indexed in wizard order):
slot_index is the single ordering authority for the entire pipeline. The model never sees it; it lives only on the backend.
2. Group → Dispatch → Per-Group Ref Maps
Group slots by type while preserving each slot’s original slot_index:
ref → atomic_uid map and a parallel ref → slot_index map:
n1 in the T2 call has no relation to n1 in the T1 call. Maps live only for the duration of their group’s call.
3. Dispatch (sequential, one per group)
Per §22B.3.A: a sequence of three calls in fixed order T1 → T2 → T3 (whichever groups are non-empty). The wizard shows a single progress indicator that advances per group. Each call follows the §22B.3 lifecycle independently (own 15-attempt retry chain, own validation pipeline).
\{ "items": [...] \} in payload-positional order — the model produces one item per news_item entry it received. Walk the response and re-attach slot_index via the per-group map:
source_refs resolves to more than one distinct slot_index (model fabrication across slots), the entire response is rejected per §22B.3.C step 5.
5. Merge into slot order + Identity Assignment
Collect all re-associated items from all groups, sort by slot_index, then run §22B.3.D post-processing in that order. Because slot_index ascends 1..N, the resulting qa_id sequence numbers (assigned by §22B.3.D step 2) honour wizard order.
qa_id, source_atomic_uids, type. Those are immutable post-assignment.
Provenance split — two destinations for each item, never mixed:
Top-level
qa node fields (set ONCE, on the first acceptance for a document; later acceptances do NOT overwrite):
Rationale: top-level fields are render-time metadata (e.g. PDF footer might show “AI-generated by gpt-4o-mini on 2026-03-01”). They must be stable; otherwise late-adds flip the displayed model name and acceptance time, which confuses users. Per-item provenance for full audit (every actor, every model, every acceptance time) lives on the
qa_items rows.
The transaction:
qa_late_added event fires the PDF regen pipeline outside this transaction.
8. Final shape of the assembled qa node — matches §11.1 byte-for-byte
After commit, documents.content_json.qa matches the canonical §11.1 schema exactly. One items[] entry per accepted question, in slot_index order, each with a unique sequential qa_id. Top-level fields (generated_by_role, generated_at, model_used) are render-only metadata set ONCE on first acceptance. Per-item provenance does NOT appear here — it lives on qa_items rows.
Concrete output for the 5-slot wizard run above (mixed T1+T2+T3), produced by the §22B.3.E step 7 transaction:
source_atomic_uids on item 5 (slot 5 had 2 news_items selected — entirely valid per the wizard model).
Single sources of truth — non-overlapping:
documents.content_json.qa→ rendering (WeasyPrint PDF, PptxGenJS PPTX, HTML export, translation). Lean, matches §11.1.qa_itemsrows → relational joins (per-item provenance, aggregation linkage in §10/Phase 10, telemetry, late-add audit trail). Carries every field both content_json has AND the provenance fields content_json deliberately omits.
seqstarts from the existingMAX(qa_id_seq)for the document (not from 0). New items append:QA-\{doc-id\}-006,QA-\{doc-id\}-007, ….is_late_add = true→ setsdocuments.qa_late_added = trueand fires theqa_late_addedpipeline event that triggers Stage-3 PDF regen (§13.6).
permissions.qa = true) hit “Accept Q&A” on the same document simultaneously, the SELECT FOR UPDATE in step 7 serialises them. The second-committing transaction observes the first’s writes and re-assigns its qa_ids starting from the updated max sequence. Both sets persist in slot order, just with a sequence-window offset for the second. Neither loses items; neither overwrites the other.
22B.4 Spend Guardrails
Per-organisation monthly token cap (configurable by Super Admin per Org; default applies system-wide):
Cap-breach behaviour:
- Wizard “Generate” button is disabled; tooltip surfaces “Your organisation has reached its monthly Q&A generation limit. Contact your administrator to increase the limit or wait for the next billing cycle.”
- No partial-completion: a request that would push the running token total over the cap is rejected up front (estimated tokens computed from prompt size + max completion tokens for the requested type).
- Every OpenRouter call is logged in a
openrouter_call_logtable with: timestamp, org_id, document_id, model used, attempt index, prompt_tokens, completion_tokens, total_tokens, success boolean, validation_errors (if any). - Super Admin dashboard surfaces aggregate spend per Org per month and a system-wide total.
22B.5 RBAC for Q&A Actions
A new boolean
permissions.qa is added to the sub_members.permissions JSON (defaults to false).
22B.6 Translation Interaction
When the source document has translations (is-translated = true), the qa node is included in the translation request to Google Translate (§18). The translated qa node is written into each parallel translated document. Q&A items in a translated document use the same qa_id value as in the source — this enables back-mapping during aggregation.