Practical guide 02 / 05
AI for government becomes useful when staff reach a defensible answer faster without gaining access to information they should not see. A language model alone cannot provide that outcome. The application needs authorised sources, enforceable permissions and an evaluation process that can expose incorrect answers.
Start with a bounded knowledge assistant
Internal search across approved manuals, operating instructions or technical documentation can be a practical first use case. Retrieval-Augmented Generation, or RAG, supplies selected search results as context for the model. The output should point to its evidence and acknowledge missing support. Germany’s Data Protection Conference announced dedicated RAG guidance in October 2025. [1]
The problem is not a missing chat interface
An officer needs a current procedure. The organisation has an old PDF, an updated intranet page and supplementary instructions in a team folder. A fast summary built from the wrong version does not remove work. It transfers the effort into checking and correcting the result.
The meaningful success measure is therefore not how natural the answer sounds. It is how quickly the user reaches a result that can be checked against valid, accessible evidence. That distinction should shape the architecture and the procurement specification.
Three problems a government chatbot does not solve by itself
Indexing can bypass existing document permissions
A technical ingestion account may read more than the end user. That access must not be inherited by the assistant. Apply user permissions during retrieval and protect document titles, snippets and cached outputs as well as full documents. A shared answer cache can otherwise leak information between groups with different access rights.
Correct documents can still produce incorrect answers
RAG does not eliminate hallucinations. Retrieval may select an obsolete version, and the model may overlook an exception or misrepresent a passage. Keep provenance, version and applicability attached to source material. A citation is useful only when the referenced passage actually supports the generated claim.
Documents can contain hostile instructions
An attacker need not modify the system prompt directly. Manipulated content in a retrieved document can attempt to redirect the model. The BSI has documented indirect prompt injection as a risk in application-integrated language models. [2] Treat retrieved material as untrusted input, not as an authority to change the application’s rules.
Design implication: The first knowledge assistant should not make administrative decisions or have unrestricted write access to case-management systems. A well-bounded read-only service is already a substantial production system.
A sovereign RAG architecture for sensitive knowledge
Enforce permissions before building model context
Authenticate users through the organisation’s identity system. Filter search against current entitlements and recheck selected content before passing it to the model. Define how permission changes and deletions propagate between the source and search index. Embeddings should not be assumed anonymous simply because they are numerical representations.
Put approved models behind a controlled interface
A model gateway can govern approved services, permitted data categories, logging and cost limits. It can make replacement easier, but it does not make models interchangeable by default. Context limits, tool-calling behaviour and output formats still require testing. Open weights are not automatically open-source software or evidence of suitable licence terms.
Make the answer a reviewable proposal
Show original sources and relevant passages. Distinguish supported statements from inferences. When evidence is missing or contradictory, the application should make that visible. Apply the same discipline to telemetry: full prompts and responses should not be copied into broadly accessible logs by default.
Secure development needs to cover the entire application. NIST SP 800-218A extends the SSDF with practices for generative AI. [3] Running a model locally does not replace permissions, secure updates, source maintenance or domain-specific evaluation.
Turn the demo into an evidence-based pilot
1. Define a knowledge domain and an accountable owner
Choose a bounded source collection with clear ownership and an update process. Determine the lawful basis for processing, retention, responsibilities and whether a data protection impact assessment is required. Where personal data is processed, GDPR obligations remain relevant; EU hosting alone does not settle the assessment. [4]
2. Create an independent evaluation set
Use domain-reviewed questions with expected evidence. Include unanswerable questions, conflicting versions and attempts to retrieve material outside the user’s permissions. Do not rely solely on the development team’s demonstration questions. A confidence score generated by the model is not a substitute for measured performance.
3. Separate quality dimensions at acceptance
Measure retrieval relevance, factual correctness, source support, appropriate abstention and permission isolation independently. An average score must not conceal a serious access-control failure. Agree the acceptable error profile before testing. Include review time and cost per usable answer, not only model response time.
4. Close the improvement loop without uncontrolled learning
Capture structured feedback: outdated source, missing result, incorrect answer or permission concern. Improve the appropriate layer, rerun the evaluation set and release a versioned change. User feedback must not automatically become unrestricted training data containing confidential information.
Only expand to additional collections after the initial scope meets the agreed evidence and control requirements. Agents that write to business systems are a separate design step with their own permissions, approvals and operational safeguards.
What does secure knowledge search really cost?
Model tokens are only one component. Include source connectors, permission synchronisation, index maintenance, evaluation, monitoring, support and human review. A smaller model may be the economic choice when it meets the agreed task requirements. A larger model cannot compensate for unreliable source material.
An illustrative capacity model: 8,000 searches per year currently take twelve minutes each. If assisted search plus review takes seven minutes, the theoretical difference is about 667 staff hours. Source maintenance and operations must be deducted. These are planning assumptions, not measured Insight42 customer savings.
Commission evidence, not merely a working chat window
Request a data-flow map including telemetry, a permission model, deletion and update procedures, documented evaluation results and an operating plan. Include a model-switch exercise: does an alternative preserve adequate task quality, source display and the required controls?
Scope a practical first step with Insight42
Assess one knowledge workflow for value, data risk and technical feasibility. Insight42’s published focus includes secure AI, cloud and data platforms. [5] A bounded engagement can produce a use-case brief, data-flow model, architecture proposal and acceptance catalogue. Confirm these deliverables before commissioning a pilot.
Contact: support@insight42.com. Describe the source collection, intended users and the research task that causes the greatest friction. Use an abstract description rather than sending sensitive documents or case records in the first message.
Frequently asked questions about AI for government
Do we need to train our own language model?
Usually that is not the first question for internal knowledge search. Evaluate existing models with a controlled retrieval and permission layer. The measured performance on the actual task should determine whether additional adaptation or training is justified.
Can RAG guarantee correct answers?
No. It supplies evidence but does not remove retrieval errors or misinterpretation. A dependable service needs evaluations, visible sources and an explicit path for declining unsupported questions. Consequential outputs still require appropriate domain review.
Is a locally hosted model automatically GDPR-compliant?
No. Local deployment can avoid some external data flows, but lawful processing, minimisation, access control, retention and operational safeguards still need assessment. Location is one architectural property, not a complete compliance determination. [4]
Can the assistant later process cases automatically?
It can be extended, but write actions need a separate design: allowed operations, transaction boundaries, approvals, audit records and reversal procedures. Knowledge retrieval and action-taking automation have different risk profiles.
The takeaway: Sovereign AI is a controlled application built around dependable evidence, not simply a model running on an organisation-owned server.
Sources and technical references
[1] DSK: Mitteilung zur RAG-Orientierungshilfe vom 17.10.2025
[2] BSI: Indirect Prompt Injections, Warnung vom 15.07.2023
[3] NIST: SP 800-218A, Secure development for generative AI
[4] EUR-Lex: Regulation (EU) 2016/679 (GDPR)
Sources checked: 27 September 2026. Calculations and diagrams are illustrative.