Case study 01 / Healthcare
AI Clinical Operations
A retrieval-augmented operations assistant that helps clinical operations staff find policy, scheduling and routing information, with human review at every decision point.
Illustrative reference architecture, not a client engagement. Outcomes described here are design goals, not measured results.
Reference flow
05 stages
01Challenge
Why this problem is hard.
01
Fragmented sources
Policies, protocols and scheduling rules live in document libraries, intranet pages, ticketing systems and spreadsheets, with no single index across them.
02
Permission boundaries
Not every document is visible to every role. Retrieval that ignores source permissions leaks information across departments.
03
Version drift
Superseded policies remain discoverable. Staff need the version in force today, with its effective date and owner.
04
Accountability
Any suggested action must be attributable to a source and reviewed by a person before it affects a schedule, a referral or a patient.
02Business context
Assumptions and constraints.
Constraints the design must respect
- 01Privacy by design
- Protected health information is kept out of the retrieval corpus by default. Any PHI processing requires a deployment engineered and contracted for it.
- 02Systems of record stay authoritative
- The EHR and scheduling systems remain the source of truth. Integrations are read-only in the first release.
- 03Human oversight
- The assistant drafts and cites; staff decide. Every suggested action is reviewed before it is used.
- 04Reproducible answers
- Each answer records its sources, document versions, model and prompt version, and the person who acted on it.
03Architecture
Reference architecture.
Connectors pull policies, procedures and scheduling rules from document libraries and operational systems on a schedule and on change events. Each document carries its source permissions, owner, effective date and version.
- Document connectors
- Change detection
- Metadata extraction
- ACL capture
Concept architecture
01 / 07
Source ingestion
Connectors pull policies, procedures and scheduling rules from document libraries and operational systems on a schedule and on change events. Each document carries its source permissions, owner, effective date and version.
- Document connectors
- Change detection
- Metadata extraction
- ACL capture
Key patterns
- Permission-aware RAG
- Scoped read-only tools
- Citation enforcement
- Evaluation gates
04Technology
Representative technology.
01
05 items
AI & retrieval
- LLM via private endpoint
- Embedding models
- Hybrid search index
- Re-ranking model
- Agent orchestration layer
02
04 items
Data & integration
- Document connectors
- FHIR APIs (read-only, where needed)
- Event-driven sync
- Metadata catalog
03
04 items
Security
- SSO with OIDC
- Role- and attribute-based access control
- Secrets management
- Private networking
04
04 items
Operations
- OpenTelemetry tracing
- Evaluation harness
- Prompt and model registry
- Infrastructure as code
05Engineering approach
Principles behind the design.
- 01
Permissions before relevance
Access filtering happens inside retrieval, not after generation. A model never sees a chunk the user could not open in the source system.
- 02
Cite or decline
The assistant answers only from retrieved sources. When retrieval finds nothing adequate, it says so and points to the owning team.
- 03
Narrow tools, no write paths
Agents call a small set of read-only tools with explicit schemas. Write actions stay with staff, in the systems they already use.
- 04
Evaluation as a release gate
Changes to prompts, models or the index ship only when the regression suite passes, with results stored next to the release.
06Implementation
A phased delivery path.
Delivery path
04 phases
- 01Discovery & risk framing
- 02Retrieval foundation
- 03Assisted workflows
- 04Scale & operate
- Phase 0101
Discovery & risk framing
Map question types, sources, permission models and failure consequences with operations, privacy and security stakeholders.
Deliverables
- Use-case inventory
- Source and ACL map
- Risk register
- Evaluation question set
- Phase 0202
Retrieval foundation
Build ingestion, permission-aware hybrid retrieval and cited answers for a single department.
Deliverables
- Priority connectors
- Index with ACL filters
- Citation interface
- Retrieval quality baseline
- Phase 0303
Assisted workflows
Add scoped tools, guardrails and the review workflow, then run with a pilot group in read-only mode.
Deliverables
- Tool registry
- Guardrail policies
- Review and feedback loop
- Pilot runbook
- Phase 0404
Scale & operate
Extend to further sites and departments with monitoring, audit export and an ownership model for content.
Deliverables
- Multi-site rollout plan
- Dashboards and alerts
- Content ownership model
- Operational handover
07Intended outcomes
Design goals, not results.
Design goal 01Intended
The current answer, quickly
Staff reach the policy or procedure in force, with its source and effective date, without searching several systems.
Design goal 02Intended
No widened access
Retrieval applies the same access rules as the source systems, so the assistant cannot expose content a user could not already open.
Design goal 03Intended
Every answer traceable
Any response can be reconstructed from its trace: sources, versions, model, prompt and the reviewer's decision.
Design goal 04Intended
Safe to change
Regression gates on prompt, model and content updates protect known-good answers as the system evolves.
No figures are attached to these goals. Actual results depend on the environment, data and delivery, and would be measured against baselines agreed at the start of a real engagement.
08Lessons & risks
Engineering lessons and risks to manage.
Risk 01
Content quality sets the ceiling
Retrieval cannot fix outdated or contradictory policies. Content needs named owners and review dates before launch.
Risk 02
Permission sync is a live dependency
Access lists change. If index permissions lag the source, the design must fail closed and re-check access at query time.
Risk 03
Fluent answers invite over-trust
Citations, effective dates and a clear not-found state reduce automation bias. Interface design is part of the safety case.
Risk 04
Evaluation sets decay
Approved answers must be reviewed as policies change, or the regression suite starts protecting the wrong behavior.
Start an engineering conversation
Facing a similar engineering problem?
Bring us the problem and the constraints around it. We'll help architect the system.