Build the assistant around a versioned policy register, not a folder of loosely named PDFs. Retrieve the policy that applies to the question and relevant date, then show its owner, version, effective date and supporting section beside the answer. Application code should verify that citation before displaying it. When evidence is missing or conflicts, the assistant should explain the gap and route the question to a person.
The proposed workflow below is for policy explanation and preparation for human review. It does not authorise payments, employment decisions, legal conclusions or security exceptions.
1. Define what a usable policy answer must contain
A usable answer should identify both the rule and the exact policy revision supporting it. A document title alone cannot tell an employee whether the assistant used today's rule or a superseded one.
Start with a narrow policy family, such as travel booking procedures. Ask its owner to define the questions the assistant may explain, the audiences it serves and the decisions it must leave to people. Write these boundaries into the application requirements before choosing a model.
Propose a consistent answer layout: a short explanation, the applicable date, a supporting section, a version citation and any unresolved conditions. If the question involves an exception, distinguish the ordinary rule from the exception approval process.
For example, an assistant may explain which evidence a travel request needs. It should not declare the request approved. For employment, tax, legal, payment or security consequences, the relevant professional or accountable manager must judge the case.
The distinction between AI agents and AI assistants helps scope this build: answering policy questions does not require giving the system authority to act on them.
2. Create a policy register with immutable revisions
Create one record for each policy revision, and keep its identity stable after publication. Do not replace an older revision's content while retaining the same version label.
The proposed register should contain a stable policy key, title, version label, owner, effective-from date, effective-until date where known, publication state, audience, business scope and a link to the exact revision. Keep a separate technical mapping between that record and the indexed file. A content fingerprint can help identify copies that claim to be the same revision but contain different text.
Treat approval date, upload date and effective date as separate fields. A document uploaded most recently may describe a future rule. An older upload may still be the applicable policy.
Ask the owner to resolve missing dates and uncertain scope before making a document eligible for answers. Do not infer authority from filenames such as “final” or “latest”.
Retain superseded revisions for authorised historical questions, but exclude them from ordinary current-policy retrieval. This makes history available without inviting the assistant to mix old and new rules.
3. Select applicable revisions before searching their content
Resolve scope and date before asking the model to explain a policy. Semantic similarity should find relevant passages within eligible evidence, not decide which revision has authority.
Use an explicit question date when supplied. Otherwise, a proposed default is the current date in the business's agreed timezone, shown in the answer. Ask a clarification when the event date matters, such as a booking made before a policy change for travel after it.
Application code should check the user's permitted audience and business scope, then select eligible revisions from the register. Where multiple revisions overlap without a documented precedence rule, return a conflict rather than choosing the higher version number.
Source: OpenAI's function calling guide describes a flow in which the model requests a function and the application executes it. A proposed read-only function could resolve eligible policy records using validated inputs. Your application must enforce permissions and date rules itself.
Never let a user-supplied department or a model-generated function argument grant access. A security owner should assess access controls, logging and provider data handling before sensitive policies enter the system.
4. Retrieve passages and preserve their evidence trail
Retrieve from the eligible revisions and retain the passage-to-file mapping throughout answer generation. This is where a document search capability becomes useful, but it is only one part of the design.
Source: OpenAI's file search documentation describes semantic and keyword search over uploaded files in vector stores, with file citations in the response. It also instructs developers to check that files are ready before using them. Those capabilities do not establish which business policy is authoritative.
In this proposed design, use the register to constrain the searchable evidence, then map every retrieved file back to its revision record. If your selected integration cannot reliably constrain retrieval, separate eligible evidence into controlled search sets or use an application-managed retrieval step. Do not rely solely on a prompt saying “use the latest policy”.
Preserve headings, numbered clauses, tables and exceptions when preparing documents. A passage about a limit may depend on the following paragraph about exclusions.
This is a practical use of retrieval-augmented generation: supplied evidence supports the answer, while your application determines which evidence may be used.
5. Render citations from verified records, not generated labels
Build the visible citation from trusted register fields after validating the answer's evidence references. Do not ask the model to invent a version label or construct a document URL.
A proposed response structure includes answer text, applicable date, evidence references, supporting section and an outcome such as answered, clarification needed or owner review needed. Allow an empty evidence list for a blocked response rather than forcing a plausible citation.
Source: OpenAI's structured outputs guide describes schema-constrained responses. This can support a predictable response layout, but valid structure is not proof that a passage supports an answer. Check compatibility with the chosen model and API before adopting the schema.
For each evidence reference, application code should verify the file mapping, applicable revision, user access and resolvable section. Review whether the passage actually supports each material claim. A correct citation attached to an unsupported conclusion still fails.
Render a citation in a consistent form: “Policy title | version | effective from | owner | section”, linked to the exact authorised revision. For answers using several policies, attach citations to the relevant claims rather than putting an undifferentiated list underneath.
6. Test questions whose answers changed between revisions
Test the assistant against changed policy answers, not just questions that work equally well with every revision. This reveals whether version selection affects the response correctly.
Ask policy owners to prepare paired questions from actual revisions. Record the question, audience, relevant date, expected revision, supporting section, expected explanation and expected human route. These become a reference set for evaluation, not evidence of performance before testing.
Include current questions, historical questions, future-effective policies, overlapping dates, missing metadata, inaccessible documents and duplicate version labels. Also test a conversation that moves from a historical question to a current one. The assistant must reselect evidence rather than carry the old revision forward.
Score failures separately: wrong revision, wrong rule, unsupported claim, broken link, access failure and missed escalation. A proposed release gate is that every critical case passes, with unresolved failures blocking the affected policy family. The business owner should decide what counts as critical.
A team exploring custom AI agents should use these cases as build requirements. Potential improvements in traceability need evaluation; a vendor's search capability alone does not prove fewer errors or better compliance.
7. Assign human ownership to exceptions and policy changes
Give every unresolved policy question a named human route, and retest affected answers whenever the policy changes. An assistant that only says “contact someone” leaves the employee with another search task.
For missing evidence, identify the policy owner or helpdesk route. For conflicting revisions, describe the conflict without deciding which document wins. For case-specific exceptions, show the normal rule and the person authorised to consider an exception, where the policy supports that route.
Send reviewers the question, relevant date, permitted context, candidate revisions and unresolved issue. Avoid copying unnecessary personal information into logs or escalation messages. Appropriate security and privacy specialists should decide retention and access arrangements.
When an owner publishes a revision, check its metadata, prepare its searchable content and rerun changed-answer tests before making it eligible. Revalidate cached answers when their supporting revision changes. Keep a way to withdraw an affected policy family while corrections are made.
For broader planning, the AI automation service can frame the integration work, while what AI agents are explains the surrounding concepts. The immediate requirement remains a read-only, traceable answer workflow.
Confirm readiness before release
Use this proposed checklist for each policy family before allowing staff to rely on its answers. Record pass, fail or not applicable beside every item, with an accountable owner for unresolved items.
Policy-version acceptance checklist
- Boundary: Define permitted questions, audiences and decisions reserved for people.
- Identity: Assign a stable policy key and an immutable record for each revision.
- Metadata: Confirm title, version, owner, effective dates, publication state and business scope.
- Evidence: Link each indexed file and supporting section to its exact revision.
- Access: Enforce user permissions before retrieval and when opening source links.
- Selection: Resolve the relevant date and scope before choosing eligible revisions.
- Conflicts: Block missing metadata, overlapping authority and mismatched duplicate content.
- Output: Show the answer, applicable date, version citation, section and unresolved conditions.
- Validation: Reject unknown evidence references and unsupported material claims.
- Testing: Check changed answers, historical dates, future revisions and denied access.
- Human route: Name the owner who handles ambiguity and consequential exceptions.
- Maintenance: Retest changed clauses and withdraw affected answers when evidence fails.
Proposed completion rule: The policy owner confirms the expected answers, the technical owner confirms evidence mappings, and the security owner confirms access boundaries. Unresolved critical failures keep that policy family unavailable for ordinary answers.
Worked example: current answer, duplicate and missing date
The assistant should answer the normal case and stop when authority is uncertain. Every record, date, amount and rule in this walkthrough is hypothetical.
Suppose a travel policy's version 2 allows accommodation requests up to R1,200 per night from 1 January to 31 August 2026. Version 3 allows requests up to R1,500 from 1 September 2026. Both require separate manager approval, and the hypothetical owner is Travel Operations.
Normal case: An employee asks which limit applies to a booking on 5 October 2026. The application selects version 3 and retrieves its accommodation clause. The answer explains the hypothetical R1,500 limit, preserves the approval condition and cites version 3, its effective date, owner and section. It does not approve the booking.
Duplicate case: Two files claim to be version 3, but one states R1,400. The proposed validator detects differing content under one revision identity. The assistant withholds the limit and routes both candidates to Travel Operations. The owner identifies the authoritative revision; the technical team repairs the mapping and reruns the test.
Missing-date case: A file labelled version 4 has no effective date. The assistant does not treat its higher number as authority. The owner must resolve whether it is future policy, current policy or an unpublished document before it becomes eligible evidence.
FAQs
Can we use the upload date as the policy effective date?
No. In this proposed workflow, upload date records when a file entered the system; effective date determines when its rule applies. Keep both if useful, but ask the policy owner to confirm the effective date. If that date is missing and affects the answer, block the revision rather than guessing. The employee should receive a clear explanation and a named route for clarification.
What should happen when an employee asks about an old policy?
Use the question's relevant historical date to select an authorised archived revision, and label the answer as historical. Do not silently substitute today's policy. If the question asks whether an old event was lawful, payable or grounds for an employment decision, the assistant can assemble the policy evidence, but the relevant human specialist must interpret the case. Historical retrieval is not a legal conclusion.
Is a file citation enough to prove the answer used the right version?
No. A file citation identifies evidence, but your application still needs to verify the revision mapping, date, scope and supporting passage. It should also check that the cited rule supports the actual claim. Test these checks with deliberately wrong revisions and mismatched copies. A neat citation format can make an answer easier to inspect; it cannot by itself establish correctness.
If your business needs policy answers that staff can check against exact revisions, start with one policy family and its changed-answer tests. For help scoping the register, validation and human handover, get in touch with Symaxx.