Schema policies
Policies (under organization settings) can apply to prompts, decision schemas, or both. A schema policy is checked when you publish a decision schema version. A version that fails an error-severity policy is flagged and never runs — not through the gateway, the playground, or a team agent's make_decision.
What you can write
Examples of rules organizations use:
- Every choice option has a description a non-expert reviewer could understand.
- Instructions must not ask the model to infer protected attributes (age, ethnicity, religion, health).
- Score schemas use a 1–5 scale with a label on every level.
- Inputs that carry customer PII are named
*_redacted. - The examples cover every option at least once.
Policies are either:
- Plain language (LLM judge) — the platform renders the schema (including examples) and asks a chat model whether it complies.
- Structured checks — deterministic rules such as requiring descriptions, allowing only certain kinds, a minimum number of examples, input-key naming patterns, or score scale bounds.
Create them from Policies → Create Policy, choose Decision schemas under Applies to, and pick a template or a blank LLM / structured kind.
After publish
- The new version goes to pending while policies run.
- It ends passed, warning, flagged, or failed (the judge errored — re-evaluate).
- Flagged (and pending/failed while policies are assigned) cannot be resolved as
latest, cannot be pinned, and cannot be run in the playground. Draft a new version that passes, or fix the policy and re-evaluate.
Open the schema's Governance tab for the latest evaluation, evidence linked to schema paths, and a draft Check now before you publish.
What is not covered
Ad-hoc runs that send a full schema body (playground drafts, or schemaDefinition on the API) are not reviewed by schema policies. Runtime checks on each execution are a different feature (guardrails).