Documentation

Schema policies

Policies (under organization settings) can apply to prompts, decision schemas, or both. A schema policy is checked when you publish a decision schema version. A version that fails an error-severity policy is flagged and never runs — not through the gateway, the playground, or a team agent's make_decision.

What you can write

Examples of rules organizations use:

  • Every choice option has a description a non-expert reviewer could understand.
  • Instructions must not ask the model to infer protected attributes (age, ethnicity, religion, health).
  • Score schemas use a 1–5 scale with a label on every level.
  • Inputs that carry customer PII are named *_redacted.
  • The examples cover every option at least once.

Policies are either:

  • Plain language (LLM judge) — the platform renders the schema (including examples) and asks a chat model whether it complies.
  • Structured checks — deterministic rules such as requiring descriptions, allowing only certain kinds, a minimum number of examples, input-key naming patterns, or score scale bounds.

Create them from Policies → Create Policy, choose Decision schemas under Applies to, and pick a template or a blank LLM / structured kind.

After publish

  1. The new version goes to pending while policies run.
  2. It ends passed, warning, flagged, or failed (the judge errored — re-evaluate).
  3. Flagged (and pending/failed while policies are assigned) cannot be resolved as latest, cannot be pinned, and cannot be run in the playground. Draft a new version that passes, or fix the policy and re-evaluate.

Open the schema's Governance tab for the latest evaluation, evidence linked to schema paths, and a draft Check now before you publish.

What is not covered

Ad-hoc runs that send a full schema body (playground drafts, or schemaDefinition on the API) are not reviewed by schema policies. Runtime checks on each execution are a different feature (guardrails).