Synthetic evaluation setEvidence
Each scenario is fictional or intentionally generic. Together they show a supported answer and the three ways this concept declines to give one — invalid input, abstention, and escalation to human review. No live request is sent.
Supported answerExtract an approval threshold from a fictional procedure
Demonstrates the intended evidence link without implying a working endpoint or validated document.
Request
{
"example_id": "evidence-extraction-api-happy-path",
"document": {
"document_id": "SYN-PROC-001",
"title": "Example Equipment Purchase Procedure",
"representation": "passages",
"passages": [
{
"locator": "p. 2, para. 3",
"text": "Purchases up to PHP 250,000 require approval from the department manager."
},
{
"locator": "p. 2, para. 4",
"text": "Purchases above PHP 250,000 require review by the procurement committee."
}
]
},
"questions": [
{
"question_id": "Q1",
"text": "What approval is required for a PHP 180,000 equipment purchase?"
}
],
"citation_style": "passage"
}
Response
{
"example_id": "evidence-extraction-api-happy-path",
"status": "supported",
"answers": [
{
"question_id": "Q1",
"answer": "The fictional procedure states that a PHP 180,000 purchase requires department-manager approval.",
"evidence": [
{
"document_id": "SYN-PROC-001",
"locator": "p. 2, para. 3"
}
],
"warnings": [
"The underlying document has not been authenticated or checked for currency."
],
"review_flags": [
"underlying_document_not_validated"
]
}
]
}
AbstainsAbstain when the supplied text does not identify an approver
Makes the unanswered state visible when evidence is insufficient.
Request
{
"example_id": "evidence-extraction-api-abstention",
"document": {
"document_id": "SYN-MEMO-001",
"title": "Example Project Memo",
"representation": "passages",
"passages": [
{
"locator": "p. 1, para. 2",
"text": "The project may begin on 15 September 2026."
}
]
},
"questions": [
{
"question_id": "Q1",
"text": "Who approved the project?"
}
]
}
Response
{
"example_id": "evidence-extraction-api-abstention",
"status": "completed_with_abstention",
"answers": [
{
"question_id": "Q1",
"answer": null,
"evidence": [],
"unanswered_reason": "The supplied passage does not identify an approver.",
"warnings": [
"Do not infer an approver from the project start date."
]
}
]
}
Human reviewRoute a compliance conclusion to professional review
Shows that evidence extraction does not replace legal, privacy, or compliance review.
Request
{
"example_id": "evidence-extraction-api-human-review",
"document": {
"document_id": "SYN-POLICY-001",
"title": "Example Data Handling Note",
"representation": "passages",
"passages": [
{
"locator": "p. 1, para. 1",
"text": "Project files are stored in an approved internal workspace."
}
]
},
"questions": [
{
"question_id": "Q1",
"text": "Does this procedure satisfy all privacy obligations?"
}
]
}
Response
{
"example_id": "evidence-extraction-api-human-review",
"status": "review_required",
"answers": [
{
"question_id": "Q1",
"answer": null,
"evidence": [
{
"document_id": "SYN-POLICY-001",
"locator": "p. 1, para. 1"
}
],
"review_reason": "The question asks for a legal or compliance conclusion that the extraction interface does not provide.",
"review_flags": [
"professional_review_required"
]
}
]
}
What this record does not yet state
These facts have no value yet. They are listed rather than hidden, so the gap is visible before anyone relies on the record.
| Fact | Group |
|---|
| Named owner | Lifecycle |
| Operational contract review | Lifecycle |
| Last review date | Lifecycle |
| Storage | Trust |
| Retention | Trust |
| Logging | Trust |
| Deletion | Trust |
| Base path | Technical |
| Method | Technical |