Create an optional hook system for externally-managed verification

This commit is contained in:
Ohad Livne 2026-02-10 18:52:52 +02:00
parent e8250ba10c
commit 320322d48b
Signed by: libohad-dev
GPG key ID: 34FDC68B51191A4D
5 changed files with 99 additions and 9 deletions

View file

@ -75,11 +75,13 @@ After completing a task, if the next task's dependencies are all met, continue i
After completing a task, check whether any verification milestones in `spec/tasks.md` have all their tasks complete. For each newly-completed milestone:
**a. Run the milestone's verification method** (e.g., integration tests, property-based tests, recovery tests). These are the tests that validate TLA+ properties end-to-end across multiple components.
**a. Run the milestone's verification**: If `spec/hooks/run-checks` exists and is executable, delegate the milestone's verification to it (see `/formspec.init` for the hook interface). The request should include the milestone's properties, verification methods, and hints (e.g., the test commands from the plan). The external system runs the checks independently and returns results.
**b. Handle milestone verification failures**: If the integration test fails, diagnose whether the issue is in one of the component tasks (go back and fix it) or a cross-component interaction that wasn't caught by unit tests. If it reveals a spec inconsistency, STOP as described above.
If the hook does not exist or fails, run the milestone's verification method locally (e.g., integration tests, property-based tests, recovery tests). These are the tests that validate TLA+ properties end-to-end across multiple components.
**c. Update the verification matrix**: After the milestone's verification passes, update `spec/verification/matrix.md`. Create the file if it doesn't exist, using this format:
**b. Handle milestone verification failures**: If the verification fails (whether reported by the hook or observed locally), diagnose whether the issue is in one of the component tasks (go back and fix it) or a cross-component interaction that wasn't caught by unit tests. If it reveals a spec inconsistency, STOP as described above.
**c. Update the verification matrix**: If the hook succeeded, it is the record-keeper — skip the markdown table update. If checks ran locally (no hook, or hook failed), update `spec/verification/matrix.md` directly. Create the file if it doesn't exist, using this format:
```markdown
# Verification Matrix