Support standalone runners and complete app-folder reviews (#172)

* Document standalone review runner contract

Keep model selection and scheduling outside BCQuality while allowing orchestrators to run isolated review leaves concurrently with deterministic rollup semantics.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Support complete app folder reviews

Define folder-path as a current-state review scope and accept it across the standalone adapter, broad coordinator, and every AL review leaf.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Add walk-up app review quick start

Put the complete app-folder installation and prompt flow directly in the README so partners can discover the standalone experience without reading integration details first.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Organize conceptual guides under docs

Move architecture and standalone runner documentation out of the repository root, add a documentation index, and update all inbound links.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Jesper Schulz-Wedde <jesper.schulzwedde@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Jesper Schulz-Wedde 2026-09-09 16:55:35 +02:00 • committed by GitHub
parent 8584217c75
commit 17bb84a25e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 229 additions and 51 deletions

View file

@ -4,7 +4,7 @@ id: al-code-review
version: 1
title: AL code review
description: Reviews AL source changes by composing the AL review leaf skills, one per knowledge domain.
inputs: [pr-diff, file-path]
inputs: [pr-diff, file-path, folder-path]
outputs: [findings-report]
bc-version: [all]
technologies: [al]
@ -35,7 +35,7 @@ Reviews AL source changes by composing the leaf AL review skills. This is the ca
`al-code-review` does not evaluate knowledge files directly. It invokes each of its sub-skills against the same task input, collects their findings-reports, and then performs its own **self-review pass** over the diff using the agent's built-in BC and AL knowledge. BCQuality knowledge is an additive layer: anything the sub-skills found is cited from BCQuality, and anything the agent finds on its own is validated against BCQuality (cited if matched, suppressed if contradicted, surfaced as an **agent finding** otherwise). The result is a single rolled-up findings-report that mixes knowledge-backed and agent findings, each clearly tagged via `from-sub-skill`.
An orchestrator invokes this skill with either a `pr-diff` (the standard PR-review entry point) or a `file-path` (single-file review). The skill produces a single JSON document conforming to the DO output contract, extended with `sub-results` and — when applicable — `skipped-sub-skills`.
An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-path`. The skill produces a single JSON document conforming to the DO output contract, extended with `sub-results` and — when applicable — `skipped-sub-skills`.
## Source
@ -63,18 +63,18 @@ The worklist is the list of sub-skills judged relevant by the previous step. Eve
### Execution discipline (mandatory)
The Action step is a sequence of **discrete iterations**, not one combined generation. The contract requires the super-skill to invoke each sub-skill in turn and then perform a self-review pass. Concretely this means:
The Action step consists of **discrete leaf invocations**, not one combined generation. Invocation scheduling belongs to the orchestrator: independent leaves may run serially or concurrently, but their evaluation contexts and findings-reports remain isolated. Concretely this means:
- **Isolate leaf invocations when the host supports it.** For fast/small models, each sub-skill SHOULD run in a fresh model call or child context containing only the task input, READ/DO contracts, the leaf instructions, a domain-filtered slice of the current knowledge index, and articles that leaf worklists. Preserve each index row's exact `path`; the leaf must copy references from that slice. The coordinator then collects the resulting JSON. This is the preferred fast-model profile: it bounds context, prevents later leaves from being skipped as attention is exhausted, and removes any reason to synthesize article paths.
- Treat each sub-skill in the worklist as its own pass: read the sub-skill's instructions, apply its Source → Relevance → Worklist → Action steps to the orchestrator-supplied inputs, and produce that sub-skill's complete findings-report before moving on.
- Treat each sub-skill in the worklist as its own pass: read the sub-skill's instructions, apply its Source → Relevance → Worklist → Action steps to the orchestrator-supplied inputs, and produce that sub-skill's complete findings-report independently.
- Do not collapse multiple sub-skills into one shared reasoning step. Each sub-skill has a distinct knowledge subset and a distinct evaluation procedure; sharing one rolled-up scan dilutes per-skill attention and causes leaves to silently underreport (this has been observed in production: leaf skills returned empty `findings[]` while their standalone runs against the same diff produced multiple matches).
- The agent self-review pass is its own final iteration. Begin it only after every sub-skill in the worklist has completed and its sub-result is recorded.
- Sub-skills are independent: re-walking the diff once per sub-skill is correct and expected. The output schema accommodates this — `sub-results` carries one entry per sub-skill, each a complete findings-report.
- Sub-skills are independent: re-walking the diff once per sub-skill is correct and expected. The output schema accommodates this — `sub-results` carries one entry per sub-skill, each a complete findings-report, in the frontmatter `sub-skills` order regardless of completion order.
- When isolated calls are unavailable and the current model cannot finish every leaf within its budget, return `partial` with completed `sub-results` and name the first unevaluated sub-skill in `outcome-reason`. Never silently mark the remaining leaves clean.
### Roll up sub-skill findings
For each sub-skill in the worklist, executed one at a time per the discipline above:
For each sub-skill in the worklist:
1. Invoke the sub-skill with the orchestrator's inputs, passing only the subset each sub-skill declares in its `inputs`.
2. Capture the sub-skill's complete findings-report verbatim and append it to `sub-results`.
@ -116,7 +116,7 @@ Sub-skills MAY also emit `suggested-code` when their knowledge file unambiguousl
### Summary and rollup
Aggregate `summary.counts` and `summary.coverage` as the sums across invoked sub-skills whose `outcome` is not `failed`. Agent findings emitted by the super-skill itself contribute to `summary.counts` but not to `summary.coverage` (coverage is a sub-skill worklist metric and is undefined for self-review).
Calculate `summary.counts` from the final top-level `findings[]`, after failed sub-results have been excluded and duplicates have been merged. Aggregate `summary.coverage` as the sums across invoked sub-skills whose `outcome` is not `failed`. Agent findings emitted by the super-skill itself contribute to `summary.counts` but not to `summary.coverage` (coverage is a sub-skill worklist metric and is undefined for self-review).
`suppressed[]` at the super-skill level remains empty. Knowledge-file-level suppression is reported by each sub-skill within its own entry in `sub-results`.