mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-08-07 01:46:53 +01:00
Add AL code generation skill contracts
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd6344b4-eafd-4a58-a1c9-5a2d96fa2938
This commit is contained in:
parent
be1b92b624
commit
b0d71418fa
17 changed files with 1885 additions and 39 deletions
|
|
@ -19,23 +19,23 @@ flowchart LR
|
|||
O[Orchestrator<br/>AL-Go] -->|1 trigger + task context| A[Agent]
|
||||
A -->|2 invoke entry.md| E[Entry<br/>routing skill]
|
||||
E -->|3 dispatch record| A
|
||||
A -->|4 invoke dispatched skill| S[Action skill<br/>e.g. al-code-review]
|
||||
A -->|4 invoke dispatched skill| S[Action skill<br/>review or generation]
|
||||
S -->|5 execute| P[Source → Relevance<br/>→ Worklist → Action<br/>reading READ · DO on demand]
|
||||
P -->|6 emit| R[Findings · References<br/>· Confidence]
|
||||
P -->|6 emit negotiated contract| R[Findings report<br/>or generated-files report]
|
||||
R -->|7 integrate| O
|
||||
```
|
||||
|
||||
### 1. Orchestrator triggers
|
||||
The orchestrator has a URL setting that points at BCQuality (default: `github.com/microsoft/BCQuality`) and a task to perform. It hands the agent a **task context** — goal, inputs available (`pr-diff`, `file-path`, …), technologies, BC version, enabled layers — and says: *your source of truth lives at that URL; start by invoking `/skills/entry.md`*.
|
||||
The orchestrator has a URL setting that points at BCQuality (default: `github.com/microsoft/BCQuality`) and a task to perform. It hands the agent a **task context** — goal, optional explicit `action`, inputs available (`pr-diff`, `file-path`, `requirement-spec`, …), accepted output kind/version, technologies, BC version, and enabled layers — and says: *your source of truth lives at that URL; start by invoking `/skills/entry.md`*.
|
||||
|
||||
### 2. Agent invokes Entry
|
||||
The agent reads `/skills/entry.md` and runs it against the task context. Entry applies its Source → Relevance → Worklist → Action steps over the action skills under `*/skills/**/*.md` and returns a **dispatch record**: the set of action skills to invoke, plus a list of candidates it skipped (with reasons). Routing is a skill, not orchestrator logic.
|
||||
|
||||
### 3. Agent consumes the dispatch record
|
||||
The dispatch record names one or more action skills and the subset of inputs each should receive. If the outcome is `no-match` or `failed`, the agent returns the record to the orchestrator unchanged.
|
||||
The dispatch record names one or more action skills, the subset of inputs each should receive, and each skill's declared output kind/version. If the outcome is `no-match` or `failed`, the agent returns the record to the orchestrator unchanged.
|
||||
|
||||
### 4. Agent invokes each dispatched action skill
|
||||
Action skills live inside the layers — `/microsoft/skills/`, `/community/skills/`, `/custom/skills/` — so their authority is carried by their location. For a PR review, Entry typically dispatches `microsoft/skills/review/al-code-review.md`. The agent reads the file and executes it.
|
||||
Action skills live inside the layers — `/microsoft/skills/`, `/community/skills/`, `/custom/skills/` — so their authority is carried by their location. For a PR review, Entry typically dispatches `microsoft/skills/review/al-code-review.md`. For explicit bounded generation, it dispatches `microsoft/skills/generate/al-code-generation.md`. The agent reads the file and executes it.
|
||||
|
||||
### 5. Action skill executes the four-step pattern
|
||||
|
||||
|
|
@ -54,14 +54,14 @@ At this point the agent reads READ and DO on demand — it needs READ to interpr
|
|||
|
||||
### 5a. The knowledge index (Source acceleration)
|
||||
|
||||
Discovering candidates at the Source step naively means opening every file under a domain folder just to read its frontmatter `keywords` — on a large corpus that is hundreds of file reads per review. To avoid this, BCQuality maintains a **knowledge index**: a single artifact (`knowledge-index.json`) that lists every article surviving the consumer's layer/allow-deny filtering and carries, per article, the exact inputs the Source/Worklist steps consume — `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint.
|
||||
Discovering candidates at the Source step naively means opening every file under a domain folder just to read its frontmatter `keywords`. To avoid this, BCQuality maintains a **knowledge index**: a single artifact (`knowledge-index.json`) that lists every article surviving the consumer's layer/allow-deny filtering and carries, per article, the exact inputs the Source/Worklist steps consume — `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint. Review retains its documented fallback; generation is index-only and never blindly walks the repository.
|
||||
|
||||
The index is **owned and produced by BCQuality**, not by each consumer: its generator (`tools/Build-KnowledgeIndex.ps1`) ships here, next to the skills and knowledge it derives from, so the index schema stays in lockstep with the Source contract and every consumer gets the same faithful index for free instead of re-implementing the parser. The consuming orchestrator does **not** build or invoke the index — it only prunes its clone to policy as it already does. The index is then (re)generated by BCQuality itself: **Entry's preparation step runs `Build-KnowledgeIndex.ps1` over the live, already-pruned clone** at the start of every run (see `skills/entry.md`), and BCQuality CI (`.github/workflows/knowledge-index.yml`) validates that the generator is healthy and deterministic. Building over the *pruned* clone — rather than shipping a committed full-corpus index that consumers trust — keeps the index exact for any consumer policy: it can never list an article the consumer denied, so policy-excluded rules cannot leak into discovery.
|
||||
|
||||
The index changes only *how candidates are discovered*, never *which are selected*. The Worklist predicate is unchanged — `keywords` still drive selection — and the agent still opens each worklisted article **in full** to read its `## Best Practice` / `## Anti Pattern` rule bodies; the index is discovery metadata only and never substitutes for the article body. When no index is present, skills fall back to path-based discovery (collect by domain folder), so review still works.
|
||||
|
||||
### 6. Agent emits structured output
|
||||
The output contract is defined in the DO meta-skill so that every action skill — today's and next year's — produces the same shape:
|
||||
Output contracts are defined in DO and published schemas so consumers negotiate the exact kind/version rather than infer it from skill identity. A findings report carries:
|
||||
|
||||
- **Outcome** — `completed`, `not-applicable`, `no-knowledge`, `partial`, or `failed`. An orchestrator can distinguish a clean run from a no-op from a failure without guessing.
|
||||
- **Findings** — what the skill observed (severity, message, optional location).
|
||||
|
|
@ -69,10 +69,28 @@ The output contract is defined in the DO meta-skill so that every action skill
|
|||
- **Confidence** — per-finding evidence strength.
|
||||
- **Suppressed** — knowledge files that were discarded by layer precedence or configuration, so reviewers can see what was overridden.
|
||||
|
||||
The orchestrator parses this **without skill-specific logic**. This is the point of the contract: orchestrators and action skills evolve independently.
|
||||
The orchestrator parses this **without skill-ID-specific logic**. For generation, it validates [`requirement-spec` v1](schemas/requirement-spec-v1.schema.json) and [`generated-files-report` v1](schemas/generated-files-report-v1.schema.json) atomically and fails closed before materialization.
|
||||
|
||||
### 7. Orchestrator integrates
|
||||
The orchestrator turns findings into PR comments, build gates, or IDE diagnostics, and links the references back to the knowledge files so the PR author — human or agent — can read the guidance.
|
||||
For review, the orchestrator turns findings into PR comments, build gates, or IDE diagnostics. For generation, BCQuality stops at a create-only report: the consumer independently enforces symlink, destination, size, and ID-range policy before staging, then owns compilation, analysis, tests, delivery, approval, and publishing.
|
||||
|
||||
## Generation capability and staged rollout
|
||||
|
||||
Generation routing is deterministic:
|
||||
|
||||
| Inputs and capability | Result |
|
||||
| --- | --- |
|
||||
| `action: generate`, `requirement-spec`, accepts `generated-files-report` v1 | Generation only |
|
||||
| Legacy `pr-diff` or `file-path`, no action | Review only |
|
||||
| Both input families with `action: generate` | Generation only |
|
||||
| Both input families with `action: review` | Review only |
|
||||
| Both input families without action | Failed as `ambiguous-action` |
|
||||
|
||||
Entry never relies on fuzzy goal text when both families are present. A consumer such as current BCAppsBCQuality that does not advertise `action`, `requirement-spec`, and `generated-files-report` v1 remains generation-ineligible.
|
||||
|
||||
BCQuality owns contracts, routing, index-only retrieval, deterministic ranking, guidance application, and immutable report references. Consumer work is deliberately deferred and staged: first strict parser fixtures, then a dry-run patch artifact, then isolated compilation/analysis/test validation, then an environment-approved draft PR. Consumers continue to own delivery, human approval, and publishing.
|
||||
|
||||
Stable schema paths, the stable `al-code-generation` skill ID, immutable commit-scoped references, and explicit candidate/worklist/omission coverage are designed to let BC-Bench distinguish Haiku baseline routing/retrieval failures from treatment-generation failures.
|
||||
|
||||
## Knowledge-backed and agent findings
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue