mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-10-05 14:46:55 +01:00
Organize conceptual guides under docs
Move architecture and standalone runner documentation out of the repository root, add a documentation index, and update all inbound links. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
parent
e92a1dc833
commit
7212e921ee
5 changed files with 18 additions and 7 deletions
8
docs/README.md
Normal file
8
docs/README.md
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
# Documentation
|
||||
|
||||
- [How agents consume BCQuality](agent-consumption.md) explains the operational
|
||||
flow from Entry dispatch through structured findings and integration.
|
||||
- [Build a lightweight standalone review runner](standalone-runner.md) explains
|
||||
the walk-up app-folder flow and how an external runner can add model
|
||||
selection, concurrency, retries, and telemetry without moving orchestration
|
||||
into BCQuality.
|
||||
112
docs/agent-consumption.md
Normal file
112
docs/agent-consumption.md
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
# How agents consume BCQuality
|
||||
|
||||
BCQuality is content — knowledge files and skills. It is consumed by agents that live elsewhere (AL-Go, a VS Code extension, a GitHub Agent invocation, etc.). This document explains the end-to-end flow, so that skill authors, orchestrator maintainers, and contributors share one mental model.
|
||||
|
||||
For the high-level framing and repo structure, start with the
|
||||
[README](../README.md). This document is the operational view.
|
||||
|
||||
## The actors
|
||||
|
||||
- **Orchestrator** — the tool that triggers work (e.g. AL-Go on a pull request, or a VS Code extension on save). Lives *outside* BCQuality. Knows *when* to run something, not *what* to run.
|
||||
- **Agent** — an LLM-driven process spawned by the orchestrator. The agent has no built-in knowledge of BC or of BCQuality's conventions. It knows how to read instructions and call tools.
|
||||
- **BCQuality repo** — two kinds of content:
|
||||
- **Global skills** in `/skills/` — the `entry.md` entry-point skill plus the READ · DO · WRITE contracts that govern the rest of the repo.
|
||||
- **Layer content** in `/microsoft/`, `/community/`, and `/custom/` — knowledge files and action skills grouped by authority.
|
||||
|
||||
When BCQuality is installed as a standalone plugin, it additionally exposes
|
||||
`skills/al-code-review/SKILL.md`. This is a host-format adapter, not another
|
||||
action skill: it creates the task context and enters the same flow at Entry.
|
||||
|
||||
## The flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
O[Orchestrator<br/>AL-Go] -->|1 trigger + task context| A[Agent]
|
||||
A -->|2 invoke entry.md| E[Entry<br/>routing skill]
|
||||
E -->|3 dispatch record| A
|
||||
A -->|4 invoke dispatched skill| S[Action skill<br/>e.g. al-code-review]
|
||||
S -->|5 execute| P[Source → Relevance<br/>→ Worklist → Action<br/>reading READ · DO on demand]
|
||||
P -->|6 emit| R[Findings · Domain labels<br/>· References · Confidence]
|
||||
R -->|7 integrate| O
|
||||
```
|
||||
|
||||
### 1. Orchestrator triggers
|
||||
The orchestrator has a URL setting that points at BCQuality (default: `github.com/microsoft/BCQuality`) and a task to perform. It hands the agent a **task context** — goal, inputs available (`pr-diff`, `file-path`, …), technologies, BC version, enabled layers — and says: *your source of truth lives at that URL; start by invoking `/skills/entry.md`*.
|
||||
|
||||
### 2. Agent invokes Entry
|
||||
The agent reads `/skills/entry.md` and runs it against the task context. Entry applies its Source → Relevance → Worklist → Action steps over the action skills under `*/skills/**/*.md` and returns a **dispatch record**: the set of action skills to invoke, plus a list of candidates it skipped (with reasons). Routing is a skill, not orchestrator logic.
|
||||
|
||||
For a standalone plugin installation, the host activates the
|
||||
`skills/al-code-review/SKILL.md` adapter first. That adapter preserves the
|
||||
caller's actual goal, constructs the task context, and invokes Entry. It does
|
||||
not select the internal `microsoft/skills/review/al-code-review.md` action skill
|
||||
itself or duplicate Entry's preparation, routing, and failure semantics.
|
||||
|
||||
### 3. Agent consumes the dispatch record
|
||||
The dispatch record names one or more action skills and the subset of inputs each should receive. If the outcome is `no-match` or `failed`, the agent returns the record to the orchestrator unchanged.
|
||||
|
||||
### 4. Agent invokes each dispatched action skill
|
||||
Action skills live inside the layers — `/microsoft/skills/`, `/community/skills/`, `/custom/skills/` — so their authority is carried by their location. For a PR review, Entry typically dispatches `microsoft/skills/review/al-code-review.md`. The agent reads the file and executes it.
|
||||
|
||||
### 5. Action skill executes the four-step pattern
|
||||
|
||||
Each action skill is a markdown file that specifies what to do at each step. The template is always the same:
|
||||
|
||||
| Step | What happens |
|
||||
| --- | --- |
|
||||
| **Source** | Declare which knowledge folders and tags to search. |
|
||||
| **Relevance** | Filter by frontmatter — `bc-version`, `technologies`, `countries`, `application-area`. |
|
||||
| **Worklist** | Narrow from N candidates to the M that apply to this specific task. |
|
||||
| **Action** | Apply the relevant knowledge and produce structured output. |
|
||||
|
||||
Example: the Microsoft-owned performance review skill selects `performance` entries across every enabled layer, filters to `bc-version: 26` and `technologies: [al]`, narrows the candidate files to those that apply to the changed objects, and then evaluates each file against the diff. Its canonical corpus lives beside it under `/microsoft/knowledge/performance/`; cross-layer entries are limited to custom overrides or short-lived promotion work.
|
||||
|
||||
At this point the agent reads READ and DO on demand — it needs READ to interpret each knowledge file's frontmatter and sections, and DO to shape its output. Those contracts are fetched when first needed, not as part of bootstrap.
|
||||
|
||||
### 5a. The knowledge index (Source acceleration)
|
||||
|
||||
Discovering candidates at the Source step naively means opening every file under a domain folder just to read its frontmatter `keywords` — on a large corpus that is hundreds of file reads per review. To avoid this, BCQuality maintains a **knowledge index**: a single artifact (`knowledge-index.json`) that lists every article surviving the consumer's layer/allow-deny filtering and carries, per article, the exact inputs the Source/Worklist steps consume — `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint.
|
||||
|
||||
The index is **owned and produced by BCQuality**, not by each consumer: its generator (`tools/Build-KnowledgeIndex.ps1`) ships here, next to the skills and knowledge it derives from, so the index schema stays in lockstep with the Source contract and every consumer gets the same faithful index for free instead of re-implementing the parser. The consuming orchestrator does **not** build or invoke the index — it only prunes its clone to policy as it already does. The index is then (re)generated by BCQuality itself: **Entry's preparation step runs `Build-KnowledgeIndex.ps1` over the live, already-pruned clone** at the start of every run (see `skills/entry.md`), and BCQuality CI (`.github/workflows/knowledge-index.yml`) validates that the generator is healthy and deterministic. Building over the *pruned* clone — rather than shipping a committed full-corpus index that consumers trust — keeps the index exact for any consumer policy: it can never list an article the consumer denied, so policy-excluded rules cannot leak into discovery.
|
||||
|
||||
The index changes only *how candidates are discovered*, never *which are selected*. The Worklist predicate is unchanged — `keywords` still drive selection — and the agent still opens each worklisted article **in full** to read its `## Best Practice` / `## Anti Pattern` rule bodies; the index is discovery metadata only and never substitutes for the article body. When no index is present, skills fall back to path-based discovery (collect by domain folder), so review still works.
|
||||
|
||||
### 6. Agent emits structured output
|
||||
The output contract is defined in the DO meta-skill so that every action skill — today's and next year's — produces the same shape:
|
||||
|
||||
- **Outcome** — `completed`, `not-applicable`, `no-knowledge`, `partial`, or `failed`. An orchestrator can distinguish a clean run from a no-op from a failure without guessing.
|
||||
- **Findings** — what the skill observed (severity, message, optional location).
|
||||
- **Domain** — the producer-owned, human-readable display label on each review finding.
|
||||
- **References** — structured objects (`path` plus optional commit `sha`) pointing to the knowledge files that informed each finding.
|
||||
- **Confidence** — per-finding evidence strength.
|
||||
- **Suppressed** — knowledge files that were discarded by layer precedence or configuration, so reviewers can see what was overridden.
|
||||
|
||||
The orchestrator parses this **without skill-specific logic**. This is the point of the contract: orchestrators and action skills evolve independently.
|
||||
|
||||
### 7. Orchestrator integrates
|
||||
The orchestrator turns findings into PR comments, build gates, or IDE diagnostics, and links the references back to the knowledge files so the PR author — human or agent — can read the guidance.
|
||||
|
||||
## Knowledge-backed and agent findings
|
||||
|
||||
BCQuality is an **additive** knowledge layer. The agent surfaces two kinds of findings, both shaped to the same DO output contract:
|
||||
|
||||
- **Knowledge-backed findings** carry one or more entries in `references[]` pointing at BCQuality knowledge files. Their `id` is the primary file's repo-relative path. Leaf sub-skills set `domain` to their human-readable display label, and super-skills preserve it verbatim during rollup.
|
||||
- **Agent findings** carry an empty `references: []`, use a slug `id` prefixed `agent:`, and have `confidence` capped at `medium`. A leaf can emit one strictly within its own domain and uses that leaf's display label. A super-skill can emit a cross-cutting agent finding with `from-sub-skill: "agent"` and `domain: "Agent"`. Their `message` is self-contained because there is no knowledge-file footer to fall back on.
|
||||
|
||||
Before a skill emits an agent finding, it validates the candidate against the BCQuality knowledge already loaded for the task: a matching file upgrades the candidate to a knowledge-backed finding (and merges or deduplicates against relevant existing output); a contradicting file suppresses the candidate. Only candidates with no BCQuality coverage become agent findings.
|
||||
|
||||
Orchestrators MUST tolerate an absent `domain` in reports from older producers. When it is present, treat it as display text rather than an identifier: preserve the full string and its case, whitespace, punctuation, and non-ASCII characters, escaping only for the target rendering format. Do not tokenize it on spaces or use a lowercased or slugified form as the sole metadata or deduplication key, because distinct labels can collapse to the same slug. Retain the exact string, use a lossless encoding, or use a collision-resistant digest instead. Orchestrators MAY render knowledge-backed and agent findings differently and MAY apply independent severity floors; `references: []` and the `agent:` id prefix distinguish agent findings, while `from-sub-skill: "agent"` identifies those emitted by the super-skill itself.
|
||||
|
||||
## Why this architecture
|
||||
|
||||
- **Entry is the only hardcoded thing.** Orchestrators ship with one convention — *"invoke `/skills/entry.md` first"* — and nothing else. New action skills and new knowledge files are picked up automatically because Entry discovers them at dispatch time.
|
||||
- **Standalone installation adds an adapter, not another policy layer.** The
|
||||
plugin's host-format `al-code-review` skill only translates the invocation
|
||||
into Entry's task context. Entry and the dispatched action skills remain
|
||||
authoritative.
|
||||
- **Layers decide authority, not code.** The agent sees `/microsoft/` and `/community/` together; if two files conflict, the precedence rule defined in READ resolves it. A partner fork can disable `/community/` — that's a config choice, not a code change.
|
||||
- **Knowledge and skills evolve independently within their owning layer.** A new knowledge file requires no skill changes because existing skills pick it up via frontmatter filters. Layer placement still follows skill ownership, so promoting a skill also promotes its canonical corpus.
|
||||
|
||||
## The mental model, in one sentence
|
||||
|
||||
The orchestrator knows **when** to run; Entry decides **which skill** to run; the action skills define **what** to do; the meta-skills teach the agent **how** to behave; the knowledge files are **what** the agent knows.
|
||||
103
docs/standalone-runner.md
Normal file
103
docs/standalone-runner.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
# Build a lightweight standalone review runner
|
||||
|
||||
BCQuality provides review knowledge, routing, execution instructions, and
|
||||
structured output contracts. It intentionally does not choose models, schedule
|
||||
agents, retry failures, or collect usage telemetry. A standalone runner can add
|
||||
those host-specific capabilities without copying Business Central rules out of
|
||||
BCQuality.
|
||||
|
||||
Use the built-in standalone plugin when the host's default execution is
|
||||
sufficient. Build a runner when you need explicit control over cost, latency,
|
||||
concurrency, or integration with another review surface.
|
||||
|
||||
## Keep BCQuality current
|
||||
|
||||
Install or update the plugin with GitHub Copilot CLI:
|
||||
|
||||
```shell
|
||||
copilot plugin install microsoft/BCQuality
|
||||
copilot plugin update bcquality
|
||||
```
|
||||
|
||||
A runner that reads BCQuality from a checkout should pin a commit or release
|
||||
and upgrade it deliberately. Do not copy knowledge files or action-skill prose
|
||||
into the runner; doing so creates a second, drifting quality policy.
|
||||
|
||||
## Review a complete app folder
|
||||
|
||||
For a committed app, generated fixture, or source tree that has no meaningful
|
||||
diff, supply the app's root directory as `folder-path`. The review scope is
|
||||
every relevant file below that directory, including `app.json` and AL source.
|
||||
The folder does not need to be a Git repository.
|
||||
|
||||
With the standalone plugin installed, start a fresh Copilot session in the app
|
||||
folder and ask:
|
||||
|
||||
> Use the installed `al-code-review` skill to review the complete Business
|
||||
> Central app in this folder. Execute every dispatched review domain and return
|
||||
> the complete BCQuality findings report.
|
||||
|
||||
The adapter maps this request to `folder-path`; Entry routes it to the broad
|
||||
review super-skill. Because a folder is a current-state snapshot, the review
|
||||
must not invent a previous app version when evaluating comparison-only rules.
|
||||
|
||||
## Minimal runner flow
|
||||
|
||||
1. Give the agent the review input and a task context containing the user's
|
||||
actual goal, available input types, and any known BC applicability
|
||||
dimensions.
|
||||
2. Invoke `skills/entry.md`. Entry prepares the knowledge index and returns the
|
||||
action skills to run. Do not reproduce its routing logic.
|
||||
3. Execute every dispatched action skill with the exact input subset in its
|
||||
dispatch record. Read `skills/read.md` and `skills/do.md` on demand.
|
||||
4. When an action skill declares `sub-skills`, execute every relevant leaf as a
|
||||
discrete invocation. Leaves are independent and may be scheduled serially
|
||||
or concurrently.
|
||||
5. Collect each complete findings-report into `sub-results` in the declared
|
||||
`sub-skills` order, not completion order. Run the super-skill self-review
|
||||
only after all leaves have finished.
|
||||
6. Apply the DO composition, failure, deduplication, reference-integrity, and
|
||||
outcome rules. Return strict JSON before rendering it for people or another
|
||||
system.
|
||||
|
||||
The runner must never inspect the diff to skip a review domain. A leaf decides
|
||||
its own task-level applicability and reports `not-applicable` or
|
||||
`no-knowledge`.
|
||||
|
||||
## Runner-owned choices
|
||||
|
||||
Keep these settings and behaviors outside BCQuality:
|
||||
|
||||
- coordinator and leaf models;
|
||||
- serial or concurrent scheduling and maximum concurrency;
|
||||
- retries, timeouts, and rate-limit handling;
|
||||
- token, cost, duration, and actual-concurrency telemetry;
|
||||
- conversion of the findings report into Markdown, annotations, or PR
|
||||
comments.
|
||||
|
||||
Model selection and requested concurrency are deployment choices, not review
|
||||
rules. Evaluate them against representative applications before making them a
|
||||
default. Report actual usage and concurrency only when the host exposes native
|
||||
evidence; do not infer them from the requested profile.
|
||||
|
||||
## Failure and output checklist
|
||||
|
||||
A compatible runner:
|
||||
|
||||
- invokes every worklisted leaf exactly once unless a documented retry replaces
|
||||
a failed attempt;
|
||||
- keeps leaf contexts isolated and passes only the inputs they declare;
|
||||
- preserves every leaf report, including failed reports, in `sub-results`;
|
||||
- excludes unreliable findings from failed leaves and returns `partial` when
|
||||
only part of the review is reliable;
|
||||
- orders `sub-results` by the declared worklist and orders rendered findings
|
||||
deterministically;
|
||||
- calculates top-level severity counts from deduplicated top-level findings,
|
||||
not by summing leaf counts;
|
||||
- preserves knowledge paths verbatim and verifies references before publishing;
|
||||
- records the BCQuality commit or release used for the run.
|
||||
|
||||
BC-ALAgents, AL-Go, a Copilot custom agent, or a small host-native plugin can
|
||||
all implement this runner contract. They remain optional consumers:
|
||||
BCQuality's knowledge and skills stay independent of their orchestration
|
||||
choices.
|
||||
Loading…
Add table
Add a link
Reference in a new issue