diff --git a/README.md b/README.md index e780e31..0742a41 100644 --- a/README.md +++ b/README.md @@ -30,9 +30,9 @@ All three layers are enabled by default when an agent consumes BCQuality. Conten Skills define how agents consume knowledge. They come in two flavors: - **Meta-skills** (`/skills/`) — the three globally shared skills that bootstrap every interaction with BCQuality: - 1. **Schema + Use** (READ) — how to read a knowledge file: interpret frontmatter, parse sections, understand layer precedence. This is the consumer's reference — any agent or skill that reads knowledge files depends on it. - 2. **Action Skill** (DO) — the template every action skill follows. Defines the four-step pattern (Source → Relevance → Worklist → Action) and the structured output format that orchestrators expect. This is the skill author's reference. - 3. **New Knowledge** (WRITE) — how to author a valid knowledge file. References Schema + Use for the format specification and adds authoring rules (atomicity, section guidance). This is the contributor's reference. + 1. **Schema + Use** (READ, [`skills/read.md`](skills/read.md)) — how to read a knowledge file: interpret frontmatter, parse sections, understand layer precedence. This is the consumer's reference — any agent or skill that reads knowledge files depends on it. + 2. **Action Skill** (DO, [`skills/do.md`](skills/do.md)) — the template every action skill follows. Defines the four-step pattern (Source → Relevance → Worklist → Action) and the structured output format that orchestrators expect. This is the skill author's reference. + 3. **New Knowledge** (WRITE, [`skills/write.md`](skills/write.md)) — how to author a valid knowledge file. References Schema + Use for the format specification and adds authoring rules (atomicity, section guidance). This is the contributor's reference. Schema + Use and New Knowledge are deliberately separate: one is the reader's contract, the other is the writer's guide. New Knowledge depends on Schema + Use but does not duplicate it. @@ -94,7 +94,7 @@ Action skills follow a four-step pattern: 3. **Worklist** — narrow from N candidates to the M that apply to the current task 4. **Action** — apply the relevant knowledge and produce structured output -Every action skill produces output in a common format that orchestrators can consume without skill-specific parsing. The format includes findings (what the skill observed), references (which knowledge files informed each finding), and confidence signals. This contract is defined in the Action Skill meta-skill so that orchestrators and action skills remain independently evolvable. +Every action skill produces output in a common format that orchestrators can consume without skill-specific parsing. The format is JSON and includes an `outcome` (so a clean run, a not-applicable skill, and a partial failure are all distinguishable), `findings` (what the skill observed), structured `references` back to the knowledge files that informed each finding, per-finding `confidence`, and a `suppressed` list recording any knowledge files overridden by layer precedence. This contract is defined in the Action Skill meta-skill so that orchestrators and action skills remain independently evolvable. The meta-skills in `/skills/` define this pattern. Every concrete action skill follows it. diff --git a/agent-consumption.md b/agent-consumption.md index caf0098..e62f2ec 100644 --- a/agent-consumption.md +++ b/agent-consumption.md @@ -55,9 +55,11 @@ Example: a performance review skill sources from `/microsoft/knowledge/performan ### 5. Agent emits structured output The output contract is defined in the DO meta-skill so that every action skill — today's and next year's — produces the same shape: -- **Findings** — what the skill observed (severity, message, location). -- **References** — which knowledge files informed each finding. -- **Confidence** — how sure the skill is. +- **Outcome** — `completed`, `not-applicable`, `no-knowledge`, `partial`, or `failed`. An orchestrator can distinguish a clean run from a no-op from a failure without guessing. +- **Findings** — what the skill observed (severity, message, optional location). +- **References** — structured objects (`path` plus optional commit `sha`) pointing to the knowledge files that informed each finding. +- **Confidence** — per-finding evidence strength. +- **Suppressed** — knowledge files that were discarded by layer precedence or configuration, so reviewers can see what was overridden. The orchestrator parses this **without skill-specific logic**. This is the point of the contract: orchestrators and action skills evolve independently. diff --git a/skills/do.md b/skills/do.md new file mode 100644 index 0000000..6434b68 --- /dev/null +++ b/skills/do.md @@ -0,0 +1,191 @@ +--- +kind: meta-skill +id: do +version: 1 +title: Action Skill — the template every action skill follows +--- + +# DO + +An action skill is a markdown file that tells an agent how to do one concrete job — review a pull request, audit telemetry usage, generate a skeleton — using knowledge files from BCQuality. This document is the template every action skill follows. Orchestrators rely on the template to consume any skill without skill-specific parsing. + +This contract is stable. Changes require a PR approved by both maintainers. + +## What an action skill is + +An action skill is a single markdown file with YAML frontmatter. It lives inside a layer: + +- `/microsoft/skills/` — platform-endorsed action skills. +- `/community/skills/` — community-contributed action skills. +- `/custom/skills/` — partner or customer action skills (typically in a consumer repo, not in BCQuality itself). + +Action skills do not live at the repo root. The three meta-skills in `/skills/` are the only files that sit outside a layer. + +## Frontmatter schema + +```yaml +--- +kind: action-skill +id: al-code-review +version: 1 +title: AL code review +description: Reviews AL source changes against performance and security guidance. +inputs: [pr-diff, object-list] +outputs: [findings-report] +bc-version: [26..28] +technologies: [al] +countries: [w1] +application-area: [all] +--- +``` + +`kind`, `id`, `version`, `title`, `description`, `inputs`, `outputs` are required and specific to action skills. + +`bc-version`, `technologies`, `countries`, `application-area` are optional filters that let an orchestrator pre-select applicable skills for a task. They follow the same semantics as in READ. + +`inputs` is a list of abstract input types the skill consumes. Standard values: `pr-diff`, `object-list`, `file-path`, `repository`, `telemetry-query`. `outputs` is always a single-element list naming the output kind; today only `findings-report` is defined. + +## Required sections + +Every action skill MUST contain these five sections, in order: + +- `## Source` — declares which folders and tags to search for knowledge. +- `## Relevance` — declares how to filter the candidates. +- `## Worklist` — declares how to narrow filtered candidates to the set that applies to this task. +- `## Action` — declares what the skill does with the narrowed set. +- `## Output` — declares the shape of the produced output; typically a reference to the contract below. + +## The four-step pattern + +**Source.** List the folders and tag filters to collect candidates from. Sources span layers: an action skill sources from the same `domain` subfolder across every enabled layer. Example: *"Source from `/*/knowledge/performance/` and `/*/knowledge/security/`."* + +**Relevance.** Apply frontmatter filters to the candidates. Typical filters: match `bc-version` against the target environment, match `technologies` against the languages in scope, match `countries` and `application-area` against the consuming codebase's context. The exact matching rules are defined in READ (*Frontmatter matching semantics*). Files that do not match are discarded. + +**Worklist.** Narrow the relevant candidates to the subset that applies to the current task. This is where the task-specific signal enters: the objects changed in the PR, the queries being audited, the skeleton being generated. Typical moves: match `keywords` against task vocabulary, match file topics against changed objects, deduplicate by concern. + +**Action.** Execute the skill's work against the worklist. Evaluate each item in the worklist against the task input and emit findings. The action step is where skill behavior differs; the preceding three steps are uniform. + +## Output contract + +Every action skill emits a single JSON document that conforms to this schema: + +```json +{ + "skill": { "id": "string", "version": 1 }, + "outcome": "completed | not-applicable | no-knowledge | partial | failed", + "outcome-reason": "string", + "summary": { + "counts": { "blocker": 0, "major": 0, "minor": 0, "info": 0 }, + "coverage": { "worklist-size": 0, "items-evaluated": 0 } + }, + "findings": [ + { + "id": "string", + "severity": "blocker | major | minor | info", + "message": "string", + "location": { + "file": "string", + "line": 0, + "range": { "start-line": 0, "end-line": 0 } + }, + "references": [ + { "path": "string", "sha": "string" } + ], + "confidence": "high | medium | low" + } + ], + "suppressed": [ + { + "reference": { "path": "string", "sha": "string" }, + "reason": "layer-precedence | configuration" + } + ] +} +``` + +### Field semantics + +**`outcome`** (required) — + +- `completed` — the skill ran end-to-end; `findings` reflects the full result (including the empty set). +- `not-applicable` — the skill's frontmatter filters did not match the task context; the skill declined to run. +- `no-knowledge` — the skill ran but found no applicable knowledge files; `findings` MUST be empty. +- `partial` — the skill evaluated part of its worklist but did not finish. `summary.coverage` reflects the evaluated subset. Set `outcome-reason` to explain. +- `failed` — the skill encountered an error and produced no reliable findings. Set `outcome-reason`. Consumers SHOULD ignore `findings` on a failed outcome. + +`outcome-reason` is optional for `completed`, `not-applicable`, and `no-knowledge`; required for `partial` and `failed`. + +An empty `findings` array with `outcome: completed` means the skill ran and found nothing to flag. Orchestrators MUST NOT conflate this with `not-applicable` or `no-knowledge`. + +**`findings[].id`** — a stable identifier for the rule or concern that produced the finding. For citation-based findings (any finding with a non-empty `references`), `id` MUST equal `references[0].path` — the primary knowledge file's repo-relative path. For skills that detect concerns without a direct citation, `id` is a skill-defined slug (kebab-case, stable across versions of the skill). The same `id` produced in two runs MUST refer to the same concern; consumers MAY deduplicate findings by `id`. + +**`findings[].severity`** — see the taxonomy below. + +**`findings[].message`** — human-readable explanation of the finding. Single short paragraph. No markdown formatting assumptions. + +**`findings[].location`** — optional. When present: + +- `file` MUST be a repo-relative path using forward slashes. +- `line` is the primary line number, 1-based. +- `range` is optional and describes a contiguous line span; `start-line` and `end-line` are 1-based and inclusive. `start-line` MUST equal `line` when both are present. + +Findings without a `location` are permitted (for example, repository-wide observations). + +**`findings[].references`** — array of knowledge-file references. Each reference is an object: + +- `path` (required) — repo-relative path to the knowledge file, forward slashes. +- `sha` (optional) — commit SHA the skill read when producing the finding. Consumers SHOULD include `sha` when the skill was invoked with a specific repo state. + +The first reference is the **primary** reference: the knowledge file the finding most directly cites. Additional references provide supporting context and are not ranked. `references` MAY be empty for findings the skill generates without a knowledge-file citation. + +**`findings[].confidence`** — the skill's confidence that the finding is a true positive, given the evidence it evaluated. Not applicability confidence, not severity confidence. Values: `high`, `medium`, `low`. + +**`suppressed`** — MUST list every knowledge file that was discarded due to layer precedence or consumer configuration, whenever that file would otherwise have contributed to the worklist. Each entry contains: + +- `reference` — the suppressed file (same object shape as `findings[].references`). +- `reason` — `layer-precedence` when another layer won under READ's precedence rules; `configuration` when the consumer disabled the file's layer. + +Severity taxonomy: + +- `blocker` — violates platform-level guarantees; the work cannot proceed as-is. +- `major` — significant defect; should be fixed before merge. +- `minor` — quality concern; worth flagging but not a gate. +- `info` — observation or context; not actionable on its own. + +## Worked example + +A minimal action skill that cites applicable guidance for a changed AL file, without generating findings of its own: + +```yaml +--- +kind: action-skill +id: cite-applicable-guidance +version: 1 +title: Cite applicable guidance +description: Lists knowledge files relevant to a changed AL file. +inputs: [file-path] +outputs: [findings-report] +technologies: [al] +--- +``` + +```markdown +## Source +All files under `/*/knowledge/` across enabled layers. + +## Relevance +Filter by `technologies: [al]` and `bc-version` matching the target environment. + +## Worklist +Intersect `keywords` with tokens derived from the target file's object name and changed members. + +## Action +For each worklist entry, emit one finding with severity `info`, a message naming the concern, and a reference object pointing to the knowledge file. + +## Output +Conforms to the DO output contract. +``` + +## How orchestrators consume output + +An orchestrator invokes an action skill with an input appropriate to the skill's declared `inputs`, receives the JSON output, and maps findings to its delivery surface (PR comments, build gates, IDE diagnostics). The orchestrator MUST NOT interpret skill-specific fields beyond the schema above. Skills that need richer semantics MUST encode them within the schema (for example, by adding structured `message` text) rather than extending the output shape. diff --git a/skills/read.md b/skills/read.md new file mode 100644 index 0000000..1d25ac1 --- /dev/null +++ b/skills/read.md @@ -0,0 +1,127 @@ +--- +kind: meta-skill +id: read +version: 1 +title: Schema + Use — how to read a knowledge file +--- + +# READ + +Every consumer of BCQuality — an agent, an action skill, a human reviewer — reads this file first. It defines what a knowledge file is, what fields it contains, what they mean, and how to reconcile multiple files. + +This contract is stable. Changes require a PR approved by both maintainers. + +## What a knowledge file is + +A knowledge file is a single markdown file that covers **one concern** in Business Central development. It has: + +- A YAML frontmatter block with the fields below. All fields are required. +- A `## Description` section. Required. +- Optional sections — typically `## Best Practice` and `## Anti Pattern`, but any `##` section is permitted. +- No fenced code blocks. Sample code lives in separate sample files referenced by path. + +A file that violates any of these rules is invalid and MUST be skipped by consumers. Do not attempt to partially parse invalid files. + +## Frontmatter schema (v1) + +```yaml +--- +bc-version: [26, 27, 28] # or the range shorthand [26..28] +domain: performance +keywords: [query, filtering, partial] +technologies: [al] +countries: [w1] +application-area: [all] +--- +``` + +All six fields are required. Missing or empty fields invalidate the file. + +### Fields + +**`bc-version`** — Array. The Business Central major versions this file applies to. Two forms are accepted: + +- Explicit list: `[26, 27, 28]`. +- Range shorthand: `[26..28]` means every integer from 26 through 28 inclusive. + +Consumers MUST expand ranges to the full set before comparison. + +**`domain`** — String. A single domain tag that places the file within a broader area of concern. Standard values include `performance`, `security`, `ux`, `telemetry`, `testing`, `api`, `pipelines`, `finance`, `supply-chain`, `manufacturing`, `jobs`. New domains may be introduced by contributors; no closed enumeration is enforced at the schema level. Consumers MUST treat unknown domains as valid. + +**`keywords`** — Array of strings. Free-text tags used for retrieval. Between 3 and 10 tags is typical. Tags are lowercase, kebab-case, and describe the concern in the vocabulary an engineer or agent would search for. + +**`technologies`** — Array of strings. The technologies the file applies to. Examples: `al`, `javascript`, `powershell`, `kql`, `azure-devops`, `github-actions`. A file that applies across technologies lists all of them explicitly. The sentinel `all` is not permitted for this field. + +**`countries`** — Array of strings. ISO 3166-1 alpha-2 country codes (lowercase: `us`, `de`, `dk`) for localization-specific guidance. Use the sentinel `[w1]` for guidance that applies worldwide. `[w1]` is mutually exclusive with country codes; do not combine. + +**`application-area`** — Array of strings. The BC application areas the file applies to. Examples: `finance`, `manufacturing`, `jobs`, `warehousing`, `service`. Use the sentinel `[all]` for guidance that applies regardless of application area. `[all]` is mutually exclusive with specific areas. + +## Sections + +**`## Description`** is required. It states the concern: what the topic is and why it matters. It is the primary retrieval target when a consumer decides whether a file is relevant. + +Two further sections are recognized as **normative** — consumers MAY rely on their content for conflict detection and guidance extraction: + +- **`## Best Practice`** — the recommended approach. +- **`## Anti Pattern`** — what to avoid and the reasoning. + +Any other `##` section is permitted and is **non-normative**: consumers MUST NOT treat its contents as binding guidance. Non-normative sections (for example `## See also` or `## Applies to`) are for human context; they are ignored by conflict detection and by the filtering rules below. Consumers MUST NOT fail on unknown sections. + +## Layer precedence + +A knowledge file lives in one of three layers, determined by its path: + +- `/microsoft/knowledge/**` — platform-endorsed. +- `/community/knowledge/**` — community-curated. +- `/custom/knowledge/**` — partner or customer overrides (typically in a consumer repo, not in BCQuality). + +The default consumption model is **additive**: an action skill sees files from every enabled layer and may surface findings from all of them. A consumer MAY be configured to disable a layer; in that case, files in the disabled layer are invisible to the consumer. + +When two files give **directly contradictory normative guidance**, the conflict is resolved by layer precedence: + +1. `/custom/` wins over `/microsoft/` and `/community/`. +2. `/microsoft/` wins over `/community/`. + +A conflict exists when both of the following are true: + +- **Applicability overlaps.** The files' frontmatter filters (`bc-version`, `technologies`, `countries`, `application-area`) have a non-empty intersection under the matching rules below. `domain` is a retrieval aid; it is not part of the applicability test. +- **Normative guidance contradicts.** Content in the `## Best Practice` or `## Anti Pattern` sections is logically incompatible (one recommends what the other forbids, or vice versa). Non-normative sections are not considered. + +Conflict detection is the consumer's responsibility; BCQuality does not enforce conflict-free content. When a consumer suppresses a losing file due to precedence or configuration, it **MUST** record the suppression in its output (see DO) so reviewers can see what was overridden. + +## Frontmatter matching semantics + +When a consumer filters or matches files against a task context, these rules apply: + +- **`bc-version`** — the target BC version MUST be an element of the file's expanded `bc-version` set. Range shorthand (`[26..28]`) MUST be expanded before comparison. +- **`technologies`** — non-empty intersection between the task's technologies and the file's technologies. There is no sentinel for this field. +- **`countries`** — the file matches if its set contains `w1`, or if there is a non-empty intersection with the task's countries. +- **`application-area`** — the file matches if its set contains `all`, or if there is a non-empty intersection with the task's application areas. + +A file is **applicable** to a task when all four rules match. Applicability is also the basis for conflict detection above. + +### When the task context is partial + +A task context may omit one or more dimensions (for example, a skill invoked against a raw file path with no known target BC version). For any omitted dimension: + +- If the file's value for that dimension is a universal sentinel (`w1` for countries, `all` for application-area), the rule matches. +- Otherwise the rule is treated as **unknown**, not as a match and not as a failure. + +A file with any `unknown` rule is **conditionally applicable**. A consumer MAY include conditionally applicable files in the worklist; if it does, every finding derived from such a file MUST have `confidence` no higher than `medium` and MUST record the unknown dimensions in the finding's `message`. A consumer MAY be configured to exclude conditionally applicable files entirely. + +Consumers MUST NOT silently treat missing context as a match. + +## Citing a knowledge file + +A consumer that produces output referencing a knowledge file MUST cite it by its repo-relative path (for example, `microsoft/knowledge/performance/filter-before-find.md`). Line numbers are not stable references; use the file path only. If a commit SHA is available to the consumer, it SHOULD be included alongside the path. + +## Retrieval workflow + +The standard workflow for finding applicable files: + +1. Collect candidates by path (typically by `domain` subfolder, across enabled layers). +2. Filter by frontmatter using the matching rules above. Files that are not applicable are discarded. +3. Rank or narrow by `keywords` relevance to the task. +4. Resolve conflicts via layer precedence. + +Steps 1–3 are deterministic; step 4 is applied only when conflicts are detected. diff --git a/skills/write.md b/skills/write.md new file mode 100644 index 0000000..998e286 --- /dev/null +++ b/skills/write.md @@ -0,0 +1,80 @@ +--- +kind: meta-skill +id: write +version: 1 +title: New Knowledge — how to author a knowledge file +--- + +# WRITE + +Anyone — human or agent — adding a knowledge file to BCQuality follows this guide. READ is the format specification; WRITE is the authoring guide. This file does not restate the schema; consult READ for field-by-field semantics. + +## Before you start + +Read `skills/read.md` first. A file that does not conform to READ will be rejected. WRITE assumes READ is already understood. + +## The atomicity rule + +One knowledge file covers **one concern**. If two ideas would share a file, split them into two files and cross-reference from the Description. A good test: could an action skill reasonably want to cite one without the other? If yes, they are two concerns. + +Symptoms that a file is trying to be two: + +- Two Best Practice sections. +- A Description that uses "and" to join two topics. +- `keywords` that span two unrelated vocabularies. + +## Size + +Target under 100 lines. Ideal under 50. Long files almost always mean two concerns; the fix is to split, not to compress. + +## Sections + +**Description** is the only required section and the one that matters most. It states the concern and why it matters in two to five sentences. It is the primary retrieval target — write it as if a skill is deciding whether to load this file based on this text alone. + +**Best Practice** is optional but recommended when the concern has a clear preferred approach. State the approach and the reasoning. Keep it to the *what* and *why*; the *how* (code) belongs in a sample file. + +**Anti Pattern** is optional but recommended when the concern has a common wrong approach. State the pattern, the consequence, and the signal a reviewer or agent can use to detect it. Anti Pattern sections are highly actionable for review skills; write them with detection in mind. + +Custom `##` sections are permitted when they serve the concern (for example, `## Applies to` for scope caveats or `## See also` for related files). Consumers are not required to understand them, so do not put load-bearing content there. + +## No fenced code blocks + +Knowledge files do not contain code. Samples live in separate files under `/samples/` (or the layer's sample folder) and are referenced by path. This keeps knowledge files retrieval-friendly and prevents code from drifting out of sync with BC platform changes buried inside prose. + +## Choosing frontmatter values + +**`bc-version`.** Claim only the versions you have evidence for. If the guidance is known to apply from BC 24 onward and you have tested against 26–28, write `[26..28]`, not `[24..28]`. Under-claim; a future contributor can widen the range. + +**`domain`.** Pick one. If two fit, the file is probably two concerns. If no existing domain fits, introduce a new one — domains are open. Prefer existing domains when they are a reasonable fit, to keep retrieval predictable. + +**`keywords`.** Three to ten, lowercase, kebab-case. Write them as the search terms an engineer or agent would use to find this concern. Include synonyms when a concept has multiple common names (for example, `find-set` and `findset`). Do not duplicate the domain or title as a keyword. + +**`technologies`.** List every technology the guidance touches, explicitly. Do not use a sentinel; there is no `[all]` for technologies. If the guidance is truly technology-agnostic, the file probably belongs in a different repository. + +**`countries`.** Default to `[w1]` (worldwide). Use ISO country codes only when the guidance is specific to a localization — tax regulations, electronic invoicing formats, country-specific VAT rules. `[w1]` is mutually exclusive with country codes. + +**`application-area`.** Default to `[all]`. Restrict only when the guidance is specific to a BC application area — posting routines in Finance, lot tracking in Warehouse Management. `[all]` is mutually exclusive with specific areas. + +## File naming + +`kebab-case.md`. The name should echo the concern: `filter-before-find.md`, `avoid-implicit-commit.md`, `telemetry-for-failed-posts.md`. Avoid generic names (`performance.md`) and version numbers in the name. + +## Choosing a layer + +- **`/microsoft/knowledge//`** — platform-endorsed guidance. Authored or approved by the BC platform team. Use this layer only when the guidance reflects a platform guarantee or official recommendation. +- **`/community/knowledge//`** — shared community patterns. The default layer for contributions from outside the platform team. Content here can be promoted to `/microsoft/` once it proves itself. +- **`/custom/knowledge//`** — partner or customer overrides. Generally does not appear in the BCQuality repository itself; `/custom/` lives in consumer repositories. + +## Pre-PR checklist + +Before opening a pull request: + +- Frontmatter has all six required fields with valid values (see READ). +- The file has a `## Description` section. +- No fenced code blocks. +- File is under 100 lines. +- File covers one concern. +- File is in the correct layer and domain folder. +- Name is kebab-case and descriptive. + +Agents scaffolding new files SHOULD run this checklist programmatically before emitting the file.