mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-10-07 07:36:54 +01:00
Merge current main into development guidance
Reconcile the read-only guidance output with the machine-readable skill index, adopt linked sample references required by bounded retrieval, and update the guidance regression fixture for the retrieval helper dependency. Permit only the known endpoint-DLP metadata stream during read-only evidence capture. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 638b66d2-9f06-4f60-8781-808709e1485c
This commit is contained in:
commit
8f025ac679
127 changed files with 5251 additions and 136 deletions
77
skills/do.md
77
skills/do.md
|
|
@ -112,6 +112,12 @@ Every action skill MUST contain these five sections, in order:
|
|||
|
||||
An action skill with `outputs: [findings-report]` emits a single JSON document that conforms to this schema:
|
||||
|
||||
The machine-readable structural schema is
|
||||
[`schemas/findings-report.schema.json`](../schemas/findings-report.schema.json).
|
||||
The rules below remain authoritative for semantic checks that JSON Schema
|
||||
cannot perform by itself, including summary arithmetic, reference existence,
|
||||
source-scope locations, and article-body retrieval.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "string", "version": 1 },
|
||||
|
|
@ -165,6 +171,71 @@ The emitted document MUST be strict, valid JSON per [RFC 8259](https://www.rfc-e
|
|||
|
||||
AL source is the common failure case. Quoted identifiers (for example `Rec."No."`) and multi-line snippets routinely appear in `message`, `suggested-code`, and `suggested-code-omission-reason`, and each embedded quote or newline MUST be escaped when placed in a string value. A `suggested-code` payload that spans several lines is a single JSON string with `\n` separators, not a literal multi-line block. Emit the document as one JSON value with no trailing commentary, and do not rely on the consumer to repair unescaped output.
|
||||
|
||||
### Consumer acceptance gate
|
||||
|
||||
Capture the exact Task return as the immutable raw audit payload and primary
|
||||
transport. Preserve it unchanged in private run artifacts or host logs before
|
||||
creating any derived value. The accepted findings-report is either that exact
|
||||
return or the bounded normalized candidate described below; the raw audit
|
||||
payload never changes.
|
||||
|
||||
Before the full acceptance gate, a coordinator MAY create a normalized
|
||||
candidate copy only through this deterministic procedure:
|
||||
|
||||
1. Parse the exact return as strict JSON and provisionally check the complete
|
||||
report without mutating it. Every acceptance rule below MUST already pass
|
||||
except for one or more findings whose optional `location.range` has
|
||||
`start-line != line`.
|
||||
2. Each such finding is eligible only when `location.line`,
|
||||
`location.range.start-line`, and `location.range.end-line` are positive
|
||||
integers, `start-line <= line <= end-line`, and the finding does not contain
|
||||
the `suggested-code` field. Field presence disqualifies normalization even
|
||||
if its value is empty because suggested code may be bound to the reported
|
||||
range.
|
||||
3. Deep-copy the complete parsed report. In the candidate copy, remove only
|
||||
`location.range` from every eligible finding. Retain `location.line` and
|
||||
every other value unchanged. Do not add normalization metadata to the
|
||||
findings-report.
|
||||
4. Record each removed range separately in private run telemetry or artifacts,
|
||||
associated with the immutable raw audit payload. This record is
|
||||
runner-owned and is not part of the declared report schema.
|
||||
5. Validate the entire normalized candidate with the existing full consumer
|
||||
acceptance gate below. Only a candidate that passes every rule becomes the
|
||||
accepted copy used for rollup. If any other validation defect exists, or
|
||||
full validation fails, discard the candidate, preserve the raw payload, and
|
||||
fail the complete leaf as before.
|
||||
|
||||
This exception does not infer missing fields, alter references or paths, clamp
|
||||
line numbers, repair JSON, normalize a reversed or out-of-bounds range, remove
|
||||
a range from a finding containing `suggested-code`, or salvage arbitrary
|
||||
individual findings.
|
||||
|
||||
Before accepting either the exact return or an eligible normalized candidate
|
||||
as a findings-report, a coordinator or host MUST validate it deterministically:
|
||||
|
||||
1. Validate every required field, enum, type, conditional requirement, summary
|
||||
count, coverage value, and leaf/super-skill constraint against this output
|
||||
contract.
|
||||
2. For every knowledge-backed finding, verify each `references[].path` is an
|
||||
exact repo-relative knowledge path that exists in the live BCQuality
|
||||
snapshot, and verify `findings[].id` exactly equals
|
||||
`references[0].path`. Verify each path is also present in the coordinator's
|
||||
recorded set of complete article bodies retrieved for that leaf; catalog
|
||||
membership alone is insufficient. Keep optional `references[].sha`
|
||||
separate: it is commit provenance, not an article content hash.
|
||||
3. For every `location`, verify `file` is an exact source path in the supplied
|
||||
review scope, the file exists in that source snapshot, and `line` and any
|
||||
inclusive range identify existing lines with `start-line == line` and
|
||||
`end-line >= start-line`.
|
||||
|
||||
Validation failure invalidates the complete return; consumers MUST NOT salvage
|
||||
individual findings, infer missing fields, reconstruct JSON, clamp ranges,
|
||||
rewrite paths, or otherwise silently repair model output. Preserve the invalid
|
||||
raw payload unchanged. Record a separate failed validation result for that leaf
|
||||
with no findings, and derive the super-skill outcome as `partial` or `failed`
|
||||
using the normal rollup rules. Worker-side report-file persistence is optional
|
||||
and never replaces validation of the accepted exact or normalized copy.
|
||||
|
||||
### Field semantics
|
||||
|
||||
**`outcome`** (required) —
|
||||
|
|
@ -360,6 +431,12 @@ the declared worklist order, and wait for every invocation to finish before
|
|||
performing any super-skill self-review or final rollup. Scheduling MUST NOT
|
||||
change relevance, coverage, failure, reference-integrity, or output semantics.
|
||||
|
||||
Orchestrators SHOULD generate `skill-index.json` with
|
||||
`tools/Build-SkillIndex.ps1` and consume the super-skill's ordered `subSkills`
|
||||
from that index instead of parsing Markdown. Action-skill frontmatter remains
|
||||
the source of truth; the generated index conforms to
|
||||
`schemas/skill-index.schema.json`.
|
||||
|
||||
### Section interpretation for super-skills
|
||||
|
||||
The five required sections still apply. Their meaning shifts from knowledge files to sub-skills:
|
||||
|
|
|
|||
|
|
@ -149,3 +149,58 @@ The standard workflow for finding applicable files:
|
|||
4. Resolve conflicts via layer precedence.
|
||||
|
||||
Steps 1–3 are deterministic; step 4 is applied only when conflicts are detected.
|
||||
|
||||
### Bounded retrieval for review skills
|
||||
|
||||
Resolve `$root` to the BCQuality root, not the reviewed source. Entry prepares
|
||||
the index once before dispatch; that prepared index is the catalog snapshot and
|
||||
leaves use it read-only. Catalog retrieval validates the complete index metadata
|
||||
and returned paths without reopening or rehashing article bodies. Post-Entry
|
||||
body changes therefore take effect only after Entry rebuilds the index; exact
|
||||
body retrieval rejects a selected article whose content hash differs from its
|
||||
prepared row. In one PowerShell tool session, invoke the helpers with `&` so
|
||||
array arguments remain arrays:
|
||||
|
||||
```powershell
|
||||
& (Join-Path $root 'tools\Search-Knowledge.ps1') -Domain $domain -Technologies @('al')
|
||||
& (Join-Path $root 'tools\Get-KnowledgeArticles.ps1') -Paths @($exactPath)
|
||||
```
|
||||
|
||||
Pass enabled layers and only task dimensions that are actually known. Catalog
|
||||
retrieval returns every domain and READ-applicable row: it does not rank,
|
||||
sample, apply top-k, deduplicate by basename, or omit rows based on query text.
|
||||
Consume every page by passing `continuation.offset` as `-Offset` and
|
||||
`continuation.snapshot` as `-Snapshot` with the unchanged request until
|
||||
`complete` is `true`. Each page repeats request context, defaults, and totals.
|
||||
An omitted applicability field on a row inherits that page's `defaults`; it
|
||||
does not mean unknown task context. Preserve every row's exact `path`, `layer`,
|
||||
complete `keywords`, `title`, one-line `description`, non-default applicability
|
||||
fields, explicit `applicability`, and `unknownDimensions`.
|
||||
|
||||
Apply the leaf's existing Relevance and Worklist to the complete catalog union.
|
||||
Split the resulting exact paths into stable chunks of at most eight; never pass
|
||||
more paths than `-MaxArticles` (whose maximum is eight). Request article bodies
|
||||
only by one such chunk. Consume every
|
||||
returned `body`, then request `remainingPaths` with
|
||||
`continuation.snapshot` as `-Snapshot` until `complete` is `true`, preserving
|
||||
the other request settings. Continuation is confined to that chunk. Bodies are
|
||||
original strict UTF-8 text with source byte counts and SHA-256 content hashes;
|
||||
they are never summarized or truncated. Samples are not loaded unless
|
||||
requested explicitly with `-Samples` and exact sibling paths; their sibling
|
||||
article must match its prepared hash and contain the exact READ link.
|
||||
|
||||
The default serialized response limit is 16,000 bytes including its output
|
||||
newline. Never combine pages or bodies into an unbounded prompt. A malformed or
|
||||
internally inconsistent prepared index, changed continuation snapshot, selected
|
||||
article hash mismatch, invalid continuation, unsafe or missing path, invalid
|
||||
UTF-8, broken sample link, oversized path chunk, or row/envelope that cannot fit
|
||||
fails explicitly. Entry is the only index preparation point: a leaf does not
|
||||
rebuild. If PowerShell, a helper, or a valid prepared index is unavailable,
|
||||
discover exact paths across the enabled domain folders and use native bounded
|
||||
reads through EOF, validating frontmatter per READ and never treating retrieval
|
||||
failure as an empty result.
|
||||
|
||||
The helpers' `sha256` and `bytes` fields describe the retrieved file content.
|
||||
They are not citation provenance. Optional findings `references[].sha` is the
|
||||
BCQuality commit SHA the skill reviewed; omit it when that provenance is not
|
||||
available or would misrepresent uncommitted content.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue