mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-08-06 09:26:52 +01:00
Emit human-readable domain label on review findings (#54)
* Emit human-readable domain label on review findings Add an optional findings[].domain field to the DO review output contract so each finding carries its own human-readable review-domain display label. Leaf review skills set it on every finding they emit; the al-code-review super-skill copies it verbatim during rollup and sets it to "Agent" for its own cross-cutting agent findings. This decouples consumers from BCQuality's domain taxonomy: they render finding.domain verbatim instead of maintaining a sub-skill-id -> label map. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Define domain display-label constraints Clarify that review domains may contain internal whitespace, punctuation, case-sensitive text, and non-ASCII characters. Require consumers to preserve and safely encode the complete label instead of relying on lossy slugs, matching the replacement BC-ALAgents consumer. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 77d0a40e-8bf5-40ac-a450-40eb0255db03 --------- Co-authored-by: Jesper Schulz-Wedde <jesper.schulzwedde@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Jesper Schulz-Wedde
GitHub
parent
809af9708e
commit
ae04938c03
19 changed files with 91 additions and 56 deletions
|
|
@ -82,7 +82,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"AppSource"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -105,6 +105,7 @@ Output conforms to the DO output contract. A populated example:
|
|||
{ "path": "microsoft/knowledge/appsource/object-affixes-prevent-collisions.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"domain": "AppSource",
|
||||
"suggested-code": "field(50100; \"Loyalty Points ABC\"; Integer)"
|
||||
}
|
||||
],
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Breaking Changes"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -100,7 +100,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/breaking-changes/do-not-change-published-procedure-signatures.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Breaking Changes"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/breaking-changes/choose-access-modifiers-deliberately.md",
|
||||
|
|
@ -113,7 +114,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/breaking-changes/choose-access-modifiers-deliberately.md" }
|
||||
],
|
||||
"confidence": "medium"
|
||||
"confidence": "medium",
|
||||
"domain": "Breaking Changes"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -76,7 +76,7 @@ For each sub-skill in the worklist, executed one at a time per the discipline ab
|
|||
1. Invoke the sub-skill with the orchestrator's inputs, passing only the subset each sub-skill declares in its `inputs`.
|
||||
2. Capture the sub-skill's complete findings-report verbatim and append it to `sub-results`.
|
||||
3. If the sub-skill's `outcome` is `failed`, stop here for this sub-skill: its findings are not reliable per the DO contract and MUST NOT be copied into the super-skill's top-level `findings[]` or counted in `summary.counts`.
|
||||
4. Otherwise, append each entry from the sub-skill's `findings[]` to the super-skill's top-level `findings[]`, setting `from-sub-skill` to the sub-skill's `skill.id`. For non-citation findings (those whose `id` is a skill-defined slug rather than a reference path), prefix `id` with `<from-sub-skill>:` to prevent collisions across sub-skills. Other finding fields are preserved.
|
||||
4. Otherwise, append each entry from the sub-skill's `findings[]` to the super-skill's top-level `findings[]`, setting `from-sub-skill` to the sub-skill's `skill.id` and preserving each finding's optional `domain` field verbatim, including its absence. For non-citation findings (those whose `id` is a skill-defined slug rather than a reference path), prefix `id` with `<from-sub-skill>:` to prevent collisions across sub-skills. Other finding fields are preserved.
|
||||
|
||||
### Agent self-review pass
|
||||
|
||||
|
|
@ -89,11 +89,12 @@ Frame the pass by cross-cutting concerns — architecture, error handling, resou
|
|||
For every candidate the agent identifies in this pass:
|
||||
|
||||
1. **Validate against BCQuality knowledge.** Check the candidate against the knowledge files the sub-skills have already loaded for this task (visible via their `references` and `suppressed` lists in `sub-results`).
|
||||
- If a BCQuality knowledge file matches the candidate, upgrade it to a knowledge-backed finding: cite the file in `references`, set `id` to the file's path, set `from-sub-skill` to the sub-skill that owns that knowledge domain, and merge with or deduplicate against any sub-skill finding that already covers the same concern at the same location.
|
||||
- If a BCQuality knowledge file matches the candidate, upgrade it to a knowledge-backed finding: cite the file in `references`, set `id` to the file's path, set `from-sub-skill` to the sub-skill that owns that knowledge domain, set `domain` to the human-readable label required by that sub-skill's Output contract, and merge with or deduplicate against any sub-skill finding that already covers the same concern at the same location.
|
||||
- If a BCQuality knowledge file **explicitly contradicts** the candidate (its `## Best Practice` or `## Anti Pattern` says the opposite of what the agent flagged), suppress the candidate and do not surface it.
|
||||
- Otherwise the candidate has no BCQuality coverage; emit it as a super-skill agent finding.
|
||||
2. **Emit agent finding.** Per DO's *Agent findings* rules:
|
||||
- `from-sub-skill: "agent"` (the super-skill itself produced it)
|
||||
- `domain: "Agent"` (the display label for super-skill cross-cutting findings)
|
||||
- `references: []`
|
||||
- `id` is a skill-defined slug prefixed with `agent:` (for example, `agent:missing-error-handling-on-http-call`).
|
||||
- `confidence` capped at `medium`.
|
||||
|
|
@ -143,7 +144,8 @@ Output conforms to the DO output contract, extended with `sub-results` and `skip
|
|||
{ "path": "microsoft/knowledge/performance/apply-filters-before-iterating.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"from-sub-skill": "al-performance-review"
|
||||
"from-sub-skill": "al-performance-review",
|
||||
"domain": "Performance"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/performance/use-setloadfields-for-partial-records.md",
|
||||
|
|
@ -157,7 +159,8 @@ Output conforms to the DO output contract, extended with `sub-results` and `skip
|
|||
{ "path": "microsoft/knowledge/performance/use-setloadfields-for-partial-records.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"from-sub-skill": "al-performance-review"
|
||||
"from-sub-skill": "al-performance-review",
|
||||
"domain": "Performance"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/security/secrettext-for-credentials.md",
|
||||
|
|
@ -172,7 +175,8 @@ Output conforms to the DO output contract, extended with `sub-results` and `skip
|
|||
{ "path": "microsoft/knowledge/security/secrettext-for-credentials.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"from-sub-skill": "al-security-review"
|
||||
"from-sub-skill": "al-security-review",
|
||||
"domain": "Security"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/security/secrets-isolated-storage.md",
|
||||
|
|
@ -186,7 +190,8 @@ Output conforms to the DO output contract, extended with `sub-results` and `skip
|
|||
{ "path": "microsoft/knowledge/security/secrets-isolated-storage.md" }
|
||||
],
|
||||
"confidence": "medium",
|
||||
"from-sub-skill": "al-security-review"
|
||||
"from-sub-skill": "al-security-review",
|
||||
"domain": "Security"
|
||||
},
|
||||
{
|
||||
"id": "agent:missing-error-handling-on-http-client",
|
||||
|
|
@ -199,7 +204,8 @@ Output conforms to the DO output contract, extended with `sub-results` and `skip
|
|||
},
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"from-sub-skill": "agent"
|
||||
"from-sub-skill": "agent",
|
||||
"domain": "Agent"
|
||||
}
|
||||
],
|
||||
"suppressed": [],
|
||||
|
|
@ -224,7 +230,8 @@ Output conforms to the DO output contract, extended with `sub-results` and `skip
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/performance/apply-filters-before-iterating.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Performance"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/performance/use-setloadfields-for-partial-records.md",
|
||||
|
|
@ -237,7 +244,8 @@ Output conforms to the DO output contract, extended with `sub-results` and `skip
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/performance/use-setloadfields-for-partial-records.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Performance"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
@ -262,7 +270,8 @@ Output conforms to the DO output contract, extended with `sub-results` and `skip
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/security/secrettext-for-credentials.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Security"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/security/secrets-isolated-storage.md",
|
||||
|
|
@ -275,7 +284,8 @@ Output conforms to the DO output contract, extended with `sub-results` and `skip
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/security/secrets-isolated-storage.md" }
|
||||
],
|
||||
"confidence": "medium"
|
||||
"confidence": "medium",
|
||||
"domain": "Security"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -85,7 +85,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Data Modeling"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -108,6 +108,7 @@ Output conforms to the DO output contract. A populated example:
|
|||
{ "path": "microsoft/knowledge/data-modeling/set-last-date-modified-in-onmodify-and-onrename.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"domain": "Data Modeling",
|
||||
"suggested-code": "trigger OnRename()\nbegin\n \"Last Date Modified\" := Today();\nend;"
|
||||
}
|
||||
],
|
||||
|
|
|
|||
|
|
@ -78,7 +78,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Error Handling"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -101,7 +101,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/error-handling/prefer-errorinfo-for-actionable-errors.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Error Handling"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/error-handling/errortype-internal-vs-client-for-diagnostics.md",
|
||||
|
|
@ -114,7 +115,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/error-handling/errortype-internal-vs-client-for-diagnostics.md" }
|
||||
],
|
||||
"confidence": "medium"
|
||||
"confidence": "medium",
|
||||
"domain": "Error Handling"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -96,7 +96,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Events"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -119,7 +119,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/events/publish-thin-onbefore-onafter-integration-events.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Events"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/events/use-ishandled-to-make-base-behaviour-overridable.md",
|
||||
|
|
@ -132,7 +133,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/events/use-ishandled-to-make-base-behaviour-overridable.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Events"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -85,7 +85,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Interfaces"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -108,7 +108,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/interfaces/prefer-interface-over-case-branching.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Interfaces"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/interfaces/set-defaultimplementation-on-enum.md",
|
||||
|
|
@ -121,7 +122,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/interfaces/set-defaultimplementation-on-enum.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Interfaces"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -85,7 +85,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Performance"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -108,7 +108,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/performance/apply-filters-before-iterating.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Performance"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/performance/use-setloadfields-for-partial-records.md",
|
||||
|
|
@ -121,7 +122,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/performance/use-setloadfields-for-partial-records.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Performance"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -79,7 +79,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Privacy"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -102,7 +102,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/privacy/data-classification-required-on-pii-fields.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Privacy"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Security"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -100,7 +100,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/security/secrettext-for-credentials.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Security"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/security/secrets-isolated-storage.md",
|
||||
|
|
@ -113,7 +114,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/security/secrets-isolated-storage.md" }
|
||||
],
|
||||
"confidence": "medium"
|
||||
"confidence": "medium",
|
||||
"domain": "Security"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Style"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -99,7 +99,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/style/label-suffix-approved-list.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Style"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. The empty-corpus case produces:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Telemetry"`. The empty-corpus case produces:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
|
|||
|
|
@ -84,7 +84,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Testing"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -107,6 +107,7 @@ Output conforms to the DO output contract. A populated example:
|
|||
{ "path": "microsoft/knowledge/testing/asserterror-needs-expectederror-and-code.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"domain": "Testing",
|
||||
"suggested-code": "asserterror PostInvalidOrder();\nAssert.ExpectedError(ExpectedPostingErr);"
|
||||
}
|
||||
],
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Accessibility"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -99,7 +99,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/ui/show-caption-on-editable-fields.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Accessibility"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Upgrade"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -102,7 +102,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/upgrade/enum-values-additive-at-end.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Upgrade"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ Outcome selection:
|
|||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
Output conforms to the DO output contract. Every finding this skill emits MUST set `findings[].domain` to `"Web Services"`. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -103,7 +103,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/web-services/set-required-api-page-properties.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Web Services"
|
||||
},
|
||||
{
|
||||
"id": "microsoft/knowledge/web-services/expose-systemid-as-the-api-key.md",
|
||||
|
|
@ -116,7 +117,8 @@ Output conforms to the DO output contract. A populated example:
|
|||
"references": [
|
||||
{ "path": "microsoft/knowledge/web-services/expose-systemid-as-the-api-key.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
"confidence": "high",
|
||||
"domain": "Web Services"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue