mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-08-06 09:26:52 +01:00
Require suggested-code or omission reason for mechanical findings
BCQ now wins on coverage in the synthetic complex benchmark, but only 2 of 25 rendered comments carried GitHub suggestion blocks while the AIRHack reviewer emitted suggestions for every finding. Tighten the DO contract and leaf-skill instructions so suggested-code is no longer a soft affordance for mechanical fixes. Changes: - Add optional findings[].suggested-code-omission-reason to the DO schema. - Document suggested-code as expected for small, local, mechanical findings, with examples (delete unreachable code, Count() > 0 -> not IsEmpty(), move local Label, add missing ToolTip/OptionCaption, replace string-concatenated Error, change permission token, add an obvious else/guard branch). - Require omission reasons when a mechanical-looking finding omits suggested-code. - Update all AL leaf skills and al-code-review guidance with the same stronger contract. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
parent
b342775971
commit
d994f2ad0e
8 changed files with 38 additions and 10 deletions
|
|
@ -98,13 +98,13 @@ For every candidate the agent identifies in this pass:
|
|||
- `id` is a skill-defined slug prefixed with `agent:` (for example, `agent:missing-error-handling-on-http-call`).
|
||||
- `confidence` capped at `medium`.
|
||||
- `message` is non-empty and self-contained, describing both the issue and a concrete recommendation. A consumer rendering the finding has no knowledge-file footer to fall back on.
|
||||
- `suggested-code` SHOULD be set when the fix is small and mechanical. Omit it when the appropriate fix depends on context the agent cannot determine.
|
||||
- `suggested-code` MUST be set when the fix is small, local, and mechanical. If a mechanical-looking finding omits it, set `suggested-code-omission-reason` with the reason (for example, the fix spans non-contiguous code or requires choosing a real production value).
|
||||
|
||||
Leaf-level agent findings (those with `references: []` inside a sub-skill's report) are rolled up into the super-skill's top-level `findings[]` like any other sub-skill finding — they keep their `from-sub-skill: <leaf-id>` attribution and are not rewritten. They are not subject to the "MUST validate against knowledge" step above, because each leaf has already validated within its own domain.
|
||||
|
||||
### Suggested-code guidance
|
||||
|
||||
For both knowledge-backed findings rolled up from sub-skills and agent findings emitted in the self-review pass, populate `findings[].suggested-code` whenever a concrete code replacement is unambiguous from the diff context. The payload MUST be a literal replacement for the source lines covered by `location` (typically a single line, or the line range in `location.range`) — no diff markers, fences, or commentary. Examples of good candidates: deleting dead code after `exit`, replacing `Count() > 0` with `not IsEmpty()`, moving an inline `Label` declaration to a codeunit-level `var` block, fixing whitespace or keyword casing. Skip `suggested-code` when the fix requires choosing between multiple defensible alternatives or when the surrounding context the agent cannot see could change the answer.
|
||||
For both knowledge-backed findings rolled up from sub-skills and agent findings emitted in the self-review pass, populate `findings[].suggested-code` whenever a concrete code replacement is unambiguous from the diff context. This is a MUST for small, local, mechanical fixes. The payload MUST be a literal replacement for the source lines covered by `location` (typically a single line, or the line range in `location.range`) — no diff markers, fences, or commentary. Examples of good candidates: deleting dead code after `exit`, replacing `Count() > 0` with `not IsEmpty()`, moving an inline `Label` declaration to a codeunit-level `var` block, adding a missing property, replacing string-concatenated `Error`, changing an over-broad permission token, fixing whitespace or keyword casing. Skip `suggested-code` only when the fix requires choosing between multiple defensible alternatives, when the fix spans non-contiguous code, or when the surrounding context the agent cannot see could change the answer. If a mechanical-looking finding omits `suggested-code`, set `suggested-code-omission-reason`.
|
||||
|
||||
Sub-skills MAY also emit `suggested-code` when their knowledge file unambiguously implies the replacement (the `.good.al` and `.bad.al` companion examples are useful here). The super-skill copies the field through unchanged.
|
||||
|
||||
|
|
@ -313,3 +313,4 @@ The empty-corpus case — BCQuality's state until knowledge files land — rolls
|
|||
]
|
||||
}
|
||||
```
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue