mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-08-08 18:31:36 +01:00
Define mandatory suggested-code classes
Follow-up to BCQuality#22. The previous contract made suggested-code expected for mechanical findings, but still left too much judgement to the model. In the synthetic benchmark, BCQ produced omission reasons for 11 mechanical-looking findings instead of suggestions. Add explicit must-suggest classes where omission is not allowed when location points to the affected line/range and the replacement can be expressed as one contiguous block: - missing OptionCaption - missing ToolTip - formatter-only casing/spacing defects - Count() existence/repeated-count fixes - unreachable/dead contiguous blocks - over-broad permission tokens with an obvious narrower token - obvious DataClassification additions - local Label moves when the target range includes both old and new declaration locations Update all AL leaf skills and al-code-review to reference these must-suggest classes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
parent
07efef6bc7
commit
d0e4298518
8 changed files with 32 additions and 8 deletions
|
|
@ -64,8 +64,9 @@ Set `confidence` to:
|
|||
After evaluating each worklist entry, also consider whether the diff exhibits a security defect the agent recognises from its general AL knowledge that no knowledge file in the worklist covers. Such candidates are agent findings within this skill's domain — emit them with `references: []`, an `id` slug prefixed with `agent:`, `confidence` capped at `medium`, and a `message` that is self-contained (describing both the issue and a concrete recommendation, since there is no knowledge-file footer for the consumer to fall back on). The scope is strictly security; defects outside this domain belong to other leaves and MUST NOT be emitted here. Before emitting, check the worklist for a knowledge file that matches the candidate — if one exists, upgrade the candidate to a knowledge-backed finding instead. See `skills/do.md` for the full contract.
|
||||
|
||||
For every emitted finding, decide whether the fix is mechanical. A fix is mechanical when it is small, local, and unambiguous from the diff context (for example: delete unreachable lines; replace `Count() > 0` with `not IsEmpty()`; move a local `Label` to object scope; add a missing `ToolTip`, `OptionCaption`, or `DataClassification`; replace a string-concatenated `Error` with a Label-backed call; change an over-broad permission token; or add an obvious `else`/guard branch). For mechanical findings, emit `findings[].suggested-code` with the literal replacement for the source lines indicated by `location`. The payload must be a verbatim replacement — no diff markers, no fences, no commentary — that the consumer can render as a one-click suggestion. When a `.good.al` companion exists and the diff context matches the `.bad.al` shape, adapt the `.good.al` replacement into `suggested-code`.
|
||||
For must-suggest classes, omission is not allowed when `location` points to the affected line/range and the replacement can be expressed as one contiguous block. Must-suggest classes are: missing `OptionCaption`; missing `ToolTip`; formatter-only casing/spacing defects; `Count() > 0` / repeated `Count()` existence checks; unreachable code after an unconditional terminator; over-broad permission tokens with an obvious narrower token; obvious `DataClassification` additions; and local `Label` moves when the target range includes both the local declaration and the object-level insertion point.
|
||||
|
||||
Omit `suggested-code` only when the appropriate fix depends on context the skill cannot determine, when multiple defensible replacements exist, or when the fix spans non-contiguous code. If a finding is mechanical-looking but you omit `suggested-code`, set `findings[].suggested-code-omission-reason` to a short explanation. See `skills/do.md` for the full contract.
|
||||
Outside the must-suggest classes, omit `suggested-code` only when the appropriate fix depends on context the skill cannot determine, when multiple defensible replacements exist, or when the fix spans non-contiguous code. If a finding is mechanical-looking but you omit `suggested-code`, set `findings[].suggested-code-omission-reason` to a short explanation. See `skills/do.md` for the full contract.
|
||||
|
||||
Outcome selection:
|
||||
|
||||
|
|
@ -135,3 +136,4 @@ The empty-corpus case — BCQuality's state until security knowledge files land
|
|||
}
|
||||
```
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue