bcquality/microsoft/skills/review/al-ui-review.md
Jesper Schulz-Wedde d994f2ad0e Require suggested-code or omission reason for mechanical findings
BCQ now wins on coverage in the synthetic complex benchmark, but only
2 of 25 rendered comments carried GitHub suggestion blocks while the
AIRHack reviewer emitted suggestions for every finding. Tighten the
DO contract and leaf-skill instructions so suggested-code is no longer
a soft affordance for mechanical fixes.

Changes:
- Add optional findings[].suggested-code-omission-reason to the DO
  schema.
- Document suggested-code as expected for small, local, mechanical
  findings, with examples (delete unreachable code, Count() > 0 ->
  not IsEmpty(), move local Label, add missing ToolTip/OptionCaption,
  replace string-concatenated Error, change permission token, add an
  obvious else/guard branch).
- Require omission reasons when a mechanical-looking finding omits
  suggested-code.
- Update all AL leaf skills and al-code-review guidance with the same
  stronger contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 15:00:10 +02:00

7.9 KiB

kind id version title description inputs outputs bc-version technologies countries application-area
action-skill al-ui-review 1 AL UI and accessibility review Reviews AL page and control add-in UI files against UI text, caption, tooltip, and accessibility guidance from BCQuality.
pr-diff
file-path
findings-report
all
al
javascript
w1
all

AL UI and accessibility review

Reviews AL page source and control add-in UI files against the ui knowledge domain in BCQuality and emits a findings report. This is a leaf action skill: it invokes no sub-skills. It is one of the skills composed by al-code-review.

UI findings apply to page files — files that declare PageType = ..., including *.Page.al under the standard file-naming convention — and to JavaScript/CSS/HTML files that render Business Central control add-ins. The skill returns not-applicable when the diff contains no page or control add-in UI changes.

An orchestrator invokes this skill with either a pr-diff or a file-path. The skill produces a single JSON document conforming to the DO output contract.

Source

Collect all knowledge files under */knowledge/ui/**/*.md, across every enabled layer.

Relevance

Apply the frontmatter matching rules defined in READ against the task context:

  • bc-version — the target BC version from the PR branch's app.json or the orchestrator-supplied version. If unavailable, the dimension is unknown.
  • technologies[al] or [javascript].
  • countries — the countries declared in the consuming app's app.json. If absent, unknown.
  • application-area — pass the actual set declared by the changed objects; do not substitute [all].

Discard files that are not applicable. Retain conditionally applicable files only when the orchestrator's configuration permits them; findings derived from those files MUST have confidence no higher than medium and MUST name the unknown dimensions in message.

Worklist

Narrow the relevant files to the subset that applies to the changes under review.

  • UI-file filter. UI review applies to files declaring page, pageextension, or pagecustomization, and to control add-in JavaScript/CSS/HTML that changes rendered UI. When the diff contains no such files, return outcome: "not-applicable" without evaluating knowledge files.
  • For each relevant knowledge file, compute overlap against changed page declarations and control add-in UI files, weighted toward Caption, ToolTip, AboutTitle, AboutText, OptionCaption, ShowCaption, InstructionalText, GridLayout, Style, StyleExpr, action definitions, field-level properties, DOM creation, ARIA attributes, and keyboard/focus handlers.
  • Tokens extracted from the diff (Caption, ToolTip, AboutTitle, AboutText, PageType, ShowCaption, InstructionalText, grid, fixed, GridLayout, Style, StyleExpr, Favorable, Unfavorable, Ambiguous, cuegroup, controladdin, usercontrol, aria-, tabindex, keydown, focus, innerHTML, createElement, &, Specifies, Message(, Confirm(, Error( in a page context, Disabled, Invalid, Whitelist, Blacklist, trailing punctuation patterns on captions).

A file enters the candidate worklist when its keywords intersect the extracted tokens or its topic matches a changed page element.

Once the candidate worklist is known, resolve layer-precedence conflicts per READ and record suppressions.

When the post-conflict worklist is empty because no applicable UI knowledge exists, or because configuration suppressed every candidate, emit outcome: "no-knowledge". When the worklist is empty because no applicable UI knowledge matched the page changes, emit outcome: "completed" with an empty findings array.

Action

For each worklist entry, evaluate the diff against the file's ## Best Practice and ## Anti Pattern sections. UI text findings are generally minor — they affect localization and polish rather than correctness. Accessibility findings for missing labels, broken grid semantics, semantic color without text meaning, or UI-rendering control add-in changes can be major; use minor for low-risk manual-review reminders and polish issues.

Set confidence to:

  • high when the detection is based on an unambiguous pattern match (banned term literal, missing "Specifies" opener on a field tooltip, caption exceeding documented limit).
  • medium when detection relies on heuristics (judging whether a caption is a noun phrase or a sentence phrase) or when any frontmatter dimension was unknown.
  • low when the finding is an advisory derived only from applicability.

After evaluating each worklist entry, also consider whether the diff exhibits a UI defect the agent recognises from its general AL knowledge that no knowledge file in the worklist covers. Such candidates are agent findings within this skill's domain — emit them with references: [], an id slug prefixed with agent:, confidence capped at medium, and a message that is self-contained (describing both the issue and a concrete recommendation, since there is no knowledge-file footer for the consumer to fall back on). The scope is strictly UI; defects outside this domain belong to other leaves and MUST NOT be emitted here. Before emitting, check the worklist for a knowledge file that matches the candidate — if one exists, upgrade the candidate to a knowledge-backed finding instead. See skills/do.md for the full contract.

For every emitted finding, decide whether the fix is mechanical. A fix is mechanical when it is small, local, and unambiguous from the diff context (for example: delete unreachable lines; replace Count() > 0 with not IsEmpty(); move a local Label to object scope; add a missing ToolTip, OptionCaption, or DataClassification; replace a string-concatenated Error with a Label-backed call; change an over-broad permission token; or add an obvious else/guard branch). For mechanical findings, emit findings[].suggested-code with the literal replacement for the source lines indicated by location. The payload must be a verbatim replacement — no diff markers, no fences, no commentary — that the consumer can render as a one-click suggestion. When a .good.al companion exists and the diff context matches the .bad.al shape, adapt the .good.al replacement into suggested-code.

Omit suggested-code only when the appropriate fix depends on context the skill cannot determine, when multiple defensible replacements exist, or when the fix spans non-contiguous code. If a finding is mechanical-looking but you omit suggested-code, set findings[].suggested-code-omission-reason to a short explanation. See skills/do.md for the full contract.

Outcome selection:

  • completed — the skill evaluated every worklist item.
  • no-knowledge — no applicable UI knowledge survived filtering.
  • not-applicable — the diff contains no page, pageextension, pagecustomization, or control add-in UI files.
  • partial — a budget was hit before the worklist was exhausted.
  • failed — an unrecoverable error occurred.

Output

Output conforms to the DO output contract. A populated example:

{
  "skill": { "id": "al-ui-review", "version": 1 },
  "outcome": "completed",
  "summary": {
    "counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
    "coverage": { "worklist-size": 1, "items-evaluated": 1 }
  },
  "findings": [
    {
      "id": "microsoft/knowledge/ui/field-tooltips-start-with-specifies-and-end-with-period.md",
      "severity": "minor",
      "message": "Field ToolTip is a fragment ('Customer name') — missing the 'Specifies' opener and the terminating period the house-style guidance requires.",
      "location": {
        "file": "src/Sales/CustomerCard.Page.al",
        "line": 58
      },
      "references": [
        { "path": "microsoft/knowledge/ui/field-tooltips-start-with-specifies-and-end-with-period.md" }
      ],
      "confidence": "high"
    }
  ],
  "suppressed": []
}