mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-08-06 17:36:53 +01:00
Add missing AL review leaves
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 62d512a7-fd54-43dc-8eb5-485b909c72e5
This commit is contained in:
Jesper Schulz-Wedde
parent
9214f73819
commit
aef9d36fcb
5 changed files with 521 additions and 1 deletions
128
microsoft/skills/review/al-appsource-review.md
Normal file
128
microsoft/skills/review/al-appsource-review.md
Normal file
|
|
@ -0,0 +1,128 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-appsource-review
|
||||
version: 1
|
||||
title: AL AppSource review
|
||||
description: Performs an AL AppSource review against source and app metadata guidance from BCQuality.
|
||||
inputs: [pr-diff, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL AppSource review
|
||||
|
||||
Reviews AL source and app metadata changes against the `appsource` knowledge domain in BCQuality and emits a findings report. This is a leaf action skill: it invokes no sub-skills. It is one of the skills composed by `al-code-review`.
|
||||
|
||||
An orchestrator invokes this skill with either a `pr-diff` (the standard PR-review entry point) or a `file-path` (single-file review). AppSource findings are narrow by design — they apply when the diff touches AppSourceCop configuration, AL object or extension-member names, or AppSource-facing `app.json` metadata. The skill returns `not-applicable` when none of those apply.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `appsource` as this skill's candidate set across every enabled Microsoft, community, and custom layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/appsource/**`.
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ (*Frontmatter matching semantics*) against the task context:
|
||||
|
||||
- `bc-version` — the target BC version from the PR branch's `app.json` or the orchestrator-supplied version. If unavailable, the dimension is `unknown`.
|
||||
- `technologies` — `[al]`.
|
||||
- `countries` — the countries declared in the consuming app's `app.json`. Default to the orchestrator's configured context; if absent, `unknown`.
|
||||
- `application-area` — the union of application areas declared by the changed objects. Pass the actual set; do not substitute `[all]`. If the area cannot be determined from the changes, the dimension is `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when the orchestrator's configuration permits them; findings derived from those files MUST have `confidence` no higher than `medium`, AND the finding's `message` MUST name the dimension or dimensions that were unknown.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow the relevant files to the subset that applies to the changes under review. For each relevant file, compute overlap against:
|
||||
|
||||
- The changed files and AL object types — especially `app.json`, `AppSourceCop.json`, new objects, and table/page/report extensions that add fields, keys, controls, or actions to base objects.
|
||||
- The changed object and member names, weighted toward prefix/suffix consistency with `mandatoryAffixes` or `mandatoryPrefix`, plus AppSource-facing help metadata.
|
||||
- Tokens extracted from the diff that relate to AppSource (`AppSourceCop`, `mandatoryAffixes`, `mandatoryPrefix`, `AS0011`, `prefix`, `suffix`, `tableextension`, `pageextension`, `reportextension`, `field`, `key`, `control`, `action`, `app.json`, `help`, `ContextSensitiveHelpPage`, `Copilot`, `https`).
|
||||
|
||||
A file enters the candidate worklist when its `keywords` intersect the extracted tokens or its topic (derived from the index entry's `path`, `title`, and `description`) matches a changed object type. Read an article's full file — its `## Best Practice` / `## Anti Pattern` bodies — only after it makes the worklist; candidate selection uses the index alone. When the diff contains no AppSource-related source or metadata changes by any of the above signals, return `outcome: "not-applicable"` without evaluating files.
|
||||
|
||||
The following targeted checks cover every current `appsource` article across the Microsoft and community layers. Treat each as a candidate-selection cue: when the signal appears in changed code, add the named article to the worklist and evaluate it in Action.
|
||||
|
||||
- A new or renamed object lacks the reserved prefix/suffix, or a tableextension/pageextension/reportextension adds an unaffixed field, key, control, or action to a base object despite `mandatoryAffixes`/`mandatoryPrefix` and AS0011 — `object-affixes-prevent-collisions`.
|
||||
- For BC v24 or later, `app.json` adds or changes the `help` URL to a path deeper than two levels, or a changed Copilot/context-sensitive help arrangement would ground the app under an overly broad truncated parent — `keep-copilot-help-url-to-two-path-levels`.
|
||||
|
||||
Once the candidate worklist is known, resolve layer-precedence conflicts per READ. Drop lower-precedence files whose normative guidance (`## Best Practice` or `## Anti Pattern`) directly contradicts a higher-precedence candidate, and record each dropped file in `suppressed` with `reason: "layer-precedence"`. Files that would have been candidates but are hidden because their layer is disabled in consumer configuration are recorded with `reason: "configuration"`. Files that never became candidates are NOT recorded in `suppressed`.
|
||||
|
||||
When the post-conflict worklist is empty because no applicable AppSource knowledge exists, or because configuration suppressed every candidate, emit `outcome: "no-knowledge"`. When the worklist is empty because no applicable AppSource knowledge matched the changes, emit `outcome: "completed"` with an empty `findings` array.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklist entry, evaluate the diff against the file's `## Best Practice` and `## Anti Pattern` sections. Emit findings as follows:
|
||||
|
||||
- When the diff contains a clear match for an Anti Pattern, emit a finding with severity `major` or `blocker`, a message summarizing the anti-pattern, `location` pointing to the offending line or range, and a `references` entry pointing to the knowledge file. Use `blocker` only when the knowledge file states the change violates an AppSource submission requirement; otherwise the ceiling is `major`.
|
||||
- When the diff contains code that contradicts a Best Practice without being a full anti-pattern, emit `minor` with the same reference shape.
|
||||
- When the skill cannot detect a violation but the file is clearly applicable to the change, emit `info` citing the file.
|
||||
|
||||
Set `confidence` to:
|
||||
|
||||
- `high` when the detection is based on an unambiguous pattern match (affix configuration/name or URL path depth).
|
||||
- `medium` when detection relies on heuristics or when any frontmatter dimension was `unknown`.
|
||||
- `low` when the finding is an advisory derived only from applicability.
|
||||
|
||||
After evaluating each worklist entry, also consider whether the diff exhibits an AppSource defect the agent recognises from its general AL knowledge that no knowledge file in the worklist covers. Such candidates are agent findings within this skill's domain — emit them with `references: []`, an `id` slug prefixed with `agent:`, `confidence` capped at `medium`, `severity` capped at `minor` (agent findings are advisory and non-gating), and a `message` that is self-contained (describing both the issue and a concrete recommendation, since there is no knowledge-file footer for the consumer to fall back on). Hold every candidate to the precision bar in `skills/do.md` (*Agent findings*): emit only a concrete, material AppSource defect a knowledgeable BC reviewer would agree is wrong — steelman it first and drop anything stylistic, speculative, dependent on code outside the diff, or merely a valid alternative; when in doubt, omit. The scope is strictly AppSource; defects outside this domain belong to other leaves and MUST NOT be emitted here. Before emitting, check the worklist for a knowledge file that matches the candidate — if one exists, upgrade the candidate to a knowledge-backed finding instead. See `skills/do.md` for the full contract.
|
||||
|
||||
For every emitted finding, decide whether the fix is mechanical. A fix is mechanical when it is small, local, and unambiguous from the diff context (for example: add the configured affix to one object or extension member, or replace a deep help URL with a known two-level canonical URL). For mechanical findings, emit `findings[].suggested-code` with the literal replacement for the source lines indicated by `location`. The payload must be a verbatim replacement — no diff markers, no fences, no commentary — that the consumer can render as a one-click suggestion. When a `.good.al` companion exists and the diff context matches the `.bad.al` shape, adapt the `.good.al` replacement into `suggested-code`.
|
||||
|
||||
Omit `suggested-code` only when the appropriate fix depends on context the skill cannot determine, when multiple defensible replacements exist, or when the fix spans non-contiguous code. If a finding is mechanical-looking but you omit `suggested-code`, set `findings[].suggested-code-omission-reason` to a short explanation. See `skills/do.md` for the full contract.
|
||||
|
||||
Outcome selection:
|
||||
|
||||
- `completed` — the skill evaluated every worklist item.
|
||||
- `no-knowledge` — no applicable AppSource knowledge survived filtering.
|
||||
- `not-applicable` — the diff touches no AppSource source, analyzer configuration, or app-metadata surface.
|
||||
- `partial` — a budget was hit before the worklist was exhausted.
|
||||
- `failed` — an unrecoverable error occurred.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-appsource-review", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 1, "items-evaluated": 1 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "microsoft/knowledge/appsource/object-affixes-prevent-collisions.md",
|
||||
"severity": "major",
|
||||
"message": "The tableextension adds an unaffixed Loyalty Points field to Customer, so it violates the configured AppSource affix and can collide with another extension.",
|
||||
"location": {
|
||||
"file": "src/CustomerExt.TableExt.al",
|
||||
"line": 8
|
||||
},
|
||||
"references": [
|
||||
{ "path": "microsoft/knowledge/appsource/object-affixes-prevent-collisions.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"suggested-code": "field(50100; \"Loyalty Points ABC\"; Integer)"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
|
||||
The empty-corpus case produces:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-appsource-review", "version": 1 },
|
||||
"outcome": "no-knowledge",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 0, "items-evaluated": 0 }
|
||||
},
|
||||
"findings": [],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
|
|
@ -22,6 +22,10 @@ sub-skills:
|
|||
- microsoft/skills/review/al-interfaces-review.md
|
||||
- microsoft/skills/review/al-breaking-changes-review.md
|
||||
- microsoft/skills/review/al-web-services-review.md
|
||||
- microsoft/skills/review/al-testing-review.md
|
||||
- microsoft/skills/review/al-data-modeling-review.md
|
||||
- microsoft/skills/review/al-appsource-review.md
|
||||
- microsoft/skills/review/al-telemetry-review.md
|
||||
---
|
||||
|
||||
# AL code review
|
||||
|
|
@ -34,7 +38,7 @@ An orchestrator invokes this skill with either a `pr-diff` (the standard PR-revi
|
|||
|
||||
## Source
|
||||
|
||||
The sub-skills invoked by this skill are those listed in frontmatter `sub-skills`. Additional leaf skills (for example, telemetry, testing) are added by updating the `sub-skills` list. The skill does not discover sub-skills implicitly.
|
||||
The sub-skills invoked by this skill are those listed in frontmatter `sub-skills`. Additional leaf skills are added by updating the `sub-skills` list. The skill does not discover sub-skills implicitly.
|
||||
|
||||
## Relevance
|
||||
|
||||
|
|
|
|||
131
microsoft/skills/review/al-data-modeling-review.md
Normal file
131
microsoft/skills/review/al-data-modeling-review.md
Normal file
|
|
@ -0,0 +1,131 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-data-modeling-review
|
||||
version: 1
|
||||
title: AL data-modeling review
|
||||
description: Performs an AL data-modeling review against guidance from BCQuality.
|
||||
inputs: [pr-diff, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL data-modeling review
|
||||
|
||||
Reviews AL source changes against the `data-modeling` knowledge domain in BCQuality and emits a findings report. This is a leaf action skill: it invokes no sub-skills. It is one of the skills composed by `al-code-review`.
|
||||
|
||||
An orchestrator invokes this skill with either a `pr-diff` (the standard PR-review entry point) or a `file-path` (single-file review). Data-modeling findings are narrow by design — they apply when the diff touches setup or master tables, their card pages, primary keys, number-series assignment, block enforcement, or audit fields. The skill returns `not-applicable` when none of those apply.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `data-modeling` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/data-modeling/**`.
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ (*Frontmatter matching semantics*) against the task context:
|
||||
|
||||
- `bc-version` — the target BC version from the PR branch's `app.json` or the orchestrator-supplied version. If unavailable, the dimension is `unknown`.
|
||||
- `technologies` — `[al]`.
|
||||
- `countries` — the countries declared in the consuming app's `app.json`. Default to the orchestrator's configured context; if absent, `unknown`.
|
||||
- `application-area` — the union of application areas declared by the changed objects. Pass the actual set; do not substitute `[all]`. If the area cannot be determined from the changes, the dimension is `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when the orchestrator's configuration permits them; findings derived from those files MUST have `confidence` no higher than `medium`, AND the finding's `message` MUST name the dimension or dimensions that were unknown.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow the relevant files to the subset that applies to the changes under review. For each relevant file, compute overlap against:
|
||||
|
||||
- The changed AL object names and types — especially `* Setup` singleton tables and Card pages, custom master tables, tableextensions that add master-data fields, and document or journal lines that reference a master.
|
||||
- The changed fields, keys, triggers, and procedures, weighted toward `Primary Key`, `No.`, `No. Series`, `Blocked`, `Last Date Modified`, `OnInsert`, `OnModify`, `OnRename`, reference-field `OnValidate`, and posting validation.
|
||||
- Tokens extracted from the diff that relate to data modeling (`setup`, `master`, `Primary Key`, `Code[10]`, `Code[20]`, `AutoIncrement`, `SystemId`, `No.`, `No. Series`, `NoSeriesManagement`, `Codeunit "No. Series"`, `GetNextNo`, `IsManual`, `TestManual`, `Blocked`, `TestField`, `Last Date Modified`, `Today`, `WorkDate`, `InsertAllowed`, `DeleteAllowed`, `PageType = Card`, `OnOpenPage`, `GetRecordOnce`, `OnInsert`, `OnModify`, `OnRename`).
|
||||
|
||||
A file enters the candidate worklist when its `keywords` intersect the extracted tokens or its topic (derived from the index entry's `path`, `title`, and `description`) matches a changed object type. Read an article's full file — its `## Best Practice` / `## Anti Pattern` bodies — only after it makes the worklist; candidate selection uses the index alone. When the diff contains no data-modeling changes by any of the above signals, return `outcome: "not-applicable"` without evaluating files.
|
||||
|
||||
The following targeted checks cover every current `data-modeling` article. Treat each as a candidate-selection cue: when the signal appears in changed code, add the named article to the worklist and evaluate it in Action.
|
||||
|
||||
- A `* Setup` table or its page changes singleton structure, uses a nonblank or generated key, permits insert/delete, uses a List page, or does not ensure the blank-keyed row exists — `setup-table-is-a-singleton`.
|
||||
- A custom master table changes its primary key, `No.`/`No. Series` fields, or `OnInsert` without assigning a blank `No.` from setup through a number series — `master-table-no-from-number-series-in-oninsert`.
|
||||
- BC v22 or later code introduces or retains `NoSeriesManagement`, `InitSeries`, `SelectSeries`, or `SetSeries`, or number assignment/manual-entry checks do not use codeunit `"No. Series"` methods such as `GetNextNo`, `IsManual`, or `TestManual` — `use-no-series-codeunit-not-noseriesmanagement`.
|
||||
- A master gains or changes `Blocked`, or a document line, journal line, reference-field `OnValidate`, or posting routine uses that master without `TestField(Blocked, false)` at the point of use; also cue when the check is placed only in the master's own triggers — `check-blocked-in-referencing-code-not-in-master`.
|
||||
- A master table adds or changes `Last Date Modified`, `OnModify`, or `OnRename`, but the non-editable field is not assigned `Today()` in both triggers — `set-last-date-modified-in-onmodify-and-onrename`.
|
||||
|
||||
Once the candidate worklist is known, resolve layer-precedence conflicts per READ. Drop lower-precedence files whose normative guidance (`## Best Practice` or `## Anti Pattern`) directly contradicts a higher-precedence candidate, and record each dropped file in `suppressed` with `reason: "layer-precedence"`. Files that would have been candidates but are hidden because their layer is disabled in consumer configuration are recorded with `reason: "configuration"`. Files that never became candidates are NOT recorded in `suppressed`.
|
||||
|
||||
When the post-conflict worklist is empty because no applicable data-modeling knowledge exists, or because configuration suppressed every candidate, emit `outcome: "no-knowledge"`. When the worklist is empty because no applicable data-modeling knowledge matched the changes, emit `outcome: "completed"` with an empty `findings` array.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklist entry, evaluate the diff against the file's `## Best Practice` and `## Anti Pattern` sections. Emit findings as follows:
|
||||
|
||||
- When the diff contains a clear match for an Anti Pattern, emit a finding with severity `major` or `blocker`, a message summarizing the anti-pattern, `location` pointing to the offending line or range, and a `references` entry pointing to the knowledge file. Use `blocker` only when the model can create ambiguous setup state, incompatible business identifiers, or silently stale synchronization data; otherwise the ceiling is `major`.
|
||||
- When the diff contains code that contradicts a Best Practice without being a full anti-pattern, emit `minor` with the same reference shape.
|
||||
- When the skill cannot detect a violation but the file is clearly applicable to the change, emit `info` citing the file.
|
||||
|
||||
Set `confidence` to:
|
||||
|
||||
- `high` when the detection is based on an unambiguous pattern match (object type, field, key, trigger, or API name).
|
||||
- `medium` when detection relies on heuristics or when any frontmatter dimension was `unknown`.
|
||||
- `low` when the finding is an advisory derived only from applicability.
|
||||
|
||||
After evaluating each worklist entry, also consider whether the diff exhibits a data-modeling defect the agent recognises from its general AL knowledge that no knowledge file in the worklist covers. Such candidates are agent findings within this skill's domain — emit them with `references: []`, an `id` slug prefixed with `agent:`, `confidence` capped at `medium`, `severity` capped at `minor` (agent findings are advisory and non-gating), and a `message` that is self-contained (describing both the issue and a concrete recommendation, since there is no knowledge-file footer for the consumer to fall back on). Hold every candidate to the precision bar in `skills/do.md` (*Agent findings*): emit only a concrete, material data-modeling defect a knowledgeable BC reviewer would agree is wrong — steelman it first and drop anything stylistic, speculative, dependent on code outside the diff, or merely a valid alternative; when in doubt, omit. The scope is strictly data modeling; defects outside this domain belong to other leaves and MUST NOT be emitted here. Before emitting, check the worklist for a knowledge file that matches the candidate — if one exists, upgrade the candidate to a knowledge-backed finding instead. See `skills/do.md` for the full contract.
|
||||
|
||||
For every emitted finding, decide whether the fix is mechanical. A fix is mechanical when it is small, local, and unambiguous from the diff context (for example: add `InsertAllowed = false` or `DeleteAllowed = false`; replace `WorkDate()` with `Today()`; add the same audit-field assignment to `OnRename`; or replace an obsolete number-series codeunit declaration). For mechanical findings, emit `findings[].suggested-code` with the literal replacement for the source lines indicated by `location`. The payload must be a verbatim replacement — no diff markers, no fences, no commentary — that the consumer can render as a one-click suggestion. When a `.good.al` companion exists and the diff context matches the `.bad.al` shape, adapt the `.good.al` replacement into `suggested-code`.
|
||||
|
||||
Omit `suggested-code` only when the appropriate fix depends on context the skill cannot determine, when multiple defensible replacements exist, or when the fix spans non-contiguous code. If a finding is mechanical-looking but you omit `suggested-code`, set `findings[].suggested-code-omission-reason` to a short explanation. See `skills/do.md` for the full contract.
|
||||
|
||||
Outcome selection:
|
||||
|
||||
- `completed` — the skill evaluated every worklist item.
|
||||
- `no-knowledge` — no applicable data-modeling knowledge survived filtering.
|
||||
- `not-applicable` — the diff touches no setup/master table, page, key, numbering, block-check, or audit-field surface.
|
||||
- `partial` — a budget was hit before the worklist was exhausted.
|
||||
- `failed` — an unrecoverable error occurred.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-data-modeling-review", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 1, "items-evaluated": 1 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "microsoft/knowledge/data-modeling/set-last-date-modified-in-onmodify-and-onrename.md",
|
||||
"severity": "major",
|
||||
"message": "The table updates Last Date Modified in OnModify but not OnRename, so renaming the primary key leaves the audit date stale and can hide the record from incremental integrations.",
|
||||
"location": {
|
||||
"file": "src/LoyaltyMember.Table.al",
|
||||
"line": 74
|
||||
},
|
||||
"references": [
|
||||
{ "path": "microsoft/knowledge/data-modeling/set-last-date-modified-in-onmodify-and-onrename.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"suggested-code": "trigger OnRename()\nbegin\n \"Last Date Modified\" := Today();\nend;"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
|
||||
The empty-corpus case produces:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-data-modeling-review", "version": 1 },
|
||||
"outcome": "no-knowledge",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 0, "items-evaluated": 0 }
|
||||
},
|
||||
"findings": [],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
127
microsoft/skills/review/al-telemetry-review.md
Normal file
127
microsoft/skills/review/al-telemetry-review.md
Normal file
|
|
@ -0,0 +1,127 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-telemetry-review
|
||||
version: 1
|
||||
title: AL telemetry review
|
||||
description: Performs an AL telemetry review against guidance from BCQuality.
|
||||
inputs: [pr-diff, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL telemetry review
|
||||
|
||||
Reviews AL source changes against the `telemetry` knowledge domain in BCQuality and emits a findings report. This is a leaf action skill: it invokes no sub-skills. It is one of the skills composed by `al-code-review`.
|
||||
|
||||
An orchestrator invokes this skill with either a `pr-diff` (the standard PR-review entry point) or a `file-path` (single-file review). Telemetry findings are narrow by design — they apply when the diff emits, wraps, or changes custom telemetry through `Session.LogMessage`, `Session.LogError`, `FeatureTelemetry`, or related telemetry helpers. The skill returns `not-applicable` when none of those apply.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `telemetry` as this skill's candidate set across every enabled Microsoft, community, and custom layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/telemetry/**`.
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ (*Frontmatter matching semantics*) against the task context:
|
||||
|
||||
- `bc-version` — the target BC version from the PR branch's `app.json` or the orchestrator-supplied version. If unavailable, the dimension is `unknown`.
|
||||
- `technologies` — `[al]`.
|
||||
- `countries` — the countries declared in the consuming app's `app.json`. Default to the orchestrator's configured context; if absent, `unknown`.
|
||||
- `application-area` — the union of application areas declared by the changed objects. Pass the actual set; do not substitute `[all]`. If the area cannot be determined from the changes, the dimension is `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when the orchestrator's configuration permits them; findings derived from those files MUST have `confidence` no higher than `medium`, AND the finding's `message` MUST name the dimension or dimensions that were unknown.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow the relevant files to the subset that applies to the changes under review. For each relevant file, compute overlap against:
|
||||
|
||||
- The changed AL objects and procedures — especially telemetry wrapper codeunits, feature lifecycle instrumentation, error logging, integration diagnostics, and background/session processing.
|
||||
- Calls to `Session.LogMessage`, `Session.LogError`, or `FeatureTelemetry` methods, weighted toward the event ID, verbosity, data classification, custom dimensions, and `TelemetryScope` arguments.
|
||||
- Tokens extracted from the diff that relate to telemetry (`Session.LogMessage`, `Session.LogError`, `FeatureTelemetry`, `TelemetryScope`, `ExtensionPublisher`, `All`, `Verbosity`, `DataClassification`, `CustomDimensions`, `Application Insights`, `LogUsage`, `LogError`, `LogUptake`, `Feature Uptake Status`).
|
||||
|
||||
A file enters the candidate worklist when its `keywords` intersect the extracted tokens or its topic (derived from the index entry's `path`, `title`, and `description`) matches a changed object type. Read an article's full file — its `## Best Practice` / `## Anti Pattern` bodies — only after it makes the worklist; candidate selection uses the index alone. When the diff contains no telemetry-related changes by any of the above signals, return `outcome: "not-applicable"` without evaluating files.
|
||||
|
||||
The following targeted check covers every current `telemetry` article across the Microsoft and community layers. Treat it as a candidate-selection cue: when the signal appears in changed code, add the named article to the worklist and evaluate it in Action.
|
||||
|
||||
- `Session.LogMessage` or `Session.LogError` uses `TelemetryScope::All` for publisher-only diagnostics, a telemetry wrapper defaults its scope to `All`, or a `FeatureTelemetry`/custom logging change routes signals to customer environment telemetry without a customer-actionable reason — `default-telemetryscope-to-extensionpublisher`.
|
||||
|
||||
Once the candidate worklist is known, resolve layer-precedence conflicts per READ. Drop lower-precedence files whose normative guidance (`## Best Practice` or `## Anti Pattern`) directly contradicts a higher-precedence candidate, and record each dropped file in `suppressed` with `reason: "layer-precedence"`. Files that would have been candidates but are hidden because their layer is disabled in consumer configuration are recorded with `reason: "configuration"`. Files that never became candidates are NOT recorded in `suppressed`.
|
||||
|
||||
When the post-conflict worklist is empty because no applicable telemetry knowledge exists, or because configuration suppressed every candidate, emit `outcome: "no-knowledge"`. When the worklist is empty because no applicable telemetry knowledge matched the changes, emit `outcome: "completed"` with an empty `findings` array.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklist entry, evaluate the diff against the file's `## Best Practice` and `## Anti Pattern` sections. Emit findings as follows:
|
||||
|
||||
- When the diff contains a clear match for an Anti Pattern, emit a finding with severity `major` or `blocker`, a message summarizing the anti-pattern, `location` pointing to the offending line or range, and a `references` entry pointing to the knowledge file. Use `blocker` only when the knowledge file states the anti-pattern violates a platform-level guarantee; otherwise the ceiling is `major`.
|
||||
- When the diff contains code that contradicts a Best Practice without being a full anti-pattern, emit `minor` with the same reference shape.
|
||||
- When the skill cannot detect a violation but the file is clearly applicable to the change, emit `info` citing the file.
|
||||
|
||||
Set `confidence` to:
|
||||
|
||||
- `high` when the detection is based on an unambiguous API and `TelemetryScope` argument.
|
||||
- `medium` when determining whether a signal is customer-actionable requires heuristic interpretation or when any frontmatter dimension was `unknown`.
|
||||
- `low` when the finding is an advisory derived only from applicability.
|
||||
|
||||
After evaluating each worklist entry, also consider whether the diff exhibits a telemetry defect the agent recognises from its general AL knowledge that no knowledge file in the worklist covers. Such candidates are agent findings within this skill's domain — emit them with `references: []`, an `id` slug prefixed with `agent:`, `confidence` capped at `medium`, `severity` capped at `minor` (agent findings are advisory and non-gating), and a `message` that is self-contained (describing both the issue and a concrete recommendation, since there is no knowledge-file footer for the consumer to fall back on). Hold every candidate to the precision bar in `skills/do.md` (*Agent findings*): emit only a concrete, material telemetry defect a knowledgeable BC reviewer would agree is wrong — steelman it first and drop anything stylistic, speculative, dependent on code outside the diff, or merely a valid alternative; when in doubt, omit. The scope is strictly telemetry; defects outside this domain belong to other leaves and MUST NOT be emitted here. Before emitting, check the worklist for a knowledge file that matches the candidate — if one exists, upgrade the candidate to a knowledge-backed finding instead. See `skills/do.md` for the full contract.
|
||||
|
||||
For every emitted finding, decide whether the fix is mechanical. A fix is mechanical when it is small, local, and unambiguous from the diff context (for example: replace `TelemetryScope::All` with `TelemetryScope::ExtensionPublisher` for a clearly publisher-only diagnostic). For mechanical findings, emit `findings[].suggested-code` with the literal replacement for the source lines indicated by `location`. The payload must be a verbatim replacement — no diff markers, no fences, no commentary — that the consumer can render as a one-click suggestion. When a `.good.al` companion exists and the diff context matches the `.bad.al` shape, adapt the `.good.al` replacement into `suggested-code`.
|
||||
|
||||
Omit `suggested-code` only when the appropriate fix depends on context the skill cannot determine, when multiple defensible replacements exist, or when the fix spans non-contiguous code. If a finding is mechanical-looking but you omit `suggested-code`, set `findings[].suggested-code-omission-reason` to a short explanation. See `skills/do.md` for the full contract.
|
||||
|
||||
Outcome selection:
|
||||
|
||||
- `completed` — the skill evaluated every worklist item.
|
||||
- `no-knowledge` — no applicable telemetry knowledge survived filtering.
|
||||
- `not-applicable` — the diff touches no telemetry emission, wrapper, or feature-instrumentation surface.
|
||||
- `partial` — a budget was hit before the worklist was exhausted.
|
||||
- `failed` — an unrecoverable error occurred.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-telemetry-review", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 1, "items-evaluated": 1 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "community/knowledge/telemetry/default-telemetryscope-to-extensionpublisher.md",
|
||||
"severity": "major",
|
||||
"message": "This publisher-only diagnostic uses TelemetryScope::All, which also sends it to each customer's environment telemetry and adds avoidable ingestion cost.",
|
||||
"location": {
|
||||
"file": "src/Telemetry/Diagnostics.Codeunit.al",
|
||||
"line": 31
|
||||
},
|
||||
"references": [
|
||||
{ "path": "community/knowledge/telemetry/default-telemetryscope-to-extensionpublisher.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"suggested-code": "TelemetryScope::ExtensionPublisher"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
|
||||
The empty-corpus case produces:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-telemetry-review", "version": 1 },
|
||||
"outcome": "no-knowledge",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 0, "items-evaluated": 0 }
|
||||
},
|
||||
"findings": [],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
130
microsoft/skills/review/al-testing-review.md
Normal file
130
microsoft/skills/review/al-testing-review.md
Normal file
|
|
@ -0,0 +1,130 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-testing-review
|
||||
version: 1
|
||||
title: AL testing review
|
||||
description: Performs an AL testing review against guidance from BCQuality.
|
||||
inputs: [pr-diff, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL testing review
|
||||
|
||||
Reviews AL source changes against the `testing` knowledge domain in BCQuality and emits a findings report. This is a leaf action skill: it invokes no sub-skills. It is one of the skills composed by `al-code-review`.
|
||||
|
||||
An orchestrator invokes this skill with either a `pr-diff` (the standard PR-review entry point) or a `file-path` (single-file review). Testing findings are narrow by design — they apply when the diff touches test codeunits, test runners, test methods, handlers, assertions, or fixture construction. The skill returns `not-applicable` when none of those apply.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `testing` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/testing/**`.
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ (*Frontmatter matching semantics*) against the task context:
|
||||
|
||||
- `bc-version` — the target BC version from the PR branch's `app.json` or the orchestrator-supplied version. If unavailable, the dimension is `unknown`.
|
||||
- `technologies` — `[al]`.
|
||||
- `countries` — the countries declared in the consuming app's `app.json`. Default to the orchestrator's configured context; if absent, `unknown`.
|
||||
- `application-area` — the union of application areas declared by the changed objects. Pass the actual set; do not substitute `[all]`. If the area cannot be determined from the changes, the dimension is `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when the orchestrator's configuration permits them; findings derived from those files MUST have `confidence` no higher than `medium`, AND the finding's `message` MUST name the dimension or dimensions that were unknown.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow the relevant files to the subset that applies to the changes under review. For each relevant file, compute overlap against:
|
||||
|
||||
- The changed AL object names and types — especially codeunits with `Subtype = Test`, test runner codeunits with `TestIsolation`, test libraries, and codeunits that define UI handlers.
|
||||
- The changed methods and attributes, weighted toward `[Test]`, `[TransactionModel(...)]`, `[HandlerFunctions(...)]`, handler attributes, `asserterror`, `ExpectedError`, `ExpectedErrorCode`, fixture initialization, and test-library calls.
|
||||
- Tokens extracted from the diff that relate to testing (`Subtype = Test`, `TestIsolation`, `TransactionModel`, `AutoRollback`, `AutoCommit`, `Commit`, `asserterror`, `ExpectedError`, `ExpectedErrorCode`, `HandlerFunctions`, `ConfirmHandler`, `MessageHandler`, `StrMenuHandler`, `ModalPageHandler`, `Enqueue`, `Dequeue`, `AssertEmpty`, `Library Assert`, `LibraryVariableStorage`, `LibrarySales`, `LibraryPurchase`, `LibraryERM`, `LibraryInventory`, `LibraryRandom`, `Init`, `Insert`).
|
||||
|
||||
A file enters the candidate worklist when its `keywords` intersect the extracted tokens or its topic (derived from the index entry's `path`, `title`, and `description`) matches a changed object type. Read an article's full file — its `## Best Practice` / `## Anti Pattern` bodies — only after it makes the worklist; candidate selection uses the index alone. When the diff contains no testing-related changes by any of the above signals, return `outcome: "not-applicable"` without evaluating files.
|
||||
|
||||
The following targeted checks cover every current `testing` article. Treat each as a candidate-selection cue: when the signal appears in changed code, add the named article to the worklist and evaluate it in Action.
|
||||
|
||||
- A method in a `Subtype = Test` codeunit adds or changes `[TransactionModel(...)]`, exercises code that calls `Commit`, defaults broadly to `AutoCommit`, or is run without a `TestIsolation`-enabled runner — `transactionmodel-attribute-governs-test-transactions`.
|
||||
- Test fixture code manually calls `Init`/`Insert`, invents keys or prerequisite records, or bypasses available `LibrarySales`, `LibraryPurchase`, `LibraryERM`, `LibraryInventory`, `LibraryRandom`, or equivalent library codeunits — `use-library-codeunits-for-test-fixtures`.
|
||||
- `asserterror` is added or changed without a following `Assert.ExpectedError`, `Assert.ExpectedErrorCode`, or a purpose-built assertion such as `ExpectedTestFieldError` — `asserterror-needs-expectederror-and-code`.
|
||||
- A test path raises UI, `[HandlerFunctions(...)]` does not exactly match the invoked handlers, a handler hardcodes replies instead of using enqueue/dequeue expectations, or `LibraryVariableStorage.Clear`/`AssertEmpty` is missing — `ui-handlers-in-tests`.
|
||||
|
||||
Once the candidate worklist is known, resolve layer-precedence conflicts per READ. Drop lower-precedence files whose normative guidance (`## Best Practice` or `## Anti Pattern`) directly contradicts a higher-precedence candidate, and record each dropped file in `suppressed` with `reason: "layer-precedence"`. Files that would have been candidates but are hidden because their layer is disabled in consumer configuration are recorded with `reason: "configuration"`. Files that never became candidates are NOT recorded in `suppressed`.
|
||||
|
||||
When the post-conflict worklist is empty because no applicable testing knowledge exists, or because configuration suppressed every candidate, emit `outcome: "no-knowledge"`. When the worklist is empty because no applicable testing knowledge matched the changes, emit `outcome: "completed"` with an empty `findings` array.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklist entry, evaluate the diff against the file's `## Best Practice` and `## Anti Pattern` sections. Emit findings as follows:
|
||||
|
||||
- When the diff contains a clear match for an Anti Pattern, emit a finding with severity `major` or `blocker`, a message summarizing the anti-pattern, `location` pointing to the offending line or range, and a `references` entry pointing to the knowledge file. Use `blocker` only when the test can pass while verifying the wrong behavior or can leave committed data that contaminates later tests; otherwise the ceiling is `major`.
|
||||
- When the diff contains code that contradicts a Best Practice without being a full anti-pattern, emit `minor` with the same reference shape.
|
||||
- When the skill cannot detect a violation but the file is clearly applicable to the change, emit `info` citing the file.
|
||||
|
||||
Set `confidence` to:
|
||||
|
||||
- `high` when the detection is based on an unambiguous pattern match (attribute, handler declaration, assertion sequence, or fixture call).
|
||||
- `medium` when detection relies on heuristics or when any frontmatter dimension was `unknown`.
|
||||
- `low` when the finding is an advisory derived only from applicability.
|
||||
|
||||
After evaluating each worklist entry, also consider whether the diff exhibits a testing defect the agent recognises from its general AL knowledge that no knowledge file in the worklist covers. Such candidates are agent findings within this skill's domain — emit them with `references: []`, an `id` slug prefixed with `agent:`, `confidence` capped at `medium`, `severity` capped at `minor` (agent findings are advisory and non-gating), and a `message` that is self-contained (describing both the issue and a concrete recommendation, since there is no knowledge-file footer for the consumer to fall back on). Hold every candidate to the precision bar in `skills/do.md` (*Agent findings*): emit only a concrete, material testing defect a knowledgeable BC reviewer would agree is wrong — steelman it first and drop anything stylistic, speculative, dependent on code outside the diff, or merely a valid alternative; when in doubt, omit. The scope is strictly AL testing; defects outside this domain belong to other leaves and MUST NOT be emitted here. Before emitting, check the worklist for a knowledge file that matches the candidate — if one exists, upgrade the candidate to a knowledge-backed finding instead. See `skills/do.md` for the full contract.
|
||||
|
||||
For every emitted finding, decide whether the fix is mechanical. A fix is mechanical when it is small, local, and unambiguous from the diff context (for example: add the matching `ExpectedError` assertion after `asserterror`; add or remove a handler name in `HandlerFunctions`; add `LibraryVariableStorage.Clear` or `AssertEmpty`; or replace hand-rolled fixture creation with an evident library call). For mechanical findings, emit `findings[].suggested-code` with the literal replacement for the source lines indicated by `location`. The payload must be a verbatim replacement — no diff markers, no fences, no commentary — that the consumer can render as a one-click suggestion. When a `.good.al` companion exists and the diff context matches the `.bad.al` shape, adapt the `.good.al` replacement into `suggested-code`.
|
||||
|
||||
Omit `suggested-code` only when the appropriate fix depends on context the skill cannot determine, when multiple defensible replacements exist, or when the fix spans non-contiguous code. If a finding is mechanical-looking but you omit `suggested-code`, set `findings[].suggested-code-omission-reason` to a short explanation. See `skills/do.md` for the full contract.
|
||||
|
||||
Outcome selection:
|
||||
|
||||
- `completed` — the skill evaluated every worklist item.
|
||||
- `no-knowledge` — no applicable testing knowledge survived filtering.
|
||||
- `not-applicable` — the diff touches no test codeunit, runner, method, handler, assertion, or fixture surface.
|
||||
- `partial` — a budget was hit before the worklist was exhausted.
|
||||
- `failed` — an unrecoverable error occurred.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-testing-review", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 1, "items-evaluated": 1 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "microsoft/knowledge/testing/asserterror-needs-expectederror-and-code.md",
|
||||
"severity": "major",
|
||||
"message": "The negative test uses asserterror without checking the resulting message or error code, so any unrelated setup or permission error can make the test pass.",
|
||||
"location": {
|
||||
"file": "test/SalesPostingTests.Codeunit.al",
|
||||
"line": 42
|
||||
},
|
||||
"references": [
|
||||
{ "path": "microsoft/knowledge/testing/asserterror-needs-expectederror-and-code.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"suggested-code": "asserterror PostInvalidOrder();\nAssert.ExpectedError(ExpectedPostingErr);"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
|
||||
The empty-corpus case produces:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-testing-review", "version": 1 },
|
||||
"outcome": "no-knowledge",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 0, "items-evaluated": 0 }
|
||||
},
|
||||
"findings": [],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
Loading…
Add table
Add a link
Reference in a new issue