mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-08-07 18:06:53 +01:00
Add new action skills for AL testing and documentation
- Introduced `al-test-writer` to generate AL test codeunits for production objects based on TDD principles. - Added `al-userguide-test-writer` to create test codeunits from user guide steps, mapping actions and assertions. - Implemented `bc-extension-test-guide` to generate a comprehensive TEST_GUIDE.md for Business Central extensions, covering various categories. - Created `bc-webclient-runner` to automate UI testing of the Business Central web client, capturing screenshots and asserting UI states. - Developed `page-scripting-e2e` to produce a recording plan for Page Scripting, ensuring a structured approach to browser-level testing.
This commit is contained in:
parent
822cae1b27
commit
07140e2223
76 changed files with 4353 additions and 6 deletions
73
custom/skills/integration/azure-integration-review.md
Normal file
73
custom/skills/integration/azure-integration-review.md
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: azure-integration-review
|
||||
version: 1
|
||||
title: Azure integration review
|
||||
description: The integration-plane review playbook for the Azure side of a BC integration, pairing with the BC-side validator.
|
||||
inputs: [repository, pr-diff, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [bicep, csharp]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# Azure integration review
|
||||
|
||||
Reviews the Azure side of a Business Central integration as a playbook: the webhook receiver that catches a storefront event, the Logic App that routes a shipment to a WMS, the Service Bus topic that carries Business Events, the Durable Function that schedules a retry, and the Bicep, ARM, or Terraform that provisions them. The integration plane earns its keep by keeping retry, dead-letter, observability, and credential handling outside BC, so BC stays free of external credentials and third-party schema changes do not break it. This skill pairs with the BC-side `azure-integration-validator` so the inbound, outbound, long-running, and manual arrows line up end to end. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `repository`, a `pr-diff` (a change to the integration plane), or a `file-path` (a specific artifact to review). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `integration` or `security` as the citable candidate set across every enabled layer; the playbook's rules about receivers, idempotency, retry, dead-letter, correlation, observability, long-running poll, subscription health, and secret handling can match a curated file. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The Azure-side house rules are largely not covered by a BC-focused curated file, so most findings are agent findings within this skill's domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the BC version the plane integrates with, or `unknown` if unavailable.
|
||||
- `technologies`: `[bicep, csharp]` (the infrastructure-as-code and Function handler code the playbook reviews).
|
||||
- `countries`: the consuming solution's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the integration, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the artifacts present and the playbook rules each draws. If the repository has no Azure artifacts, report that plainly rather than inventing findings. Read Bicep, ARM, and Terraform that provision Functions, Logic Apps, APIM, Service Bus, and Storage; Logic App and workflow definitions; Function app config (`host.json`, `function.json`); APIM policy XML; and pipeline files. Place each artifact on the four arrows (inbound, outbound, long-running, manual) and build the worklist against the playbook rules: receiver stages to BC, receiver acknowledges fast, idempotency key forwarded, idempotent consumer, retry in the plane, dead-letter configured, transient versus permanent classification, durable retry not a tight loop, correlation header on every hop, observability wired, 202 status poll for long-running, subscription health check, secrets in Key Vault, Managed Identity, HTTPS only. Also worklist the cross-checks with the BC side: an outbound BC call's idempotency key the plane must forward, a BC Correlation ID the plane must carry, a parked long-running message the plane must drive the poll for, and Business Event subscriptions the plane must monitor.
|
||||
|
||||
A curated `integration` or `security` file enters the worklist when its `keywords` intersect these tokens. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklisted artifact and rule, evaluate the plane against the playbook. Where a curated `integration` or `security` knowledge file states the rule, emit a knowledge-backed finding citing it: `id` equal to the file path, `severity` up to `blocker` only when the file states a platform-level guarantee, otherwise `major`, `confidence` `high` for an unambiguous match. Where no curated file covers the rule (the common case for the Azure-side checks), emit an agent finding within this skill's domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:az-receiver-stages-to-bc`, `agent:az-correlation-header`, `agent:az-subscription-health-check`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` naming the artifact, what the playbook expects, what the artifact does, and the concrete fix. When the underlying impact would otherwise be a blocker (a receiver running BC business logic inline, a missing dead-letter path swallowing poison messages, a stripped correlation id breaking end-to-end tracing), keep the emitted `severity` at `minor` but say so plainly in the `message` and flag that the rule should be promoted to a curated knowledge file before it can gate. Set `suggested-code` when the fix is a mechanical edit to a contiguous artifact span; otherwise set `suggested-code-omission-reason`. Hold every agent candidate to the precision bar in `skills/do.md`: steelman that the plane's choice is deliberate before emitting, and omit when in doubt.
|
||||
|
||||
Outcome selection: `completed` when every worklisted artifact was reviewed (including a clean plane with empty `findings`); `not-applicable` when the repository contains no Azure integration artifacts (report this in `outcome-reason`); `no-knowledge` when artifacts exist but no curated knowledge survived and no agent finding was raised; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. Playbook rules with no curated backing are agent findings (`references: []`, `agent:` id mirroring the `az-*` rule, severity capped at `minor`, gating impact stated in the message); findings citing an `integration` or `security` file carry that file path as `id` and primary reference.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "azure-integration-review", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 3, "items-evaluated": 3 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:az-correlation-header",
|
||||
"severity": "minor",
|
||||
"message": "functions/ShipmentRouter reads the inbound message but does not set the Correlation ID on the outbound Service Bus header, so a trace cannot be joined across BC, the plane, and the WMS. Read the correlation id from the inbound message and set it on the Service Bus message header and every outbound HTTP header, and log it at each step. Promote to a curated rule before it can gate.",
|
||||
"location": { "file": "functions/ShipmentRouter/run.csx" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "fix spans message construction and logging, not a single contiguous span"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
97
custom/skills/integration/azure-integration-validator.md
Normal file
97
custom/skills/integration/azure-integration-validator.md
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: azure-integration-validator
|
||||
version: 1
|
||||
title: Azure integration validator
|
||||
description: Validates the Azure plane for a BC integration, checking receivers, Service Bus, Durable Functions, retry/dead-letter, idempotency, correlation, secrets, and subscription health.
|
||||
inputs: [repository, file-path, pr-diff]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [bicep, csharp]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# Azure integration validator
|
||||
|
||||
Validates the Azure component build that sits between Business Central and external systems: webhook receivers (Functions, Logic Apps, APIM), Service Bus topics and queues, Durable Functions, and the Bicep, ARM, or Terraform that provisions them. The single question is whether, when BC stages a message or fires an event, the plane delivers it reliably, traceably, and exactly once. It reads the artifacts and reports where the plane fails its half of the contract; the developer chooses which fixes to apply. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `repository`, a `file-path` (a narrow scope such as the storefront webhook Function), or a `pr-diff` (a change touching Azure integration artifacts). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `integration` or `security` as the citable candidate set across every enabled layer: receiver-staging, idempotency, retry-and-dead-letter, correlation, observability, secret-handling, and managed-identity rules can back a finding. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The integration-plane house rules (the `az-*` checks below) are largely Azure-side and rarely map onto a BC-focused curated file, so most findings here are agent findings within this skill's domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version the plane integrates with, or `unknown` if unavailable.
|
||||
- `technologies`: `[bicep, csharp]` (the infrastructure-as-code and Function handler code the checks actually touch).
|
||||
- `countries`: the consuming solution's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the integration, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the Azure artifacts present and the integration-plane checks each draws. If the repository contains no Azure integration artifacts, do not invent findings: report that plainly. Read Bicep (`*.bicep`), ARM (`azuredeploy.json`, `*.template.json`), Terraform (`*.tf`), Logic App and workflow definitions (`workflow.json`, `*.logicapp.json`), Function app config (`host.json`, `function.json`, retry and binding config) and handler source, and APIM policy XML (inbound, backend, outbound, on-error). Build the worklist against these checks:
|
||||
|
||||
- Receiver stages to BC and does not run BC business logic or block on BC completion inline.
|
||||
- Receiver acknowledges fast (a 2xx, or 202 for async); no long synchronous work inside it.
|
||||
- Idempotency key forwarded: outbound calls carry the BC Message ID as `Idempotency-Key`; inbound receivers forward the source system id; the plane does not strip it.
|
||||
- Idempotent consumer: receivers and queue consumers dedup on the event id or business key before a second side effect.
|
||||
- Retry in the plane: explicit on the Logic App action, the Function `host.json`, or the Service Bus delivery count, not a silent default or a hand-written loop.
|
||||
- Dead-letter configured: Service Bus queues and subscriptions enable dead-lettering with a defined max delivery count.
|
||||
- Transient versus permanent: retries 408, 429, 5xx, and timeouts; routes 4xx and invalid data to DLQ or alert. Retrying a 4xx forever is the most severe failure.
|
||||
- Durable retry, not a tight loop: long retries scheduled by a Durable Function or a Logic App timer carrying the same idempotency key.
|
||||
- Correlation header: the Correlation ID is read from the inbound message, set on the Service Bus header and every outbound HTTP header, and logged at each step.
|
||||
- Observability wired: Functions and Logic Apps have Application Insights or equivalent.
|
||||
- 202 status poll: a long-running external process is parked and polled or callback-driven, then written back to the same Integration Message; no synchronous connection held open for hours.
|
||||
- Subscription health check: where the plane relies on BC Business Event subscriptions, a scheduled job lists them and alerts on drift, since they expire silently.
|
||||
- Secrets in Key Vault: credentials and connection strings come from Key Vault via Managed Identity, not inline. A literal secret is the most severe failure.
|
||||
- Managed Identity: plane-to-BC and plane-to-resource auth uses Managed Identity where supported.
|
||||
- HTTPS only: receivers and Function apps enforce HTTPS with a current TLS minimum; the Function is not public where APIM is the intended front door.
|
||||
|
||||
A curated `integration` or `security` file enters the worklist when its `keywords` intersect these tokens. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklisted artifact, evaluate it against the checks. Where a curated `integration` or `security` knowledge file states the rule (for example a secret-handling or idempotency rule), emit a knowledge-backed finding citing it: `id` equal to the file path, `severity` up to `blocker` only when the file states a platform-level guarantee, otherwise `major`, `confidence` `high` for an unambiguous match. Where no curated file covers the integration-plane check (the common case), emit an agent finding within this skill's domain: `references: []`, `id` slug prefixed `agent:` mirroring the house rule (for example `agent:az-secrets-in-keyvault`, `agent:az-dead-letter-configured`, `agent:az-classify-transient-vs-permanent`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` naming the artifact and line, what is wrong, and the concrete fix. When the underlying impact would otherwise be a blocker (a literal secret checked into source, a 4xx retried forever, a receiver running BC logic inline, a stripped idempotency key), keep the emitted `severity` at `minor` but say so plainly in the `message` and flag that the check should be promoted to a curated rule before it can gate. Set `suggested-code` when the fix is a mechanical edit to a contiguous artifact span (a Key Vault reference replacing a literal, a `maxDeliveryCount` plus dead-letter setting on a subscription); otherwise set `suggested-code-omission-reason`. Hold every agent candidate to the precision bar in `skills/do.md`: steelman that the configuration is deliberate (the secret may be a non-sensitive placeholder, the retry default may be intended) before emitting, and omit when in doubt.
|
||||
|
||||
Outcome selection: `completed` when every worklisted artifact was evaluated (including a clean plane with empty `findings`); `not-applicable` when the repository contains no Azure integration artifacts (report this in `outcome-reason`); `no-knowledge` when artifacts exist but no curated knowledge survived and no agent finding was raised; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. Integration-plane checks with no curated backing are agent findings (`references: []`, `agent:` id mirroring the `az-*` rule, severity capped at `minor`, gating impact stated in the message); findings citing an `integration` or `security` file carry that file path as `id` and primary reference.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "azure-integration-validator", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 2, "info": 0 },
|
||||
"coverage": { "worklist-size": 4, "items-evaluated": 4 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:az-secrets-in-keyvault",
|
||||
"severity": "minor",
|
||||
"message": "infra/main.bicep line 142: the WMS API key is a literal string in the Function app settings, checked into source and visible in deployment history. Impact is a blocker: move the key to Key Vault and reference it via @Microsoft.KeyVault(...), granting the Function access through its Managed Identity. Promote to a curated rule before it can gate.",
|
||||
"location": { "file": "infra/main.bicep", "line": 142 },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "fix requires creating a Key Vault secret and a reference whose name is not derivable from the diff"
|
||||
},
|
||||
{
|
||||
"id": "agent:az-dead-letter-configured",
|
||||
"severity": "minor",
|
||||
"message": "infra/servicebus.bicep line 60: the shipments subscription sets neither deadLetteringOnMessageExpiration nor maxDeliveryCount, so poison messages loop or vanish. Enable dead-lettering with a defined max delivery count and add a consumer or alert on the DLQ.",
|
||||
"location": { "file": "infra/servicebus.bicep", "line": 60 },
|
||||
"references": [],
|
||||
"confidence": "medium"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
90
custom/skills/review/al-appsource-validator.md
Normal file
90
custom/skills/review/al-appsource-validator.md
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-appsource-validator
|
||||
version: 1
|
||||
title: AL AppSource submission validation
|
||||
description: Audits an AL extension against AppSourceCop rules, app.json metadata, artefacts, links, and the dependency chain, and emits a findings report.
|
||||
inputs: [repository, object-list]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL AppSource submission validation
|
||||
|
||||
Audits a Business Central extension against the rules Microsoft's AppSource validation applies, plus the soft conventions that surface in the manual review pass, so the developer fixes gates locally rather than after a multi-day Microsoft review cycle. It reports each finding against the `AS0xxx` rule that will flag it. Coverage spans `app.json` metadata completeness, object suffix discipline, object id ranges, permission-set coverage, prohibited objects, translations, logo and screenshots, EULA and privacy and help and url links, the dependency chain, runtime-versus-target-versus-application alignment, demo and dev artefacts, telemetry consent, id collisions, and the marketplace listing checklist folded in from the AppSource validation playbook. It sources from the `security` and `style` knowledge domains and cites curated rules where present; the AppSource-specific gates the corpus does not encode are agent findings within its AppSource compliance domain. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `repository` or an `object-list`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `security` or `style` as the citable candidate set across every enabled layer: permission-set minimal-grant and wildcard guidance, captions and tooltips required on page fields, and label discipline each map onto a curated rule and MUST cite it rather than be paraphrased. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The AppSource gates themselves (a `AS0xxx` rule violation, a dead link, a missing logo, a runtime mismatch, the listing-metadata checklist) are not encoded in the corpus; for those concrete defects, emit an agent finding within this skill's AppSource compliance domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries (the `supportedCountries`), or `unknown`.
|
||||
- `application-area`: the application areas of the extension's objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the submission gates for the extension under review:
|
||||
|
||||
- `app.json` metadata: `id` a stable GUID, `name`/`publisher`/`version` matching the listing, `brief` (empty is AS0036) and `description`, `privacyStatement`, `EULA`, `help`, `url`, `logo`, `runtime`, `target` (`Cloud` for AppSource), `application`, `platform`, all set and not the AL scaffold default; `showMyCode` set only when intentional.
|
||||
- Object suffix discipline against the `AppSourceCop.json` `mandatorySuffix` (AS0040/AS0041), object ids inside `idRanges` (AS0072), no objects in the system range, no use of Microsoft `Access = Internal` platform objects.
|
||||
- Permission-set coverage (the AS0029-class tabledata gap), and `supportedCountries` each having an xliff (AS0091).
|
||||
- Logo PNG at least 350 by 350 and square; at least one screenshot present per the manifest; EULA, privacy, help, and url links resolving with a 2xx HEAD response.
|
||||
- Dependencies each with `id`/`name`/`publisher`/`version`, version either `0.0.0.0` or a real published version, `propagateDependencies` set when downstream consumers need access (AS0078/AS0079); `runtime` aligned with `target` and `application`; object ids not colliding with the platform or other dependency-chain extensions.
|
||||
- No demo or dev artefacts in src (`RunModal` in startup paths, hardcoded passwords, demo `Confirm` boxes, `Sleep` in production codeunits); telemetry consent stated in the privacy statement when `applicationInsightsConnectionString` is set.
|
||||
- Marketplace listing checklist folded in from the AppSource validation playbook: search summary 100 characters or under, description leading with the value proposition, signing via the Key Vault pipeline, README/SETUP/SUPPORT files, support email pointing at the team inbox rather than a personal address, privacy and terms URLs live.
|
||||
|
||||
A curated `security` or `style` file enters the worklist when its `keywords` intersect these tokens (for example `permission-set`, `caption`, `tooltip`, `label`). Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each gate, emit a finding.
|
||||
|
||||
When the gate maps onto a curated `security` or `style` rule (an over-broad permission grant, a page field missing a `Caption` or `ToolTip`), emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `severity` up to `blocker` only when the file states a platform-level guarantee otherwise `major`, `confidence` `high` for an unambiguous match.
|
||||
|
||||
When the gate is an AppSource-specific defect with no curated rule, emit an agent finding within this skill's AppSource compliance domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:as0036-empty-brief` or `agent:runtime-target-mismatch`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` naming the `AS0xxx` rule or listing requirement and the concrete fix. Where the impact would normally gate (any hard AppSource rejection), keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the field is intentionally set as-is before emitting, and omit when in doubt. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is a single contiguous metadata edit (setting a `brief` value, correcting a `runtime` number); otherwise set `suggested-code-omission-reason` (for example `requires creating a logo asset` or `requires a live privacy-policy URL`).
|
||||
|
||||
Outcome selection: `completed` when every gate was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the task has no extension manifest to validate; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-appsource-validator", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 14, "items-evaluated": 14 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:as0036-empty-brief",
|
||||
"severity": "minor",
|
||||
"message": "app.json brief is empty. AppSource validation rejects an empty brief under AS0036. Set brief to a one-sentence summary of 100 characters or fewer. Impact would normally be a blocker because it is a hard AppSource rejection; emitted as minor because no curated rule backs it. This concern should be promoted to a knowledge-backed rule before it can gate.",
|
||||
"location": {
|
||||
"file": "app.json",
|
||||
"line": 9
|
||||
},
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": " \"brief\": \"Stage-and-forward integration for warehouse shipments.\","
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
102
custom/skills/review/al-code-quality-reviewer.md
Normal file
102
custom/skills/review/al-code-quality-reviewer.md
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-code-quality-reviewer
|
||||
version: 1
|
||||
title: AL code quality review
|
||||
description: Reviews AL source changes for design quality, testability, and structural anti-patterns, and emits a findings report.
|
||||
inputs: [pr-diff, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL code quality review
|
||||
|
||||
Reviews Business Central production AL for design and structural problems that hurt the codebase later: direct database access from logic codeunits (the IDataAccess rule), business logic on table or page triggers, untestable seams, swallowed errors, excessive coupling and nesting, and thick event subscribers. This skill is about whether the code is well-designed and testable, not about clarity to a fresh reader (that is `al-readability-checker`) or test quality (that is the test validators). Most of what it surfaces is design judgement the curated corpus does not encode, so the bulk of its output is agent findings within a design quality remit, with curated `performance` and `security` rules cited where a structural defect maps onto one. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff` (the standard PR-review entry point) or a `file-path` (single-file review). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `performance` or `security` as the citable candidate set across every enabled layer: a structural defect such as a commit inside a loop, a redundant Get, or an integration event that leaks a secret maps onto a curated rule and MUST cite it rather than be paraphrased. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The design quality concerns this skill owns (IDataAccess routing, testability seams, coupling, error-handling robustness, subscriber discipline) are mostly not covered by the corpus; for a concrete, demonstrable defect there, emit an agent finding within this skill's design quality domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the changed production AL (exclude test objects) and the structural shapes this skill audits:
|
||||
|
||||
- Logic codeunits that call `Get`, `Find*`, `SetRange`, `Insert`, `Modify`, `Delete` directly on a record rather than routing through the project's `IDataAccess` interface or its implementation.
|
||||
- Table and page objects whose triggers carry non-trivial validation, calculation, or posting logic instead of delegating to a management codeunit.
|
||||
- Public procedures whose `Record` parameter cannot be exercised with a temporary record, and procedures that read ambient state (`UserId`, `WorkDate`, `CompanyName`, `Session`) with no override seam.
|
||||
- Procedures with high cyclomatic complexity, length over roughly 80 lines, fan-out over ten codeunits, or nesting at five levels or deeper.
|
||||
- Swallowed errors (`if not Codeunit.Run() then exit` with no handling), empty `Error('')`, and `Commit` inside a loop or without a documented reason.
|
||||
- Event subscriber codeunits that mix unrelated subscriptions, hold inline business logic, omit early exit on temporary or wrong record type, or leave `EventSubscriberInstance` unset on a non-trivial subscriber.
|
||||
|
||||
A curated `performance` or `security` file enters the worklist when its `keywords` intersect these tokens (for example `commit`, `loop`, `get`, `integrationevent`, `secret`). Read its full `## Best Practice` / `## Anti Pattern` body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklisted shape, evaluate the diff and emit findings.
|
||||
|
||||
When a defect maps onto a curated `performance` or `security` knowledge file (for example a `Commit` inside an iteration, a redundant `Get` on an already-loaded record, or an integration event exposing a secret), emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `severity` up to `blocker` only when the file states a platform-level guarantee otherwise `major`, `confidence` `high` for an unambiguous pattern match.
|
||||
|
||||
When a concrete, demonstrable design quality defect has no curated rule (a logic codeunit reaching the database directly instead of through IDataAccess, a public procedure with no test seam, a swallowed error, a thick subscriber doing inline business logic), emit an agent finding within this skill's design quality domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:direct-db-access-from-logic`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` describing both the defect and a concrete fix. Where the underlying impact would normally gate (a direct-DB-access violation the team treats as a block), keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every agent candidate to the precision bar in `skills/do.md`: steelman that the shape is a deliberate, valid choice before emitting, never emit stylistic or speculative concerns, and omit when in doubt. Defects outside design quality (pure readability, pure performance the corpus already covers) belong to other skills and MUST NOT be emitted here. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is mechanical (deleting a swallowed-error guard, moving a `Commit` out of a loop, replacing `Error('')` with a Label-backed call); otherwise set `suggested-code-omission-reason` (for example `requires introducing an IDataAccess implementation`). Group repeated instances of one concern into a single finding with a line range rather than many near-identical ones.
|
||||
|
||||
Outcome selection: `completed` when every worklist item was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the diff has no production AL to review; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-code-quality-reviewer", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 4, "items-evaluated": 4 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "microsoft/knowledge/performance/avoid-commit-inside-loops.md",
|
||||
"severity": "major",
|
||||
"message": "Commit() is called inside a repeat..until loop in EventPostingMgt. Move the commit outside the loop or split the work so the transaction boundary is not broken per row.",
|
||||
"location": {
|
||||
"file": "src/EventPostingMgt.Codeunit.al",
|
||||
"line": 88
|
||||
},
|
||||
"references": [
|
||||
{ "path": "microsoft/knowledge/performance/avoid-commit-inside-loops.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"id": "agent:direct-db-access-from-logic",
|
||||
"severity": "minor",
|
||||
"message": "EventRegistrationMgt.ReleaseRegistration calls Record.Get on Event Registration directly from a logic codeunit, bypassing the project's IDataAccess seam. This couples business logic to the data layer and blocks unit testing with a temporary record. Route the read through the IDataAccess implementation. Impact would normally be major in this codebase; emitted as minor because no curated rule backs it. This concern should be promoted to a knowledge-backed rule before it can gate.",
|
||||
"location": {
|
||||
"file": "src/EventRegistrationMgt.Codeunit.al",
|
||||
"line": 42
|
||||
},
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "requires routing through the project's IDataAccess implementation"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
87
custom/skills/review/al-event-subscriber-auditor.md
Normal file
87
custom/skills/review/al-event-subscriber-auditor.md
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-event-subscriber-auditor
|
||||
version: 1
|
||||
title: AL event subscriber audit
|
||||
description: Audits AL event subscribers for publisher existence, signature match, IsHandled contract, and thin-handler discipline, and emits a findings report.
|
||||
inputs: [pr-diff, file-path, repository]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL event subscriber audit
|
||||
|
||||
Verifies that every `[EventSubscriber]` in a Business Central extension is wired correctly: the targeted publisher still exists, the signature matches the publisher parameter for parameter, the `IsHandled` contract is honoured, and the handler is thin enough to belong in a subscriber. The failure mode is silent: a typo'd publisher name or a `var` mismatch never fires and never errors at compile time on older event shapes. This skill sources from the `style`, `performance`, and `security` domains and cites curated rules where a subscriber concern maps onto one (subscriber parameter naming, guarding subscribers before a database call, integration events that leak secrets); the binding-correctness checks are mostly agent findings within its event-subscriber domain. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff`, a `file-path`, or a `repository`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `style`, `performance`, or `security` as the citable candidate set across every enabled layer: subscriber parameter names that must match the publisher, guarding an event subscriber before a database call, and an integration event that must not expose secrets each map onto a curated rule and MUST cite it rather than be paraphrased. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The binding-correctness checks (publisher existence, exact signature match, the `IsHandled` flow, `BindSubscription` for `Manual` instances) are mostly not encoded in the corpus; for those concrete defects, emit an agent finding within this skill's event-subscriber domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the procedures decorated with `[EventSubscriber]` and their immediate radius:
|
||||
|
||||
- The targeted publisher for each subscriber, resolved from `.alpackages` symbols when the publisher is in a dependency or from the extension's own source when internal. Flag a publisher that does not resolve (typo or removed event), a signature that does not match the publisher parameter for parameter (name, type, var-ness, order), and a missing `var` on a parameter the publisher passes by var.
|
||||
- The `IsHandled` flow: a subscriber that ignores `var IsHandled: Boolean` on a first-handler-wins event, or sets it true without honouring the contract; a subscriber to `OnBeforeValidateEvent` that sets `IsHandled := true` without replicating the base-app validation.
|
||||
- `OnRun` subscriptions (almost always a mistake), missing `Element` on a control-event binding, and `EventSubscriberInstance = Manual` subscribers with no `BindSubscription` in the call path.
|
||||
- Handler thickness: subscriber bodies with more than roughly 25 lines of inline business logic that belong in a delegated codeunit; subscribers that call `Commit`, `Confirm`, or `HttpClient` inside a posting hot path.
|
||||
|
||||
A curated `style`, `performance`, or `security` file enters the worklist when its `keywords` intersect these tokens (for example `event-subscriber`, `publisher`, `guard`, `integrationevent`, `secret`). Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklisted subscriber, emit findings.
|
||||
|
||||
When a defect maps onto a curated file (subscriber parameter names that diverge from the publisher, a subscriber that hits the database with no early guard, an integration event payload that leaks a secret), emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `severity` up to `blocker` only when the file states a platform-level guarantee otherwise `major`, `confidence` `high` for an unambiguous match.
|
||||
|
||||
When a concrete binding-correctness defect has no curated rule (a publisher that does not resolve, a `var` mismatch that silently drops mutations, a broken `IsHandled` flow, a `Manual` subscriber nobody binds, a thick handler with inline business logic), emit an agent finding within this skill's event-subscriber domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:subscriber-signature-drift`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` describing why the subscriber will not fire or will misbehave at runtime and the concrete fix. Where the impact would normally gate (a signature drift that silently breaks `IsHandled`), keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the loose signature or unbound manual instance is intentional and resolved by code outside the diff before emitting, and omit when in doubt. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is mechanical (adding a missing `var` to a parameter, adding an early-exit guard); otherwise set `suggested-code-omission-reason` (for example `requires the publisher's exact signature from the dependency symbols`).
|
||||
|
||||
Outcome selection: `completed` when every subscriber was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the task has no event subscribers to audit; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-event-subscriber-auditor", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 12, "items-evaluated": 12 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:subscriber-signature-drift",
|
||||
"severity": "minor",
|
||||
"message": "Posting Subscribers.HandleAfterFinalize binds to Sales-Post.OnAfterFinalizePosting but omits the var prefix on RecRef. The publisher passes RecRef by var, so without var the subscriber receives a copy and any mutation is dropped, breaking the IsHandled flow silently. Change RecRef: RecordRef to var RecRef: RecordRef to match the publisher. Impact would normally be major; emitted as minor because no curated rule backs it. This concern should be promoted to a knowledge-backed rule before it can gate.",
|
||||
"location": {
|
||||
"file": "src/PostingSubscribers.Codeunit.al",
|
||||
"line": 18
|
||||
},
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "requires the publisher's exact parameter list from the base app symbols"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
102
custom/skills/review/al-extended-review.md
Normal file
102
custom/skills/review/al-extended-review.md
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-extended-review
|
||||
version: 1
|
||||
title: AL extended review
|
||||
description: Composes the custom-layer AL review leaves (multi-tenancy, permissions, events, obsolescence, integration, upgrade) that complement the platform al-code-review.
|
||||
inputs: [pr-diff, file-path, repository]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
sub-skills:
|
||||
- custom/skills/review/al-multitenancy-reviewer.md
|
||||
- custom/skills/review/al-permission-set-auditor.md
|
||||
- custom/skills/review/al-event-subscriber-auditor.md
|
||||
- custom/skills/review/al-obsolete-tracker.md
|
||||
- custom/skills/review/al-integration-pattern-reviewer.md
|
||||
- custom/skills/review/al-upgrade-checker.md
|
||||
---
|
||||
|
||||
# AL extended review
|
||||
|
||||
Composes the custom-layer AL review leaves that cover concerns the platform `microsoft/skills/review/al-code-review` does not: multi-tenant and cross-company safety, permission-set coverage, event-subscriber discipline, obsolescence hygiene, modern integration patterns, and upgrade-codeunit coverage. This is a super-skill: it does not evaluate knowledge files directly, it invokes its sub-skills and rolls up their findings-reports following the DO composition contract.
|
||||
|
||||
Run this alongside the platform `al-code-review` (which covers performance, security, privacy, upgrade, style, and UI) for a full review surface. An orchestrator invokes this skill with a `pr-diff`, a `file-path`, or a `repository`, and receives one JSON document conforming to the DO output contract, extended with `sub-results` and, where applicable, `skipped-sub-skills`.
|
||||
|
||||
## Source
|
||||
|
||||
The sub-skills invoked are exactly those listed in frontmatter `sub-skills`:
|
||||
|
||||
- `custom/skills/review/al-multitenancy-reviewer.md`
|
||||
- `custom/skills/review/al-permission-set-auditor.md`
|
||||
- `custom/skills/review/al-event-subscriber-auditor.md`
|
||||
- `custom/skills/review/al-obsolete-tracker.md`
|
||||
- `custom/skills/review/al-integration-pattern-reviewer.md`
|
||||
- `custom/skills/review/al-upgrade-checker.md`
|
||||
|
||||
Additional leaves are added by editing this list; the skill does not discover sub-skills implicitly. Composition is flat: every entry is a leaf skill, never another super-skill.
|
||||
|
||||
## Relevance
|
||||
|
||||
A sub-skill is relevant when the orchestrator has supplied inputs that satisfy the sub-skill's declared `inputs` and has not disabled it via configuration. Per the DO contract, this super-skill MUST NOT filter sub-skills by task content (it does not inspect the diff to guess whether a leaf will find anything). Each leaf decides its own task-level applicability and signals it by returning `outcome: "not-applicable"` or `outcome: "no-knowledge"`. Sub-skills failing the input or configuration check are not invoked and are recorded in `skipped-sub-skills` with `reason: "not-applicable"` or `reason: "configuration"`.
|
||||
|
||||
## Worklist
|
||||
|
||||
The worklist is the set of sub-skills judged relevant by the previous step. Every sub-skill in the worklist is invoked in the Action step; the rest go to `skipped-sub-skills`.
|
||||
|
||||
## Action
|
||||
|
||||
Invoke each worklisted sub-skill as its own discrete pass, one at a time, passing only the subset of inputs the sub-skill declares. Capture each sub-skill's complete findings-report verbatim into `sub-results`. For any sub-skill whose `outcome` is `failed`, do not copy its findings into the top-level `findings[]` or counts. Otherwise append each of its findings to the top-level `findings[]` with `from-sub-skill` set to the sub-skill's `skill.id`, prefixing slug `id` values (non-citation findings) with `<from-sub-skill>:` to avoid collisions; citation-based findings keyed by repo-relative path are left unchanged.
|
||||
|
||||
After every sub-skill has produced its sub-result, perform a super-skill self-review pass for cross-cutting concerns that no single leaf could surface (for example an obsolescence change that is also a cross-tenant data path, or an integration change that is also an upgrade-schema change). Validate each candidate against the knowledge the leaves already loaded: a matching knowledge file upgrades it to a knowledge-backed finding, a contradicting file suppresses it, otherwise emit it as a super-skill agent finding (`from-sub-skill: "agent"`, `references: []`, `id` slug prefixed `agent:`, `confidence` capped at `medium`, `severity` capped at `minor`, self-contained `message`). Hold every candidate to the precision bar in `skills/do.md`. Set `suggested-code` for mechanical fixes, otherwise `suggested-code-omission-reason`.
|
||||
|
||||
Derive `outcome` using the DO rollup rules, aggregate `summary.counts` and `summary.coverage` across invoked non-failed sub-skills, and populate `outcome-reason` for `partial` and `failed`. The top-level `suppressed[]` stays empty; knowledge-file suppression is reported by each leaf inside its own `sub-results` entry.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract, extended with `sub-results` (one complete findings-report per invoked sub-skill) and `skipped-sub-skills`. A representative shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-extended-review", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 2, "info": 0 },
|
||||
"coverage": { "worklist-size": 6, "items-evaluated": 6 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "custom/knowledge/integration/never-call-external-services-from-posting.md",
|
||||
"severity": "major",
|
||||
"message": "HttpClient.Send is called from an OnAfterPostSalesDoc subscriber. Posting locks are held; stage the call on the Integration Message and let the Job Queue send it.",
|
||||
"location": { "file": "src/Integration/PostHooks.Codeunit.al", "line": 42 },
|
||||
"references": [
|
||||
{ "path": "custom/knowledge/integration/never-call-external-services-from-posting.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"from-sub-skill": "al-integration-pattern-reviewer"
|
||||
},
|
||||
{
|
||||
"id": "al-permission-set-auditor:missing-object-in-permission-set",
|
||||
"severity": "minor",
|
||||
"message": "Table 50123 \"Shipment Buffer\" is defined by the extension but does not appear in any permission set. Add it before AppSource submission.",
|
||||
"location": { "file": "src/Shipment/ShipmentBuffer.Table.al", "line": 1 },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"from-sub-skill": "al-permission-set-auditor"
|
||||
}
|
||||
],
|
||||
"suppressed": [],
|
||||
"sub-results": [
|
||||
{
|
||||
"skill": { "id": "al-integration-pattern-reviewer", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": { "counts": { "blocker": 0, "major": 1, "minor": 0, "info": 0 }, "coverage": { "worklist-size": 3, "items-evaluated": 3 } },
|
||||
"findings": [],
|
||||
"suppressed": []
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
104
custom/skills/review/al-integration-pattern-reviewer.md
Normal file
104
custom/skills/review/al-integration-pattern-reviewer.md
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-integration-pattern-reviewer
|
||||
version: 1
|
||||
title: AL integration pattern review
|
||||
description: Validates AL inbound, outbound, long-running, and manual integration code against the modern integration patterns, and emits a findings report.
|
||||
inputs: [pr-diff, file-path, repository]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL integration pattern review
|
||||
|
||||
Validates Business Central integration code against the modern integration patterns. The single question is: when the external system is slow, down, or sends the same message twice, does this code stay correct? It audits staging via an Integration Message, inbound and outbound idempotency, polling framing records, Business Event versioning and payload safety, correlation propagation, staged pipelines, and the hard anti-patterns (HTTP calls from posting, synchronous wait loops in API handlers, inline posting from a webhook or poll handler). This skill is the executor for the `integration` knowledge domain: it cites the curated integration files by path for the concerns they cover, and supplements them with `performance` and `security` citations and agent findings. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff`, a `file-path`, or a `repository`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `integration`, `performance`, or `security` as the citable candidate set across every enabled layer. The `integration` domain is the primary source: cite `custom/knowledge/integration/stage-every-integration-message.md` for the staging requirement and decoupling, and `custom/knowledge/integration/never-call-external-services-from-posting.md` for the callout-from-posting anti-pattern, and any further integration files the index lists for idempotency, framing, business-events versioning, and correlation. The `performance` and `security` domains supply supporting citations: a commit inside a fetch or send loop, a user prompt inside a posting transaction, and an integration event that leaks a secret each map onto a curated rule. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Where a concrete integration defect has no curated rule, emit an agent finding within this skill's integration domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the integration code paths under review:
|
||||
|
||||
- Codeunits that call `HttpClient`, especially any reachable from a posting routine or a posting event subscriber (`OnAfterPostSalesDoc`, `OnAfterFinalizePosting`, and similar).
|
||||
- `PageType = API` pages used as inbound staging endpoints, and any handler that posts or runs business logic inline rather than writing to staging.
|
||||
- Job Queue codeunits that fetch from or push to an external system, weighted toward a missing framing record (last fetch, max window, lock), an unbounded "fetch all", a missing inbound idempotency lookup on External Reference plus Type, and a missing or non-deterministic outbound `Idempotency-Key`.
|
||||
- `[BusinessEvent]` declarations: a per-version events codeunit, a versioned name, a stable minimal DTO payload (not the BC record, no secrets), and validation before firing.
|
||||
- Correlation ID set once at the entry point and carried onto every staged message, event payload, and outbound header.
|
||||
- Long-running flows: a 202 response parked Awaiting Reply with a status URL, retry count and last error stored on the message, and staged pipelines split behind an integration-stage interface dispatched from an extensible enum with no cross-stage global state.
|
||||
- Pages and actions that let a human re-run a failed message.
|
||||
|
||||
A curated `integration`, `performance`, or `security` file enters the worklist when its `keywords` intersect these tokens (for example `staging`, `posting`, `httpclient`, `idempotency`, `business-event`, `correlation`, `commit`, `secret`). Read its full `## Best Practice` / `## Anti Pattern` body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklisted code path, check it against the patterns and emit findings.
|
||||
|
||||
When a defect maps onto a curated knowledge file, emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `confidence` `high` for an unambiguous match. Severity is `blocker` only when the file states a platform-level guarantee, otherwise `major`. The hard anti-patterns cite the integration corpus directly: an `HttpClient.Send` reachable from a posting routine or posting subscriber cites `custom/knowledge/integration/never-call-external-services-from-posting.md`; a webhook or poll handler that posts or runs business logic inline rather than staging cites `custom/knowledge/integration/stage-every-integration-message.md`. A commit inside a fetch or send loop, a user prompt inside a posting transaction, or an integration event that exposes a secret cite the matching `performance` or `security` file.
|
||||
|
||||
When a concrete, demonstrable integration defect has no curated rule (a synchronous sleep-and-poll wait loop in an inbound API handler, a missing polling framing record or lock, a missing inbound idempotency lookup keyed on the source id, a missing or non-deterministic outbound idempotency key, a mutated published Business Event signature, a dropped correlation id, cross-stage global state), emit an agent finding within this skill's integration domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:missing-inbound-idempotency-check`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` describing the failure mode under a slow, down, or duplicating external system and a concrete fix. Where the impact would normally gate (a synchronous wait loop that ties up a handler), keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the path is correct as written before emitting, and omit when in doubt. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is mechanical (adding a deterministic `Idempotency-Key` header from the Integration Message GUID, moving a `Commit` out of a loop); otherwise set `suggested-code-omission-reason` (for example `requires introducing a staging table and Job Queue sender`).
|
||||
|
||||
Outcome selection: `completed` when every worklist item was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the diff has no integration code to review; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-integration-pattern-reviewer", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 3, "items-evaluated": 3 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "custom/knowledge/integration/never-call-external-services-from-posting.md",
|
||||
"severity": "major",
|
||||
"message": "HttpClient.Send is called from WMS Notifier.NotifyWMS, which runs in OnAfterPostSalesDoc. The posting transaction holds locks on the shipment while waiting on the WMS. Stage an Integration Message inside the posting hook and let the Job Queue send it.",
|
||||
"location": {
|
||||
"file": "src/WMSNotifier.Codeunit.al",
|
||||
"line": 31
|
||||
},
|
||||
"references": [
|
||||
{ "path": "custom/knowledge/integration/never-call-external-services-from-posting.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"id": "agent:missing-outbound-idempotency-key",
|
||||
"severity": "minor",
|
||||
"message": "The outbound POST to the WMS sets no Idempotency-Key header, so a retry after a timeout can create a duplicate shipment on the remote system. Set the header to the Integration Message GUID so it is identical on every retry. Impact would normally be major; emitted as minor because no curated rule backs it. This concern should be promoted to a knowledge-backed rule before it can gate.",
|
||||
"location": {
|
||||
"file": "src/WMSSender.Codeunit.al",
|
||||
"line": 64,
|
||||
"range": { "start-line": 64, "end-line": 68 }
|
||||
},
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "requires the Integration Message GUID variable in scope at the call site"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
88
custom/skills/review/al-major-release-readiness.md
Normal file
88
custom/skills/review/al-major-release-readiness.md
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-major-release-readiness
|
||||
version: 1
|
||||
title: AL major release readiness review
|
||||
description: Reviews a PR that bumps app.json application or platform versions against major-upgrade governance and the compatibility-testing gate, and emits a findings report.
|
||||
inputs: [pr-diff, repository]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL major release readiness review
|
||||
|
||||
Governs Business Central major version bumps. A NextMajor branch is expensive (parallel maintenance), so the house rule delays it as long as the current Production version still compiles and runs cleanly. This skill reviews any PR that bumps `app.json` `application` or `platform` minimum versions, checks the compatibility-testing gate (compatibility testing is a check, not a commitment, and must not change the manifest versions), and surfaces the NextMajor branch decision. It sources from the `upgrade` knowledge domain and cites curated rules where a version-bump concern maps onto one (breaking changes only on tables without data, enum values additive at the end, no external calls in an upgrade codeunit); the governance gates the corpus does not encode are agent findings within its release-governance domain. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff` (the standard entry point for a version-bump PR) or a `repository`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `upgrade` as the citable candidate set across every enabled layer: breaking changes only on tables without data, enum values additive at the end, no external calls in an upgrade codeunit, and upgrade tags instead of version checks each map onto a curated rule and MUST cite it rather than be paraphrased, because a major bump is the moment those rules bite. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The governance gates (whether all customer environments are on the latest major before a NextMajor branch is cut, whether compatibility testing changed the manifest, whether the current Production version still compiles) are not encoded in the corpus; for those concrete defects, emit an agent finding within this skill's release-governance domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the PR branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the governance signals in the PR:
|
||||
|
||||
- Changes to `app.json` `application` or `platform` minimum versions: whether the bump is a genuine commitment or compatibility testing that should not have touched the manifest.
|
||||
- Whether the change could instead be made on the current Production version (the rule is to refactor on the current version first and branch NextMajor only when the change cannot work on the older version).
|
||||
- The driving reason for a NextMajor branch when one is implied: a new API surface only on NextMajor, a Microsoft-required schema change, or a performance feature needed for a customer SLA; documented in the PR description.
|
||||
- Schema and enum changes riding along with the bump that the `upgrade` corpus governs: a breaking change on a table that already holds data, an enum value inserted in the middle, an external call added to an upgrade codeunit.
|
||||
- Deprecation-warning handling: warnings resolvable on the current version should be fixed forward, not deferred into a manifest bump.
|
||||
|
||||
A curated `upgrade` file enters the worklist when its `keywords` intersect these tokens (for example `breaking-change`, `enum`, `upgrade`, `version`). Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each signal, emit a finding.
|
||||
|
||||
When a defect maps onto a curated `upgrade` rule (a breaking change on a table with data, a non-additive enum change, an external call in an upgrade codeunit), emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `severity` up to `blocker` only when the file states a platform-level guarantee otherwise `major`, `confidence` `high` for an unambiguous match.
|
||||
|
||||
When a governance defect has no curated rule (a manifest version bump made for compatibility testing rather than commitment, a change that could have stayed on the current version, a NextMajor branch implied with no documented driving feature, a deprecation warning deferred into a bump that was resolvable forward), emit an agent finding within this skill's release-governance domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:compatibility-test-changed-manifest`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` describing the governance gap and the concrete remedy (revert the manifest bump and keep compatibility testing as a check, document the driving feature, fix the warning forward). Where the impact would normally gate (premature NextMajor commitment that strands customers on older majors), keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the bump is a deliberate, communicated decision before emitting, and omit when in doubt. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is a single contiguous manifest revert (restoring the prior `application` or `platform` value); otherwise set `suggested-code-omission-reason` (for example `requires a PR-description note documenting the driving feature`).
|
||||
|
||||
Outcome selection: `completed` when every signal was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the PR does not bump the application or platform version; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-major-release-readiness", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 2, "items-evaluated": 2 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:compatibility-test-changed-manifest",
|
||||
"severity": "minor",
|
||||
"message": "This PR bumps app.json platform from 26.0 to 27.0 but the description frames it as compatibility testing for the next major. Compatibility testing is a check, not a commitment, and must not change the manifest versions, otherwise the build commits every tenant to the new platform. Revert the platform value and run compatibility testing against a Sandbox-NextMajor environment instead. This concern should be promoted to a knowledge-backed rule before it can gate.",
|
||||
"location": {
|
||||
"file": "app.json",
|
||||
"line": 22
|
||||
},
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": " \"platform\": \"26.0.0.0\","
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
58
custom/skills/review/al-multitenancy-reviewer.md
Normal file
58
custom/skills/review/al-multitenancy-reviewer.md
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-multitenancy-reviewer
|
||||
version: 1
|
||||
title: AL multi-tenancy review
|
||||
description: Audits AL data paths for cross-tenant and cross-company leak risk in SaaS, and emits a findings report.
|
||||
inputs: [pr-diff, file-path, repository]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL multi-tenancy review
|
||||
|
||||
Audits Business Central AL (and any companion API in the same repository) for code paths that resolve data without correctly scoping by tenant and company. The failure mode is silent: a query that returns another tenant's data, an endpoint that resolves a record by primary key without scoping the caller's company, or a Job Queue codeunit that processes rows from the wrong company. These defects do not surface in single-tenant dev sandboxes; they appear in SaaS as a cross-tenant data leak. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff`, a `file-path`, or a `repository`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `security` or `integration` as the citable candidate set across every enabled layer; multi-tenant scoping is a security-and-integration concern, so a finding that matches a curated rule (for example, permission scoping or data-classification guidance) cites that file. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Where no curated rule covers a concrete tenant-scoping defect, this skill emits an agent finding within its own domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the code paths where tenant or company scope is established or relied upon:
|
||||
|
||||
- Codeunits runnable as a Job Queue entry (`TableNo = "Job Queue Entry"`) and any `[ServiceEnabled]` web-service codeunit.
|
||||
- Procedures that call `CompanyName()`, `Company.Get`, set a `Company` filter, or resolve a record by `SystemId` across companies.
|
||||
- Reads or writes to tables classified `CustomerContent` or `OrganizationIdentifiableInformation`, especially without a company filter.
|
||||
- Outbound `HttpClient` calls to a companion API and any sibling API endpoints (`Endpoints/*.cs`, `routes/*.ts`) in the same repo.
|
||||
- `Session.LogMessage` / Application Insights calls that omit a tenant custom dimension.
|
||||
|
||||
A curated knowledge file enters the worklist when its `keywords` intersect these tokens. Read its full `## Best Practice` / `## Anti Pattern` body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklisted code path, check that tenant and company scope is established before data is resolved, that production paths never fall back to a default tenant, that cross-company writeback verifies the caller's company on both sides, that a company filter is present on every `CustomerContent` query, that `SystemId` resolution is company-scoped for per-company tables, that outbound calls carry tenant context the receiver validates, that any `Session.Companies` traversal is intentional and commented, and that logs carry the tenant id.
|
||||
|
||||
When a defect matches a curated `security` or `integration` knowledge file, emit a knowledge-backed finding citing that file: `severity` up to `blocker` only when the file states a platform-level guarantee, otherwise `major`; `id` equal to the file path; `confidence` `high` for an unambiguous match. When no curated file covers a concrete, demonstrable tenant-scoping defect, emit an agent finding within this skill's domain: `references: []`, `id` slug prefixed `agent:`, `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` describing the leak path and a concrete fix (for example, "filter the pull query by the caller's company id"). Hold every agent candidate to the precision bar in `skills/do.md`: steelman that the cross-company traversal is intentional before emitting, and omit when in doubt. Set `suggested-code` when the fix is mechanical (adding a missing `SetRange(Company, ...)` or a company-id query parameter); otherwise set `suggested-code-omission-reason`.
|
||||
|
||||
Outcome selection: `completed` when every worklist item was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the task has no AL data path to audit; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. Findings without a knowledge file are agent findings (`references: []`, `agent:` id, severity capped at `minor`); findings citing a `security` or `integration` file carry that file path as `id` and primary reference.
|
||||
89
custom/skills/review/al-obsolete-tracker.md
Normal file
89
custom/skills/review/al-obsolete-tracker.md
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-obsolete-tracker
|
||||
version: 1
|
||||
title: AL obsolete marking audit
|
||||
description: Audits Obsolete markings on AL objects for reason, tag, removal plan, and progress, and emits a findings report.
|
||||
inputs: [object-list, repository, pr-diff]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL obsolete marking audit
|
||||
|
||||
Walks the AL source for every `ObsoleteState` marking and reports whether each is well-formed and progressing toward removal. Every object, field, procedure, or enum value marked `ObsoleteState = Pending` should carry a clear `ObsoleteReason`, an `ObsoleteTag` encoding a target removal version, and a planned removal path. The skill flags orphans (Pending with no plan), broken removals (Removed with no prior Pending cycle), and abandoned obsolescence (Pending for several majors with no progress). It sources from the `upgrade` knowledge domain and cites curated obsoletion guidance where present; the hygiene checks the corpus does not encode are agent findings within its obsolescence domain. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with an `object-list`, a `repository`, or a `pr-diff`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `upgrade` as the citable candidate set across every enabled layer: obsoletion requires a reason and a tag, an enum value made obsolete must keep its ordinal with the new value appended, and the Pending-to-Removed staging cycle each map onto a curated rule and MUST cite it rather than be paraphrased. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The progress and orphan checks (Pending extending beyond two majors, a Pending symbol with live internal callers, removal versions that do not converge) are mostly not encoded; for those concrete defects, emit an agent finding within this skill's obsolescence domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the obsolescence markings under review:
|
||||
|
||||
- Every `ObsoleteState` property on tables, table extensions, fields, codeunits, procedures, pages, page extensions, enums, enum values, reports, queries, and xmlports in the supplied `object-list` or `repository`. When a `pr-diff` is the input, narrow to markings the diff adds or changes.
|
||||
- `Pending` markings missing an `ObsoleteReason`, missing an `ObsoleteTag` with a removal version, or carrying a generic reason such as "Deprecated".
|
||||
- `Removed` markings with no prior `Pending` cycle (read git history where available).
|
||||
- `Pending` markings that have extended beyond roughly two majors, `Pending` symbols still called from inside the extension, `Pending` procedures on public codeunits whose reason names no replacement, obsoleted enum values whose ordinal moved, and removal versions across the extension that do not converge on a single harvest target.
|
||||
|
||||
A curated `upgrade` file enters the worklist when its `keywords` intersect these tokens (for example `obsolete`, `obsoletion`, `enum`, `staging`, `removal`). Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each marking, emit a finding.
|
||||
|
||||
When a defect maps onto a curated `upgrade` rule (a Pending with no reason or tag, an obsoleted enum value whose ordinal changed, a Removed that skipped Pending), emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `severity` up to `blocker` only when the file states a platform-level guarantee otherwise `major`, `confidence` `high` for an unambiguous match.
|
||||
|
||||
When a concrete hygiene defect has no curated rule (Pending abandoned for several majors, a Pending symbol with live internal callers, a Pending procedure whose reason names no migration path, removal versions that do not converge), emit an agent finding within this skill's obsolescence domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:obsolete-pending-with-live-callers`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` describing the hygiene gap and a concrete fix (migrate the internal callers in this PR, commit to a removal version). Where the impact would normally gate, keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the long-lived Pending is a deliberate, documented deferral before emitting, and omit when in doubt. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is mechanical (adding an `ObsoleteTag = '2.0.0';` line next to an existing `ObsoleteReason`); otherwise set `suggested-code-omission-reason` (for example `requires migrating internal callers across multiple files`).
|
||||
|
||||
Outcome selection: `completed` when every marking was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the task has no obsolescence markings to audit; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-obsolete-tracker", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 8, "items-evaluated": 8 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "microsoft/knowledge/upgrade/obsoletion-requires-reason-and-tag.md",
|
||||
"severity": "major",
|
||||
"message": "Field 20 Old Reference No. on table Event Log has ObsoleteState = Pending and an ObsoleteReason but no ObsoleteTag, so no removal version is planned. Add an ObsoleteTag naming the next major in which the field is removed.",
|
||||
"location": {
|
||||
"file": "src/EventLog.Table.al",
|
||||
"line": 54
|
||||
},
|
||||
"references": [
|
||||
{ "path": "microsoft/knowledge/upgrade/obsoletion-requires-reason-and-tag.md" }
|
||||
],
|
||||
"confidence": "high",
|
||||
"suggested-code": " ObsoleteTag = '2.0.0';"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
100
custom/skills/review/al-performance-reviewer.md
Normal file
100
custom/skills/review/al-performance-reviewer.md
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-performance-reviewer
|
||||
version: 1
|
||||
title: AL performance anti-pattern review
|
||||
description: Reviews AL for N+1 queries, missing keys, FlowField overuse on lists, missing SetLoadFields, and FindFirst on growing tables, and emits a findings report.
|
||||
inputs: [pr-diff, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL performance anti-pattern review
|
||||
|
||||
Reads AL with a single question: at ten times tenant scale, where does this fall over? It targets the patterns the AL compiler will not warn about but that compound badly under data scale: N+1 per-row sibling lookups and per-row `CalcFields`, a `SetCurrentKey` with no matching key, `FindFirst` on tables that grow without bound (ledger entries, document headers), FlowField columns on list-page repeaters, missing `SetLoadFields`, destructive `DeleteAll`/`ModifyAll` with no filter, heavy `OnAfterGetRecord`, HTTP calls inside a loop, and `Commit` inside iterations. This skill sources from the `performance` knowledge domain and cites those files where they match. It overlaps the platform `al-performance-review`, so its primary added value is agent findings for the scale patterns the corpus does not yet cover. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff` (the standard PR-review entry point) or a `file-path` (single-file review). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `performance` as the citable candidate set across every enabled layer: a `Get` inside a loop on a large table, `CalcSums` instead of `CalcFields` in a loop, applying filters before iterating, `SetLoadFields` for partial records, `IsEmpty` for existence checks, `Get` instead of `FindFirst` on a full primary key, and commit boundaries each map onto a curated rule and MUST cite it rather than be paraphrased. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Where a concrete scale defect has no curated rule (a FlowField column added to a heavily rendered list repeater, a `FindFirst` on a known unbounded ledger table with no tight filter, a missing `HasFilter` guard before a destructive iteration), emit an agent finding within this skill's performance domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow the relevant files to the subset that applies to the changes, computing overlap against:
|
||||
|
||||
- Record-iteration code: `FindSet`/`FindFirst`/`FindLast` and `repeat..until` loops, weighted toward a per-row `Get`/`Find` on a sibling table (N+1) and a per-row `CalcFields`.
|
||||
- List pages and their `SourceTable`, weighted toward FlowField columns on the repeater and heavy `OnAfterGetRecord` bodies.
|
||||
- Keys defined on the table compared against the `SetCurrentKey` plus `SetRange`/`SetFilter` combinations the code actually uses; a `SetCurrentKey` with no matching key.
|
||||
- `FindFirst` against tables that grow without bound (Item Ledger Entry, Value Entry, G/L Entry, Vendor Ledger Entry, Sales Header, Purchase Header) with no tight `SetRange`.
|
||||
- Reads that use only a few fields and could `SetLoadFields`; `DeleteAll`/`ModifyAll` with no prior filter or `HasFilter` guard; `HttpClient.Send` or `Commit` inside an iteration.
|
||||
|
||||
A curated `performance` file enters the worklist when its `keywords` intersect these tokens or its topic matches a changed object kind. Read its full `## Best Practice` / `## Anti Pattern` body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklist entry, evaluate the diff against the file's `## Best Practice` and `## Anti Pattern` sections. When the diff clearly matches an anti-pattern (a `Get` inside a loop on a large table, a `CalcFields` per row, a `FindFirst` on a full primary key), emit a knowledge-backed finding citing the file: `id` equal to the file path, the file as primary reference, `severity` `blocker` only when the file states a platform-level guarantee otherwise `major`, `location` on the offending line or range, `confidence` `high` for an unambiguous match. When the diff contradicts a best practice without being a full anti-pattern, emit `minor` with the same reference shape.
|
||||
|
||||
When a concrete scale defect has no curated rule, emit an agent finding within this skill's performance domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:flowfield-on-list-repeater` or `agent:findfirst-on-unbounded-table`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` that states the cost at scale (one extra query per visible row per render, a full scan on a table that only grows) and a concrete fix (move the FlowField to a factbox, add a `SetRange` that resolves via a key). Where the impact would normally gate (a destructive `DeleteAll` with no filter), keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the table is small, the filter is set elsewhere, or the cost is documented and accepted before emitting, and omit when in doubt. The scope is strictly performance; defects outside this domain belong to other skills. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is mechanical (replacing `Count() > 0` with `not IsEmpty()`, adding a `SetLoadFields` before a `FindSet`, adding a `HasFilter` guard before a `DeleteAll`); otherwise set `suggested-code-omission-reason` (for example `requires choosing the right secondary key to add`).
|
||||
|
||||
Outcome selection: `completed` when every worklist item was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the diff has no AL to review; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-performance-reviewer", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 3, "items-evaluated": 3 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "microsoft/knowledge/performance/avoid-get-inside-loop-on-large-table.md",
|
||||
"severity": "major",
|
||||
"message": "Inside a FindSet over Vendor, Data Sync Mgt.SyncVendors calls Cache.Get for each row, firing one query per vendor. Pre-load the cache table into a Dictionary outside the loop and look up per row instead.",
|
||||
"location": {
|
||||
"file": "src/DataSyncMgt.Codeunit.al",
|
||||
"line": 142,
|
||||
"range": { "start-line": 142, "end-line": 146 }
|
||||
},
|
||||
"references": [
|
||||
{ "path": "microsoft/knowledge/performance/avoid-get-inside-loop-on-large-table.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"id": "agent:flowfield-on-list-repeater",
|
||||
"severity": "minor",
|
||||
"message": "A FlowField column Total Amount was added to the Vendors list repeater, so every visible row triggers a CalcFields on each render. On a 50-row list that is 50 extra aggregate queries per render. Move the field to a factbox, which renders once per selected row, or document and accept the cost. This concern should be promoted to a knowledge-backed rule before it can gate.",
|
||||
"location": {
|
||||
"file": "src/Vendors.Page.al",
|
||||
"line": 28
|
||||
},
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "fix is a layout decision between factbox placement and accepted cost"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
86
custom/skills/review/al-permission-set-auditor.md
Normal file
86
custom/skills/review/al-permission-set-auditor.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-permission-set-auditor
|
||||
version: 1
|
||||
title: AL permission set audit
|
||||
description: Audits that every object an AL extension defines appears in a permission set with appropriate scope, and emits a findings report.
|
||||
inputs: [object-list, repository, pr-diff]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL permission set audit
|
||||
|
||||
Compares the objects a Business Central AL extension defines against the entries in its permission set files and reports every gap. This catches the silent-but-fatal class of bug where a new table ships without permission, the install succeeds in the developer SUPER sandbox, and tenants hit "Permission denied" on first use. It is also the single most common cause of AppSource rejection (the AS0029 family). This skill sources from the `security` domain and cites curated permission-set guidance where present, otherwise it emits agent findings within its security and permissioning domain. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with an `object-list`, a `repository`, or a `pr-diff`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `security` as the citable candidate set across every enabled layer, weighted toward permission-set guidance: minimal-grant, avoiding wildcard grants, indirect and inherent permissions. A scope that grants more than the usage warrants maps onto a curated rule and MUST cite it rather than be paraphrased. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The object-to-permission coverage gap itself (a defined object with no permission entry, a `table` line with no `tabledata` line) is not encoded in the corpus; for those concrete defects, emit an agent finding within this skill's security and permissioning domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Build the object-to-permission map and narrow to the gaps:
|
||||
|
||||
- Walk the supplied `object-list` (or the source under `repository`) to build the set of `(ObjectType, ObjectName)` declarations. When a `pr-diff` is the input, narrow to the objects the diff adds or renames.
|
||||
- Walk every `*.PermissionSet.al` the extension defines to build the set of granted entries.
|
||||
- For each defined object, the gap set is: no matching permission entry anywhere; a `table` entry with no corresponding `tabledata` entry; a `tabledata` scope wider than the object's actual usage; a public codeunit with callable procedures and no entry, or an `Access = Internal` codeunit with a redundant entry; an orphan entry that matches no defined object; a permission set with an empty or non-meaningful `Caption`; a permission set whose own ID falls outside the `app.json` id range.
|
||||
|
||||
A curated `security` permission-set file enters the worklist when its `keywords` intersect these tokens (for example `permission-set`, `tabledata`, `wildcard`, `minimal-grant`). Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each gap, emit a finding.
|
||||
|
||||
When the gap is an over-broad or wildcard scope that maps onto a curated `security` rule (a `tabledata` grant wider than the usage, a wildcard grant), emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `severity` up to `blocker` only when the file states a platform-level guarantee otherwise `major`, `confidence` `high` for an unambiguous match.
|
||||
|
||||
When the gap is a coverage defect with no curated rule (a defined object missing from every permission set, a `table` entry with no `tabledata` line, an orphan entry, a missing or noise `Caption`, a permission set out of the id range), emit an agent finding within this skill's security and permissioning domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:object-missing-from-permission-set` or `agent:table-without-tabledata`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` naming the object, the permission set, and the exact entry to add. Where the impact would normally gate (a new table absent from every permission set, which AppSource rejects), keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the object is intentionally not granted (a pure framework object behind `Access = Internal`) before emitting, and omit when in doubt. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is the exact permission line to add (for example `tabledata "Event Registration" = RIMD;`); otherwise set `suggested-code-omission-reason`. Group repeated instances of one concern into a single finding rather than many near-identical ones.
|
||||
|
||||
Outcome selection: `completed` when every object was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the task has no objects or permission sets to compare; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-permission-set-auditor", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 47, "items-evaluated": 47 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:table-without-tabledata",
|
||||
"severity": "minor",
|
||||
"message": "Table 50100 Event Registration is granted as a table entry in permission set MyExt All but has no tabledata entry, so the metadata is exposed but read and write are denied. Add the tabledata line beneath the existing table line. Impact would normally be major because AppSourceCop AS0029 rejects this; emitted as minor because no curated rule backs it. This concern should be promoted to a knowledge-backed rule before it can gate.",
|
||||
"location": {
|
||||
"file": "src/MyExt.PermissionSet.al",
|
||||
"line": 12
|
||||
},
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": " tabledata \"Event Registration\" = RIMD;"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
86
custom/skills/review/al-readability-checker.md
Normal file
86
custom/skills/review/al-readability-checker.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-readability-checker
|
||||
version: 1
|
||||
title: AL readability review
|
||||
description: Reviews AL source changes for readability to a fresh reader, covering naming, structure, labels, and comments, and emits a findings report.
|
||||
inputs: [pr-diff, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL readability review
|
||||
|
||||
Reviews Business Central AL for whether a human reviewer who has never seen the file can understand it without help: identifier clarity (PascalCase, verb-first procedures, no Hungarian notation, no opaque abbreviations), structure (one object per file, short procedures, shallow nesting, no magic numbers), labels and captions (every user-facing string through a `Label` with a translator `Comment`), and comments (no commented-out code, no untracked TODOs). This skill sources primarily from the `style` knowledge domain and cites a curated rule wherever a readability concern maps onto one; where no curated rule exists, it emits an agent finding within its style and readability domain. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff` (the standard PR-review entry point) or a `file-path` (single-file review). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `style` or `ui` as the citable candidate set across every enabled layer: the corpus encodes most naming, label, caption, layout, and keyword-casing rules, so a readability concern that maps onto one MUST cite it rather than be paraphrased. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Where a concrete readability defect has no curated rule (an opaque project abbreviation, commented-out code, a TODO with no work-item reference), emit an agent finding within this skill's style and readability domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow the relevant files to the subset that applies to the changes, computing overlap against:
|
||||
|
||||
- Changed identifiers: object, variable, and procedure names, weighted toward Hungarian-prefixed names, single-letter variables outside trivial loops, non-verb-first procedure names, and abbreviations with no obvious BC meaning.
|
||||
- Changed structure: procedures over roughly 80 lines, nesting at five levels or deeper, more than one object per file, and magic numbers or magic strings.
|
||||
- Changed user-facing strings: `Label`, `Caption`, `ToolTip` declarations, weighted toward a missing translator `Comment`, an inline rather than object-scope `Label`, and a `Locked` flag missing on a non-translatable label.
|
||||
- Changed comments: commented-out code, TODO or FIXME without a work-item reference, and public API or library procedures missing XML doc comments.
|
||||
|
||||
A curated `style` or `ui` file enters the worklist when its `keywords` intersect these tokens or its topic matches a changed object kind. Read an article's full `## Best Practice` / `## Anti Pattern` body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklist entry, evaluate the diff against the file's `## Best Practice` and `## Anti Pattern` sections. When the diff clearly matches an anti-pattern (a `Label` with no `Comment`, a lowercase-keyword violation, a missing `Caption` on a page field), emit a knowledge-backed finding citing the file: `id` equal to the file path, `severity` `major` when the file states a hard rule otherwise `minor`, `location` on the offending line or range, `confidence` `high` for an unambiguous match. When the diff contradicts a best practice without being a full anti-pattern, emit `minor` with the same reference shape.
|
||||
|
||||
When a concrete readability defect has no curated rule (commented-out code, an opaque abbreviation that a fresh reader cannot decode, a TODO with no tracking reference), emit an agent finding within this skill's style and readability domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:opaque-abbreviation`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` naming the issue and a concrete rename or removal. Hold every candidate to the precision bar in `skills/do.md`: as a dedicated style skill, readability preferences are inside this skill's domain, but still steelman that the choice is a deliberate, established convention before emitting, and omit when in doubt. Defects outside style and readability belong to other skills. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is mechanical (adding a `Comment` to a label, lowercasing a reserved keyword, moving an inline `Label` to object scope, deleting commented-out lines); otherwise set `suggested-code-omission-reason`. Group repeated instances of one concern into a single finding with a line range rather than many near-identical ones.
|
||||
|
||||
Outcome selection: `completed` when every worklist item was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the diff has no AL to review; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-readability-checker", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 3, "items-evaluated": 3 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "microsoft/knowledge/style/label-comment-explains-placeholders.md",
|
||||
"severity": "major",
|
||||
"message": "The Label declared on line 42 has no Comment attribute. Translators need a Comment that explains the placeholders so word order is preserved in other locales.",
|
||||
"location": {
|
||||
"file": "src/EventRegistrationMgt.Codeunit.al",
|
||||
"line": 42
|
||||
},
|
||||
"references": [
|
||||
{ "path": "microsoft/knowledge/style/label-comment-explains-placeholders.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
89
custom/skills/review/al-table-refactorer.md
Normal file
89
custom/skills/review/al-table-refactorer.md
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-table-refactorer
|
||||
version: 1
|
||||
title: AL table refactor review
|
||||
description: Reviews an AL table for clarity, performance, and house rules, extracting trigger logic and reorganising keys, and emits a findings report with refactored AL.
|
||||
inputs: [file-path, object-list]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL table refactor review
|
||||
|
||||
Reviews a Business Central AL table for clarity, performance, and project house rules, and proposes concrete refactorings: extracting non-trivial validation, calculation, and posting logic out of table triggers into a management codeunit, adding an `IDataAccess` seam where one is missing, reorganising fields and keys for a known access pattern, and tidying FlowFields and CalcFormulas. Refactoring must never change observable behaviour. Because the proposed changes are mechanical AL edits, this skill emits findings that carry the refactored AL in `suggested-code`. It sources from the `performance` and `style` knowledge domains and cites curated rules where a refactoring maps onto one; structural moves the corpus does not encode are agent findings within its design and structure domain. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `file-path` (the table object) or an `object-list`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `performance` or `style` as the citable candidate set across every enabled layer: a key that should align with the filters a caller uses, a FlowField source key that needs `SumIndexFields`, a `CalcFields` that belongs outside a hot loop, declaration order, captions, and object-scope labels each map onto a curated rule and MUST cite it rather than be paraphrased. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The structural moves this skill owns (extracting trigger logic into a codeunit, adding an IDataAccess seam) are mostly not encoded; for those concrete defects, emit an agent finding within this skill's design and structure domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the table, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the refactoring opportunities in the table object:
|
||||
|
||||
- Table triggers (`OnInsert`, `OnModify`, `OnValidate` of a field) holding non-trivial validation, calculation, or posting logic that belongs in a management codeunit, leaving the triggers as thin dispatchers.
|
||||
- A missing `IDataAccess` implementation where the project's no-naive-data-access rule expects one.
|
||||
- Field organisation: primary key first, then foreign keys, then descriptive fields, then computed and FlowFields, then audit fields; a missing secondary key for an obvious non-primary access pattern; a `FindFirst`-on-full-key pattern that should be a `Get`; an auto-increment field used as a primary key.
|
||||
- FlowFields and CalcFormulas: aggregations callers re-derive that a `CalcFormula` could hold; `CalcFields` inside a hot loop; non-obvious formulas with no XML doc comment.
|
||||
- House rules: object id inside the assigned range, user-facing labels via object-scope `Label` with a `Comment`, telemetry on protected operations, captions populated, one object per file.
|
||||
|
||||
A curated `performance` or `style` file enters the worklist when its `keywords` intersect these tokens (for example `key`, `setcurrentkey`, `flowfield`, `calcfields`, `caption`, `label`). Read its full `## Best Practice` / `## Anti Pattern` body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each refactoring opportunity, emit a finding that carries the refactored AL.
|
||||
|
||||
When the refactoring maps onto a curated `performance` or `style` rule (aligning a key with a filter pattern, adding `SumIndexFields` to a FlowField source key, moving a label to object scope, populating a caption), emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `severity` up to `blocker` only when the file states a platform-level guarantee otherwise `major`, `confidence` `high` for an unambiguous match.
|
||||
|
||||
When the refactoring is a structural move with no curated rule (extracting validation from a trigger into a management codeunit, adding an IDataAccess seam, reordering fields, adding a secondary key), emit an agent finding within this skill's design and structure domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:extract-trigger-logic`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` naming the move and confirming it preserves observable behaviour. Where the impact would normally gate, keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the current shape is deliberate before emitting, and omit when in doubt.
|
||||
|
||||
Because this skill's job is to produce the refactored table, set `suggested-code` on every finding where the change is mechanical and contiguous: it carries the literal refactored AL for the lines indicated by `location` (the reorganised `keys` block, the rewritten thin trigger, the relocated `Label`). When a `.good.al` companion exists and the table matches the `.bad.al` shape, adapt the `.good.al` replacement into `suggested-code`. Omit `suggested-code` only when the move spans non-contiguous code or a behaviour-preserving rewrite cannot be determined from the table alone (for example extracting logic that needs a new sibling codeunit file), and then set `suggested-code-omission-reason`. If a change might alter observable behaviour, lower confidence and say so in the `message` rather than emitting a confident replacement.
|
||||
|
||||
Outcome selection: `completed` when every opportunity was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the input is not a table object; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-table-refactorer", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 2, "items-evaluated": 2 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:extract-trigger-logic",
|
||||
"severity": "minor",
|
||||
"message": "The OnValidate trigger of field Attendee Count holds the full attendee-cap validation inline. Extract it to Event Registration Mgt.ValidateAttendeeCount and call that from the trigger, leaving the trigger as a thin dispatcher. The extracted call preserves observable behaviour. This concern should be promoted to a knowledge-backed rule before it can gate.",
|
||||
"location": {
|
||||
"file": "src/EventRegistration.Table.al",
|
||||
"line": 40,
|
||||
"range": { "start-line": 40, "end-line": 41 }
|
||||
},
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": " trigger OnValidate()\n begin\n EventRegistrationMgt.ValidateAttendeeCount(Rec);\n end;"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
81
custom/skills/review/al-translation-auditor.md
Normal file
81
custom/skills/review/al-translation-auditor.md
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-translation-auditor
|
||||
version: 1
|
||||
title: AL translation coverage audit
|
||||
description: Audits AL source strings against xliff files and the supported countries declared in AppSourceCop.json, and emits a findings report.
|
||||
inputs: [repository, object-list]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL translation coverage audit
|
||||
|
||||
Compares every `Label`, `Caption`, `ToolTip`, `Comment`, and `Description` string in the AL source against the xliff files under the translations folder and against the `supportedCountries` declared in `AppSourceCop.json`. It catches the silent failure where an extension claims to support a market but ships only en-US text, one of the top AppSource rejection reasons. The skill checks one xliff per supported locale, every source string present and translated in every xliff, empty and verbatim targets, orphan trans-units, translator comments on substituted labels, and truncation risk. It sources from the `style` knowledge domain and cites curated label and caption rules where present; coverage gaps the corpus does not encode are agent findings within its translation domain. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `repository` or an `object-list`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `style` as the citable candidate set across every enabled layer: a `Label` or `Caption` missing its translator `Comment`, and the `Comment` that must explain placeholders, each map onto a curated rule and MUST cite it rather than be paraphrased, because a missing comment is the root cause of the mistranslations this skill audits. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The coverage checks themselves (a missing xliff for a supported locale, an untranslated or empty target, an orphan trans-unit) are not encoded in the corpus; for those concrete defects, emit an agent finding within this skill's translation domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries (the `supportedCountries` from `AppSourceCop.json`), or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Build the string-to-translation map and narrow to the gaps:
|
||||
|
||||
- Walk the AL source (or supplied `object-list`) for every `Label`, `Caption`, `ToolTip`, `Comment`, and `Description`. Walk every `*.xlf` or `*.xliff` under the translations folder. Read the `supportedCountries` array from `AppSourceCop.json`.
|
||||
- Each `supportedCountries` locale with no matching xliff file; each source string with no `<trans-unit>` in a non-default xliff or no `<target>` element; empty `<target>` elements; non-en-US targets identical to the source with no `state="needs-review"` or justifying note; orphan trans-units that match no source string; labels with `%1`/`%2` substitutions whose `Comment` is missing or did not round-trip into the xliff `note`; targets whose length exceeds the source caption's apparent `MaxLength`.
|
||||
|
||||
A curated `style` file enters the worklist when its `keywords` intersect these tokens (for example `label`, `caption`, `comment`, `translation`, `placeholder`). Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each gap, emit a finding.
|
||||
|
||||
When the gap maps onto a curated `style` rule (a substituted label with no explanatory `Comment`, a `Caption` missing its `Comment`), emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `severity` up to `blocker` only when the file states a platform-level guarantee otherwise `major`, `confidence` `high` for an unambiguous match.
|
||||
|
||||
When the gap is a coverage defect with no curated rule (a missing xliff for a supported locale, an untranslated or empty target, a verbatim target, an orphan trans-unit, a truncation risk), emit an agent finding within this skill's translation domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:missing-xliff-for-supported-country`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` naming the locale or string and the concrete remedy (generate the xliff, translate the target). Where the impact would normally gate (a missing locale that AppSource rejects), keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the locale legitimately falls back to a sibling or that an identical target is correct for that string before emitting, and omit when in doubt. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` only when the fix is an exact, contiguous edit to an AL source string (adding a `Comment` attribute to a label); a missing or untranslated xliff is not a single-line source replacement, so set `suggested-code-omission-reason` (for example `requires generating and translating an xliff file`).
|
||||
|
||||
Outcome selection: `completed` when every supported locale and source string was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the task has no source strings or translations to compare; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-translation-auditor", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 2, "items-evaluated": 2 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:missing-xliff-for-supported-country",
|
||||
"severity": "minor",
|
||||
"message": "AppSourceCop.json declares AU as a supported country but no Translations xliff for en-AU exists, so AU tenants see whatever the default xliff carries. Generate the en-AU xliff and translate it, or document an explicit fallback to en-NZ. Impact would normally be major because AppSource validation rejects this; emitted as minor because no curated rule backs it. This concern should be promoted to a knowledge-backed rule before it can gate.",
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "requires generating and translating a new xliff file"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
92
custom/skills/review/al-upgrade-checker.md
Normal file
92
custom/skills/review/al-upgrade-checker.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-upgrade-checker
|
||||
version: 1
|
||||
title: AL upgrade coverage review
|
||||
description: Verifies that schema changes in an AL extension are handled by an upgrade codeunit for existing-tenant data, and emits a findings report.
|
||||
inputs: [pr-diff, repository]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL upgrade coverage review
|
||||
|
||||
Compares a schema diff against the extension's upgrade codeunit and reports every migration that is missing or broken. It catches the silent install failure: a clean install on a dev sandbox, then `Install-NAVApp` errors on production tenants because existing data is not migrated. The skill covers new required fields, fields with `InitValue`, field renames, type or length narrowing, obsoleted fields, enum value changes, primary key changes, per-company versus per-database scope, idempotency, and upgrade-tag registration. It sources from the `upgrade` knowledge domain and cites curated rules where a schema change maps onto one; gaps the corpus does not encode are agent findings within its upgrade domain. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff` (the standard PR-review entry point) or a `repository`. It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `upgrade` as the citable candidate set across every enabled layer: `InitValue` not updating existing rows, upgrade tags instead of version checks, `DataTransfer` for bulk init, enum values added at the end, guarding database reads, and no external calls in an upgrade codeunit each map onto a curated rule and MUST cite it rather than be paraphrased. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Where a concrete migration gap has no curated rule (a per-company migration that touches a per-database surface, a non-idempotent migration, a wire-contract break that an upgrade codeunit alone cannot fix), emit an agent finding within this skill's upgrade domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Compute the schema diff and narrow to the changes that need a migration:
|
||||
|
||||
- New required or non-nullable fields with no default, and new fields with an `InitValue` that downstream code assumes is populated on existing rows.
|
||||
- Field renames (AL `Rename` moves only metadata, not data, and does not rewrite JSON keys that store the old field name).
|
||||
- Field type or length narrowing (Code[20] to Code[10], Text[100] to Text[50], Decimal to Integer) where existing rows hold out-of-bound values.
|
||||
- Obsoleted fields whose data the upgrade codeunit must migrate before the Removed cycle, and any `Obsolete Removed` with no prior `Pending`.
|
||||
- Enum value reorders or renames that break serialised ordinals or AL identifier comparisons, and additions to a non-Extensible versus Extensible enum.
|
||||
- Primary key changes that require existing rows to disambiguate.
|
||||
- The upgrade codeunit itself: `OnUpgradePerCompany` versus `OnUpgradePerDatabase` scope, idempotency of each step, and a unique `UpgradeTag` registered per step so re-publishes skip applied work.
|
||||
- Wire-contract impact when the schema change affects an exposed API shape.
|
||||
|
||||
A curated `upgrade` file enters the worklist when its `keywords` intersect these tokens (for example `initvalue`, `upgrade-tag`, `datatransfer`, `enum`, `rename`, `obsolete`). Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each schema change, check the upgrade codeunit covers it and emit findings.
|
||||
|
||||
When a gap maps onto a curated `upgrade` rule (a new field relying on `InitValue` for existing rows, a missing upgrade tag, an enum value added in the middle), emit a knowledge-backed finding citing that file: `id` equal to the file path, the file as primary reference, `severity` up to `blocker` only when the file states a platform-level guarantee otherwise `major`, `confidence` `high` for an unambiguous match. State the tenants affected in the `message`.
|
||||
|
||||
When a concrete migration gap has no curated rule (a per-company step that writes a per-database surface and risks duplicate writes, a non-idempotent migration that appends rows on re-run, a primary-key change with no disambiguation, a wire-contract break needing an API version bump), emit an agent finding within this skill's upgrade domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:non-idempotent-upgrade-step`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` describing the failure on tenants with existing data and a concrete fix. Where the impact would normally gate (a missing migration that fails the install), keep `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a knowledge-backed rule before it can gate. Hold every candidate to the precision bar in `skills/do.md`: steelman that the migration is covered by a step outside the diff before emitting, and omit when in doubt. Before emitting any agent candidate, check the worklisted knowledge for a match and upgrade it to a knowledge-backed finding if one exists.
|
||||
|
||||
Set `suggested-code` when the fix is mechanical (wrapping a step in `UpgradeTagMgt.HasUpgradeTag`/`SetUpgradeTag`); otherwise set `suggested-code-omission-reason` (for example `requires authoring a new upgrade step body`).
|
||||
|
||||
Outcome selection: `completed` when every schema change was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the diff has no schema change to review; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-upgrade-checker", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 0, "info": 0 },
|
||||
"coverage": { "worklist-size": 3, "items-evaluated": 3 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "microsoft/knowledge/upgrade/initvalue-does-not-update-existing-rows.md",
|
||||
"severity": "major",
|
||||
"message": "A required enum field Status Code was added to table Event Registration with no upgrade step. InitValue applies to new rows only, so every existing row on every tenant is left empty. Add a step to OnUpgradePerCompany that sets Status Code to Draft on existing rows, behind a fresh UpgradeTag.",
|
||||
"location": {
|
||||
"file": "src/EventRegistration.Table.al",
|
||||
"line": 30
|
||||
},
|
||||
"references": [
|
||||
{ "path": "microsoft/knowledge/upgrade/initvalue-does-not-update-existing-rows.md" }
|
||||
],
|
||||
"confidence": "high"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
90
custom/skills/testing/ai-test-driven-development.md
Normal file
90
custom/skills/testing/ai-test-driven-development.md
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: ai-test-driven-development
|
||||
version: 1
|
||||
title: AI test-driven development
|
||||
description: TDD for Copilot features and custom agents covering Evaluation suites, JSONL/YAML datasets, AITest codeunits, agent turn loops, intervention validation, and credit tracking.
|
||||
inputs: [repository, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AI test-driven development
|
||||
|
||||
Drives test-driven development for Business Central Copilot features and custom agents using the Evaluation suite (the data-driven tool where datasets describe inputs and expected outputs and the test codeunit drives the loop). It covers two flows: prompt-based AI tests for PromptDialog features (JSONL or YAML datasets) and multi-turn agent accuracy tests with intervention validation (YAML only). This is a generator-style skill: it generates AITest codeunits, dataset scaffolds, and suite XML as findings whose `suggested-code` carries the artifact, and it also reviews an existing AI test setup for misconfiguration. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `repository` (the Copilot or agent extension and its test app) and a `file-path` (the capability, PromptDialog, or agent under test, or an existing AI test codeunit or dataset to review). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `testing` as the citable candidate set across every enabled layer: AITest-codeunit attribute rules, isolation rules (`TestType = AITest`, `RequiredTestIsolation = Disabled` for agent tests), suite-setup discipline, intervention-contract rules, and credit-tracking guidance back the findings here. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Generated artifacts and configuration observations with no curated backing are agent findings within this skill's domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the repository `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown` (the suite XML may enable multilingual evaluation, so countries can matter).
|
||||
- `application-area`: the application areas of the Copilot or agent feature, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the AI test work the task requires:
|
||||
|
||||
- The flow: prompt-based AI test (PromptDialog plus an Azure OpenAI call, JSONL or YAML dataset) or agent accuracy test (multi-turn, YAML only).
|
||||
- The AITest codeunit: `Subtype = Test`, `TestType = AITest`, `TestPermissions = Disabled`, and for agent tests `RequiredTestIsolation = Disabled` (essential, because agent tasks run in a different session and span transactions).
|
||||
- The turn loop for agent tests: the `repeat ... until` delegating to `Library - Agent` (`RunTurnAndWait`, `FinalizeTurn`), with validators returning `false` and a populated `ErrorReason` rather than calling `Error()`.
|
||||
- The dataset: `test_setup` and `expected_data` keys for AI tests; `turns:` with `query`/`expected_data` for agent tests; the `intervention_request` sub-key the framework reads automatically (both directions: a declared intervention must pause with matching type and suggestions, an undeclared one must not pause); `$DateFormula-<...>$` placeholders, always quoted.
|
||||
- The suite XML: `TestRunnerId="130451"` (Isolation-Disabled runner, required for agent tests), `TestType="Agent"` versus `"AITest"`, `<Language>` children, and the install-time dataset load.
|
||||
- Suite-setup discipline: `AITTestContext.IsSuiteSetupDone()` is sticky; re-running setup needs the Reset Suite Setup action.
|
||||
- Credit and permission constraints: Evaluation runs consume Copilot credits (tracked per suite, per line, per entry; limited at environment and company level); users need the `AI TEST TOOLKIT` permission set.
|
||||
|
||||
A curated `testing` file enters the worklist when its `keywords` intersect these tokens. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
When generating, emit one finding per generated artifact (the AITest codeunit, the dataset, the suite XML, the install codeunit), each carrying the artifact in `suggested-code` with a `message` stating what was generated and how to wire it. When reviewing an existing setup, emit a finding per defect: a missing `RequiredTestIsolation = Disabled` on an agent test, a wrong `TestRunnerId`, a validator calling `Error()` instead of returning `ErrorReason`, an unquoted date placeholder, an intervention contract the dataset does not exercise in both directions, or a sticky suite-setup that silently ignores edited setup YAML. Where a curated `testing` file states the rule, emit a knowledge-backed finding citing it: `id` equal to the file path, `severity` up to `major` (`blocker` only when the file states a platform-level guarantee, for example an isolation rule whose violation makes the agent runner unusable), `confidence` `high` for an unambiguous match. Where no curated file applies, emit an agent finding within this skill's domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:generated-aitest-codeunit`, `agent:missing-disabled-isolation`, `agent:unquoted-date-placeholder`), `confidence` capped at `medium`, `severity` capped at `minor`, self-contained `message`. Put generated AL, YAML, or XML in `suggested-code`; for a mechanical fix to an existing file (adding the isolation attribute, quoting a placeholder) also set `suggested-code`. Where the fix is not local (restructuring a turn loop), set `suggested-code-omission-reason`. Hold every agent finding to the precision bar in `skills/do.md`.
|
||||
|
||||
Outcome selection: `completed` when the requested generation or review finished (including a clean review with empty `findings`); `not-applicable` when the repository has no Copilot capability, PromptDialog, or custom agent to test; `no-knowledge` when no curated knowledge survived and no agent finding was raised; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. Generated artifacts and configuration findings with no curated backing are agent findings (`references: []`, `agent:` id, severity capped at `minor`); findings citing a `testing` file carry that file path as `id` and primary reference.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "ai-test-driven-development", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 3, "items-evaluated": 3 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:missing-disabled-isolation",
|
||||
"severity": "minor",
|
||||
"message": "Agent accuracy codeunit 50202 sets TestType = AITest but not RequiredTestIsolation = Disabled. Impact is major: agent tasks run in a different session and span transactions, so the runner cannot enforce isolation and the suite fails to start. Add the attribute. Promote to a curated rule so it can gate.",
|
||||
"location": { "file": "test/MyAgentAccuracyTest.Codeunit.al", "line": 4 },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": " RequiredTestIsolation = Disabled;"
|
||||
},
|
||||
{
|
||||
"id": "agent:generated-agent-dataset",
|
||||
"severity": "info",
|
||||
"message": "Generated a YAML agent dataset with a turns chain and an intervention_request the FinalizeTurn contract enforces in both directions. Date values use quoted $DateFormula placeholders so the dataset does not drift against WorkDate. Ship it under the test app .resources/ folder and load it in an Install codeunit.",
|
||||
"location": { "file": "test/.resources/datasets/MY-DATASET.yaml" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": "name: MY-DATASET\nsuite_setup: MY-AGENT\ntests:\n - turns:\n - query:\n message: \"Release all open sales orders for next week\"\n expected_data:\n orders_released: 2"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
91
custom/skills/testing/al-test-coverage-enforcer.md
Normal file
91
custom/skills/testing/al-test-coverage-enforcer.md
Normal file
|
|
@ -0,0 +1,91 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-test-coverage-enforcer
|
||||
version: 1
|
||||
title: AL test coverage enforcer
|
||||
description: Hard coverage gate, passing only when AL coverage meets the threshold and otherwise naming every uncovered path.
|
||||
inputs: [pr-diff, repository]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL test coverage enforcer
|
||||
|
||||
Decides whether a Business Central change has enough AL test coverage to ship. Unlike `al-test-coverage-validator`, which reports, this skill gates: it passes (an empty `findings` array) only when every new or behaviour-changed production surface has at least one identifiable covering test, and otherwise emits a finding per uncovered path at gating severity. When in doubt it fails, because the cost of one extra test is low and the cost of an uncovered regression is high. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff` (the production change to gate) and a `repository` (so the test index and any coverage report can be read). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `testing` as the citable candidate set across every enabled layer: a curated rule about a coverage threshold or a mandatory regression test is the authoritative basis that lets this skill gate at `major` or `blocker`. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Where the project threshold is a house default with no curated backing, see Action for how severity is handled.
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the repository `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the production surfaces the threshold applies to:
|
||||
|
||||
- New public procedures (default threshold: every one must have at least one direct or indirect covering test, no exceptions).
|
||||
- New event subscribers (must have a test that fires the publisher in a realistic context).
|
||||
- New table triggers (`OnInsert`, `OnModify`, `OnDelete`, field `OnValidate`): each must have a covering test.
|
||||
- Modified procedures with a behaviour change: an existing or new test must assert the new behaviour. A behaviour change whose existing tests still pass unchanged is itself a gap (the tests do not exercise the new behaviour).
|
||||
- Bug fixes: must add a regression test that names the bug and fails without the fix.
|
||||
- Pure refactors with no behaviour change: existing covering tests must still apply; no new test required.
|
||||
|
||||
Compute the covering set from the test index (procedure to referencing tests) and any supplied coverage report. A curated `testing` file enters the worklist when its `keywords` intersect a coverage-threshold rule. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklisted surface, decide PASS or FAIL against the threshold. Emit a finding for every FAIL reason, each naming the specific procedure, subscriber, trigger, or bug fix. Where a curated `testing` knowledge file states the coverage requirement, emit a knowledge-backed finding citing it: `id` equal to the file path, `severity` `blocker` when the file states a platform-level guarantee, otherwise `major`; `confidence` `high` for an unambiguous gap. Where the requirement is the house default with no curated backing, the finding is an agent finding within this skill's domain (`references: []`, `id` slug prefixed `agent:` such as `agent:uncovered-new-public-procedure` or `agent:bug-fix-missing-regression-test`, `confidence` capped at `medium`). Per `skills/do.md`, an agent finding's `severity` is capped at `minor` even though this skill gates: keep the emitted severity at `minor`, state plainly in the `message` that the impact is gating (it blocks completion under the project threshold), and flag that the threshold should be promoted to a curated `testing` rule so the gate carries authoritative weight. The consuming orchestrator combines the threshold configuration with these findings to set the actual PASS/FAIL on the merge. The fix is a new test, so omit `suggested-code` and set `suggested-code-omission-reason` to `the gap is closed by adding a covering test`.
|
||||
|
||||
When no coverage report is supplied, fall back to static analysis of the test codeunits (which procedures each test references directly) and state the limitation in an `info` finding. When in doubt about whether a surface is covered, prefer to emit the finding.
|
||||
|
||||
Outcome selection: `completed` when every worklisted surface was decided (an empty `findings` array means PASS, the gate is satisfied); `not-applicable` when the diff has no new or behaviour-changed production surface (a pure refactor or doc-only change); `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. An empty `findings` array with `outcome: completed` is the PASS signal. Threshold findings with no curated backing are agent findings (`references: []`, `agent:` id, severity capped at `minor`, gating impact stated in the message); findings citing a `testing` file carry that file path as `id` and may gate at `major` or `blocker`.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-test-coverage-enforcer", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 2, "info": 0 },
|
||||
"coverage": { "worklist-size": 3, "items-evaluated": 3 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:uncovered-new-public-procedure",
|
||||
"severity": "minor",
|
||||
"message": "codeunit 50101 'Event Registration Mgt'.ReleaseRegistration is new in this diff and no test references it directly or indirectly. Impact is gating: under the project threshold this blocks completion. Promote the threshold to a curated testing rule so the gate carries authoritative weight.",
|
||||
"location": { "file": "src/Sales/EventRegistrationMgt.Codeunit.al" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "the gap is closed by adding a covering test"
|
||||
},
|
||||
{
|
||||
"id": "agent:bug-fix-missing-regression-test",
|
||||
"severity": "minor",
|
||||
"message": "The commit references work item #1234 (a fix) but no new test names the item or asserts the prior failure mode. Impact is gating: a bug fix must ship with a regression test that fails without the fix.",
|
||||
"location": { "file": "src/Sales/EventRegistrationMgt.Codeunit.al" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "the gap is closed by adding a covering test"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
91
custom/skills/testing/al-test-coverage-validator.md
Normal file
91
custom/skills/testing/al-test-coverage-validator.md
Normal file
|
|
@ -0,0 +1,91 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-test-coverage-validator
|
||||
version: 1
|
||||
title: AL test coverage validator
|
||||
description: Reports AL test coverage shape and identifies untested branches, error handlers, and edge cases without gating.
|
||||
inputs: [pr-diff, repository]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL test coverage validator
|
||||
|
||||
Reports what a production AL change is and is not covered by, and how meaningfully. For each production procedure or trigger the diff touches, it classifies coverage as covered, shallow, uncovered, or not-applicable, and surfaces uncovered branches, error handlers, and edge cases. It reports; it does not gate. The hard PASS/FAIL decision lives in `al-test-coverage-enforcer`. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff` (the production change to assess) and a `repository` (so the test codeunit index and any coverage report can be read). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `testing` as the citable candidate set across every enabled layer; coverage is structural and rarely maps onto a curated rule, but a published `testing` rule about minimum coverage for a specific area would back a finding. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Most coverage observations are agent findings within this skill's domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the repository `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the changed objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the production surface the diff touches and the coverage signal for each:
|
||||
|
||||
- Every production procedure and trigger (`OnInsert`, `OnValidate`, and so on) added or changed in the diff, excluding test files.
|
||||
- For each, the test index: which test codeunits reference it directly or via a clear chain, and whether any asserts an outcome that depends on its body.
|
||||
- Branch and error-handler coverage: uncovered `if`/`case` arms, uncovered `Error()` paths, uncovered `else` guards.
|
||||
- Edge-case coverage where the procedure can encounter them: nulls, empty sets, max values, permission failures, date boundaries.
|
||||
- Bug-fix regression coverage: when the diff message references a work item or fix, whether a test names it.
|
||||
- Mutation survivors, when supplied: mutants that survived because no test caught the logic change.
|
||||
|
||||
A curated `testing` file enters the worklist only when its `keywords` intersect a real coverage rule. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
Classify each worklisted procedure or trigger: `covered` (a test calls it and asserts an outcome exercising its body), `shallow` (a test calls it but asserts nothing depending on its return or side effects), `uncovered` (no test references it), or `n/a` (deleted or a pure pass-through). Emit a finding for every gap. These are agent findings within this skill's domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:uncovered-procedure`, `agent:shallow-coverage`, `agent:uncovered-branch`, `agent:missing-edge-case`), `confidence` capped at `medium`, `severity` capped at `minor` (this skill reports, it does not gate, so even a wholly uncovered new public procedure is `minor` here and the enforcer raises it). The `message` is self-contained: name the object and procedure, the coverage class, and the concrete gap (which branch, which edge case, which assertion is missing). Where a published `testing` rule genuinely backs a coverage requirement, upgrade that finding to knowledge-backed and cite the file. Hold every agent candidate to the precision bar in `skills/do.md`: a pass-through that genuinely needs no test is not a gap. The fix is a new or extended test rather than a local edit, so omit `suggested-code` and set `suggested-code-omission-reason` to `coverage gap is closed by adding a test, not a local code edit`.
|
||||
|
||||
When the test index is missing, report only what static analysis of the diff yields (new procedure count, new trigger count) and state the limitation in an `info` finding.
|
||||
|
||||
Outcome selection: `completed` when every touched procedure was classified (including an empty `findings` when coverage is complete); `not-applicable` when the diff has no production AL surface; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. Coverage gaps are agent findings (`references: []`, `agent:` id, severity capped at `minor`); the gating decision lives in `al-test-coverage-enforcer`.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-test-coverage-validator", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 2, "info": 0 },
|
||||
"coverage": { "worklist-size": 8, "items-evaluated": 8 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:uncovered-procedure",
|
||||
"severity": "minor",
|
||||
"message": "codeunit 50101 'Event Registration Mgt'.ReleaseRegistration is touched by the diff but no test codeunit references it directly or indirectly. Add a covering test. The enforcer will gate on this; here it is reported only.",
|
||||
"location": { "file": "src/Sales/EventRegistrationMgt.Codeunit.al" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "coverage gap is closed by adding a test, not a local code edit"
|
||||
},
|
||||
{
|
||||
"id": "agent:shallow-coverage",
|
||||
"severity": "minor",
|
||||
"message": "ValidateAttendeeCount is called by a test but the test asserts nothing depending on its outcome, so the resulting error path is not actually verified. Add an asserterror on the over-capacity case.",
|
||||
"location": { "file": "src/Sales/EventRegistrationMgt.Codeunit.al" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "coverage gap is closed by adding a test, not a local code edit"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
87
custom/skills/testing/al-test-runner.md
Normal file
87
custom/skills/testing/al-test-runner.md
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-test-runner
|
||||
version: 1
|
||||
title: AL test runner
|
||||
description: Executes AL test codeunits via the AL-Go runner or a local container and returns the run result as a findings report.
|
||||
inputs: [repository, object-list]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL test runner
|
||||
|
||||
Executes Business Central AL test codeunits and reports the run result in a shape the rest of the verifier chain can consume. It detects the project's runner (AL-Go pipeline, a Docker BC sandbox via BcContainerHelper, or a project-local build script), invokes it, parses the XUnit-style results, and maps each failure to a finding. It does not judge coverage (that is `al-test-coverage-validator` and `al-test-coverage-enforcer`), test quality (that is `al-test-validator`), or write tests (that is `al-test-writer`). This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `repository` (the project root to run in) and optionally an `object-list` (a filter narrowing the run to specific test codeunits). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `testing` or `pipelines` as the citable candidate set across every enabled layer: runner-selection, isolation, and AL-Go pipeline rules can back a finding about how the run was configured. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. A reported test failure or a runner-startup failure rarely maps onto a curated rule, so it is emitted as an agent finding within this skill's domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the repository `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the test objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the run to perform and the artifacts it produces:
|
||||
|
||||
- Runner detection, in order: an AL-Go pipeline (`.AL-Go/settings.json` plus a `BuildALGoProject` script); a Docker BC sandbox via BcContainerHelper (a `BcContainerHelperVersion` setting or a `Run-TestsInBcContainer` call); a project-local `scripts/Build.ps1` or equivalent.
|
||||
- The test codeunits to run: every test codeunit in the repository, narrowed by the `object-list` filter when supplied.
|
||||
- The results file the run emits (`TestResults.xml` or equivalent) and the runner console output.
|
||||
|
||||
A curated `testing` or `pipelines` file enters the worklist when its `keywords` intersect these tokens. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
Invoke the detected runner with the project's standard arguments, capture its output and the XUnit-style results file, and parse total, passed, failed, and skipped counts. Emit one finding per failed test: an agent finding (`references: []`, `id` slug prefixed `agent:` such as `agent:test-failed`, `confidence` capped at `medium`, `severity` capped at `minor`), with a self-contained `message` carrying the test codeunit, the test procedure, the assertion message, and the source location, and a `location` pointing at the failing line. Keep `severity` at `minor` even though a red test commonly blocks the chain, and say in the `message` that the run failed; the gating decision belongs to `al-test-coverage-enforcer` and the consuming orchestrator, not to this advisory channel. A run with many skipped tests emits an `info` finding naming the skip count. Where a curated `testing` or `pipelines` rule explains a misconfiguration the run surfaced (for example an isolation attribute that produced a spurious failure), upgrade that finding to knowledge-backed and cite the file. Mechanical fixes are rare here (the fix lives in the test or production code, not in the run), so omit `suggested-code` and set `suggested-code-omission-reason` to `fix lives in the test or production source under change`.
|
||||
|
||||
If the runner cannot start (Docker daemon down, BC image missing, AL-Go misconfigured) or exceeds the configured timeout, do not silently succeed: emit `outcome: "failed"` with `outcome-reason` carrying the exact command attempted and the error output, and an agent finding describing the startup failure.
|
||||
|
||||
Outcome selection: `completed` when the run finished and every failure was mapped to a finding (including a green run with empty `findings`); `not-applicable` when the repository has no AL test codeunit or no runner could be detected; `partial` when the run was cancelled on timeout after some tests ran (`summary.coverage` reflects the executed subset); `failed` when the runner could not start, with `outcome-reason` required.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. Test failures are agent findings (`references: []`, `agent:` id, severity capped at `minor`).
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-test-runner", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"outcome-reason": "al-go-pipeline runner, 27 tests, 1 failed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 1 },
|
||||
"coverage": { "worklist-size": 27, "items-evaluated": 27 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:test-failed",
|
||||
"severity": "minor",
|
||||
"message": "Test ReleaseRegistrationShouldFailWhenOverCapacity in codeunit 50202 'Event Registration Tests' failed: expected error 'Capacity exceeded' but got 'Permission denied'. The run is red; fix the production code or the test before the chain can gate. Runner: al-go-pipeline.",
|
||||
"location": { "file": "test/EventRegistrationTests.al", "line": 88 },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "fix lives in the test or production source under change"
|
||||
},
|
||||
{
|
||||
"id": "agent:tests-skipped",
|
||||
"severity": "info",
|
||||
"message": "1 test was skipped via an explicit Skip() call (codeunit 50202, SmokeTest). Skipped tests do not fail the run; confirm the skip is intentional.",
|
||||
"references": [],
|
||||
"confidence": "medium"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
87
custom/skills/testing/al-test-validator.md
Normal file
87
custom/skills/testing/al-test-validator.md
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-test-validator
|
||||
version: 1
|
||||
title: AL test validator
|
||||
description: Flags AL tests with no assertions, missing edge cases, weak names, poor isolation, and tests that exercise implementation not behaviour.
|
||||
inputs: [pr-diff, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL test validator
|
||||
|
||||
Reads AL test codeunits and reports whether each test is meaningful, well-named, and correctly configured. The failure modes are familiar: a `[Test]` with no `Assert.*`, a name that describes the call site instead of the expected behaviour, an isolation attribute set wrong for an AI test, a test that exercises implementation detail rather than behaviour, or a missing edge case the domain demands. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `pr-diff` (the changed test files) or a `file-path` (a single test codeunit). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `testing` as the citable candidate set across every enabled layer: test-attribute, isolation, transaction-model, and assertion-pattern rules are the authoritative basis for most findings here. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Project-specific conventions (a naming pattern, a project helper codeunit) rarely map onto a curated file, so a finding about them is an agent finding within this skill's domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the tested objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the test surface under review. Before raising findings, build a mental model of what the tests should assert: identify the business domain, the invariants and state transitions the system under test enforces, and the edge cases (empty sets, max values, null or blank fields, permission failures, date boundaries) the domain demands. Then compute overlap against:
|
||||
|
||||
- Every `[Test]` procedure in the changed or supplied test codeunits, and whether each has at least one `Assert.*` or `asserterror` expectation.
|
||||
- Codeunit configuration: `Subtype = Test`; for AI tests `TestType = AITest`, `TestPermissions = Disabled`, and for agent tests `RequiredTestIsolation = Disabled`; suite-setup guards via `AITTestContext.IsSuiteSetupDone()`.
|
||||
- Test names: behaviour-describing (`ReleaseRegistrationShouldFailWhenOverCapacity`) versus implementation-named (`TestReleaseRegistration`) versus opaque (`TestProc01`).
|
||||
- Anti-patterns: assertions inside an unguarded loop, a test calling `Commit()`, dependence on global state with no seed, deep mocking, a single test asserting multiple behaviours, undocumented `Sleep()`.
|
||||
- Mutation survivors, when supplied: which test should have caught each survivor and which assertion it lacks.
|
||||
|
||||
A curated `testing` file enters the worklist when its `keywords` intersect these tokens. Read its full `## Best Practice` / `## Anti Pattern` body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each worklisted test, evaluate it against the model and the worklisted knowledge. A missing assertion, a `Commit()` in a test, an assertion inside an unguarded loop, global-state dependence, or an isolation/transaction-model attribute wrong for the test type is a defect: when a curated `testing` file states the rule, emit a knowledge-backed finding citing it (`id` equal to the file path, `severity` up to `major`, `blocker` only when the file states a platform-level guarantee, `confidence` `high` for an unambiguous match). Implementation-named tests, single-test-multiple-behaviour, missing edge cases, and project-convention drift rarely map onto a curated file: emit them as agent findings within this skill's domain (`references: []`, `id` slug prefixed `agent:`, `confidence` capped at `medium`, `severity` capped at `minor`, self-contained `message`). When the underlying impact would otherwise be major (a missing assertion that lets a mutation survive), keep the emitted `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a curated rule before it can gate. Hold every agent candidate to the precision bar in `skills/do.md`: steelman that the test is deliberate (a no-throw contract is sometimes the real contract) before emitting, and omit when in doubt. Set `suggested-code` when the fix is mechanical (rename a test, add a `[HandlerFunctions(...)]` attribute, add the missing isolation attribute); otherwise set `suggested-code-omission-reason`.
|
||||
|
||||
Outcome selection: `completed` when every worklisted test was evaluated (including an empty `findings`); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the task carries no AL test to validate; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. Findings without a knowledge file are agent findings (`references: []`, `agent:` id, severity capped at `minor`); findings citing a `testing` file carry that file path as `id` and primary reference.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-test-validator", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 1, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 4, "items-evaluated": 4 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "microsoft/knowledge/testing/transactionmodel-attribute-governs-test-transactions.md",
|
||||
"severity": "major",
|
||||
"message": "PostAndAssertNoChange posts a document but TransactionModel is unset, so posting rolls back at test end and the state-after-post assertion never observes a committed change.",
|
||||
"location": { "file": "test/EventRegistrationTests.al", "line": 88 },
|
||||
"references": [ { "path": "microsoft/knowledge/testing/transactionmodel-attribute-governs-test-transactions.md" } ],
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"id": "agent:implementation-named-test",
|
||||
"severity": "minor",
|
||||
"message": "Test procedure TestReleaseRegistration names the call site, not the expected behaviour. Rename to describe the outcome, for example ReleaseRegistrationShouldEmitTelemetry, so a failure reads as a broken contract.",
|
||||
"location": { "file": "test/EventRegistrationTests.al", "line": 40 },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": " procedure ReleaseRegistrationShouldEmitTelemetry()"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
79
custom/skills/testing/al-test-writer.md
Normal file
79
custom/skills/testing/al-test-writer.md
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-test-writer
|
||||
version: 1
|
||||
title: AL test writer
|
||||
description: Generates AL test codeunits for a target production object as findings carrying the test source, the TDD red step.
|
||||
inputs: [object-list, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL test writer
|
||||
|
||||
Generates Business Central AL test codeunits that exercise a target production object and assert specific behaviours. This is a generator-style skill: it expresses each generated test codeunit as a finding whose `suggested-code` carries ready-to-drop AL, with the `## Action` step explaining what is generated. The contract is the TDD red step: the generated test must fail against the current production code and pass once the intended behaviour is implemented. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with an `object-list` (the target objects to cover) and a `file-path` (the production AL under test, plus the behaviour spec the caller supplies). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `testing` as the citable candidate set across every enabled layer: test-attribute, isolation, transaction-model, and assertion-pattern rules govern how the generated codeunit must be shaped, so a finding that matches a curated rule cites that file. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Where no curated rule covers a concrete generation choice, this skill emits an agent finding within its own domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the branch `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the target objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the generation work the task actually requires:
|
||||
|
||||
- One target per entry in `object-list`: a codeunit, table, page, or report whose behaviour the caller's spec describes.
|
||||
- The behaviour spec: which inputs should produce which outcome, side effect, or error. If the spec is missing for a target, that target produces no test and is reported as `info` rather than a guessed behaviour.
|
||||
- The project's existing test conventions (helper codeunits, fakes, naming pattern, assigned test object ID range) read from the supplied source.
|
||||
|
||||
A curated `testing` knowledge file enters the worklist when its `keywords` intersect the tokens of the target (`Subtype = Test`, `TestType = AITest`, `RequiredTestIsolation`, `TestPermissions`, `HandlerFunctions`, `Library Assert`, `Commit`, isolation, transaction-model). Read its full `## Best Practice` / `## Anti Pattern` body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each target in the worklist, generate an AL test codeunit. Set the codeunit attributes for the test type: `Subtype = Test` with `TestPermissions = Disabled` for a regular AL test; add `TestType = AITest` for a Copilot prompt test; add `RequiredTestIsolation = Disabled` as well for an agent accuracy test. Every generated `[Test]` procedure carries a behaviour-describing name (the `Given_When_Then` or `Behaviour_Should_Outcome` pattern), self-seeding setup with no production-data dependence, an invocation of the target procedure or trigger, at least one `Assert.*` call validating the outcome, and `[HandlerFunctions(...)]` where a modal or confirmation is expected. The generated assertions must fail against the current production code (the TDD red contract); when that cannot be guaranteed for a target, say so plainly in the finding `message`.
|
||||
|
||||
Emit one finding per generated test codeunit. Where a curated `testing` knowledge file backs the generation choice (for example an isolation or transaction-model rule the generated attributes satisfy), cite it: `id` equal to the file path, `references` carrying it, `confidence` `high` for an unambiguous match, `severity` `info` (a generator produces artifacts, not gating defects). Where no curated file applies, emit an agent finding: `references: []`, `id` slug prefixed `agent:` (for example `agent:generated-test-codeunit`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` naming the target, the asserted behaviours, and the developer follow-up (implement the behaviour so the red test goes green). Put the generated AL in `suggested-code` since the artifact is mechanical; the `message` states what was generated and the TDD-red status. When a target lacks a behaviour spec, emit an `info` finding asking for the spec and omit `suggested-code` with `suggested-code-omission-reason` set to `behaviour spec missing for target`. Hold any agent finding to the precision bar in `skills/do.md`.
|
||||
|
||||
Outcome selection: `completed` when every worklist target was processed (including when no AL could be generated for lack of a spec); `no-knowledge` when no curated knowledge survived and no agent finding was raised; `not-applicable` when the task supplies no AL target to cover; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. A populated example:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-test-writer", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 1, "items-evaluated": 1 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:generated-test-codeunit",
|
||||
"severity": "minor",
|
||||
"message": "Generated test codeunit 50202 'Event Registration Tests' covering ReleaseRegistration. Asserts 'Capacity exceeded' is raised when attendee count exceeds capacity. Fails on current code (red); implement the capacity validation in Event Registration Mgt.ReleaseRegistration to make it pass.",
|
||||
"location": { "file": "test/EventRegistrationTests.al" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": "codeunit 50202 \"Event Registration Tests\"\n{\n Subtype = Test;\n TestPermissions = Disabled;\n\n [Test]\n procedure ReleaseRegistrationShouldFailWhenOverCapacity()\n begin\n // ... arrange, act, asserterror\n end;\n}"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
88
custom/skills/testing/al-userguide-test-writer.md
Normal file
88
custom/skills/testing/al-userguide-test-writer.md
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: al-userguide-test-writer
|
||||
version: 1
|
||||
title: AL user-guide test writer
|
||||
description: Maps each USER_GUIDE.md step to a BC page, action, and assertion and emits Subtype=Test TestPage codeunits as findings.
|
||||
inputs: [file-path, repository]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# AL user-guide test writer
|
||||
|
||||
Reads a Business Central extension's end-user walkthrough (typically `USER_GUIDE.md`), maps each documented step to a page plus action plus assertion, and generates `Subtype = Test` codeunits that script the flow with AL's TestPage library so the suite can run in a container. This is a generator-style skill: each generated test codeunit is a finding whose `suggested-code` carries the AL, and the `## Action` step explains what is generated. It writes tests; it does not run them (that is `al-test-runner`). This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `file-path` (the user-guide markdown) and a `repository` (so page object names, action names, field names, and the test app `idRanges` can be resolved from the source). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `testing` or `ux` as the citable candidate set across every enabled layer: TestPage patterns, isolation rules, handler-function conventions, and page-interaction guidance shape the generated codeunits, so a finding matching a curated rule cites it. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Where no curated rule covers a generation choice or a step that cannot be mapped, this skill emits an agent finding within its own domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the repository `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the pages the guide drives, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the generation work the guide implies:
|
||||
|
||||
- One test codeunit per top-level guide section (`"UserGuide §N <Topic>_<SUFFIX>_TST"`, N the section number, suffix matching the extension's mandatory suffix), living in the test app's `idRanges`.
|
||||
- One `[Test]` procedure per documented step or substep, so a failure points at the specific step.
|
||||
- Per procedure: the page to open (`OpenNew()` / `OpenEdit()`), the field writes (display captions resolved to AL field names from the source), the action invocations (`Invoke()` on the AL action name, not the caption), and the assertions on documented outcomes (`Assert.AreEqual` / `Assert.IsTrue` via `Codeunit "Library Assert"`).
|
||||
- `[HandlerFunctions(...)]` where the guide implies a dialog or confirmation, with `asserterror` on negative paths.
|
||||
- Seed data via `LibrarySales` / `LibraryPurchase` / `LibraryInventory` / `LibraryWarehouse` or the extension's own seed library; an `IsInitialized` guard and an `Initialize()` procedure per codeunit.
|
||||
- A coverage map: which steps mapped to which procedures, and which sections could not be mapped (page does not exist, action unreachable from a TestPage, behaviour is server-side with no UI hook).
|
||||
|
||||
A curated `testing` or `ux` file enters the worklist when its `keywords` intersect these tokens. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
For each mappable section, generate a TestPage codeunit as described in the worklist and emit one finding carrying it. Where a curated `testing` or `ux` file backs a generation choice (an isolation rule, a handler-function convention, a TestPage interaction pattern), cite it: `id` equal to the file path, `references` carrying it, `severity` `info`, `confidence` `high` for an unambiguous match. Where no curated file applies, emit an agent finding: `references: []`, `id` slug prefixed `agent:` (for example `agent:generated-userguide-test`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` naming the section, the steps covered, and the file path the codeunit should be written to. Put the generated AL in `suggested-code`. For a section that cannot be cleanly mapped, emit a separate agent finding (`id` slug `agent:unmappable-userguide-step`) naming the section, the step, why it could not map, and the recommended workaround (for example a parent-subpage navigation pattern, or a note that a server-side behaviour needs a non-UI test); omit `suggested-code` and set `suggested-code-omission-reason` to `step cannot be expressed through a TestPage primitive`. When the extension has no seed library, emit an `info` finding recommending one. Hold every agent finding to the precision bar in `skills/do.md`.
|
||||
|
||||
Outcome selection: `completed` when every section was processed (mapped to a codeunit or reported as unmappable); `not-applicable` when the supplied path is not a user guide or no driven page exists in the repository; `no-knowledge` when no curated knowledge survived and no agent finding was raised; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. Generated codeunits with no curated backing are agent findings (`references: []`, `agent:` id, severity capped at `minor`, AL in `suggested-code`); findings citing a `testing` or `ux` file carry that file path as `id` and primary reference.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "al-userguide-test-writer", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 2, "items-evaluated": 2 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:generated-userguide-test",
|
||||
"severity": "minor",
|
||||
"message": "Generated TestPage codeunit for guide section '2. First-time setup', covering steps 2.1 Open Shipping Setup, 2.2 Review seeded reference data, 2.3 Set up Other Places. Write to src/UserGuide/SetupSection_TST.Codeunit.al in the test app id range.",
|
||||
"location": { "file": "src/UserGuide/SetupSection_TST.Codeunit.al" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": "codeunit 60001 \"UserGuide §2 Setup_SHP_EQL_TST\"\n{\n Subtype = Test;\n TestPermissions = Disabled;\n // [Test] procedures per documented step\n}"
|
||||
},
|
||||
{
|
||||
"id": "agent:unmappable-userguide-step",
|
||||
"severity": "minor",
|
||||
"message": "Section 5.1 'Click Add to Container on the line ribbon' is a subpage line-level action. A TestPage cannot invoke it directly; use the parent page's TestPage and a SubPage child reference (Page_PurchaseOrder.PurchLines.\"Add to Container_SHP_EQL\".Invoke()). Confirm the parent-subpage navigation matches AL conventions.",
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "step cannot be expressed through a TestPage primitive"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
93
custom/skills/testing/bc-extension-test-guide.md
Normal file
93
custom/skills/testing/bc-extension-test-guide.md
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: bc-extension-test-guide
|
||||
version: 1
|
||||
title: BC extension test guide generator
|
||||
description: Inventories every page, field, relation, enum, action, state machine, permission set, and telemetry event and emits a category-driven release-audit TEST_GUIDE.md.
|
||||
inputs: [repository, object-list]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# BC extension test guide generator
|
||||
|
||||
Produces an exhaustive `DOCS/TEST_GUIDE.md` for a Business Central AL extension. The guide is exhaustive by construction: every field, relation, action, and reachable data state in the AL source appears in at least one of twelve category inventories, so the state pivots happy-path testing misses are caught. This is a generator-style skill: the generated guide markdown is carried as a finding's `suggested-code`, and the `## Action` step explains what is generated. It produces the artifact; it does not run tests. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `repository` (the extension root with `app.json` and AL `src/`) and optionally an `object-list` (a scope narrowing the inventory). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `testing`, `ux`, `security`, or `telemetry` as the citable candidate set across every enabled layer: the twelve categories pull on lookup, type-conditional relation, visibility-refresh, state-machine, permission-boundary, cross-company isolation, and telemetry-event rules, so a category whose contract matches a curated rule cites it. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The generated guide itself, where no curated rule covers a category, is an agent finding within this skill's domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the repository `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the inventoried objects, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the full AL inventory the twelve categories require. Discover the repo (`app.json`, `.AL-Go/settings.json`, the primary app folder, the `src/**/*.al` file list), then build the inventory across these categories:
|
||||
|
||||
1. Lookup audit: every page field whose underlying table field has a `TableRelation`, both directions.
|
||||
2. Type-conditional `TableRelation`: every relation conditional on a sibling field, and each sibling enum or option value mapped to its target table.
|
||||
3. Eligibility filters: every lookup whose downstream `OnValidate` or OK handler rejects a subset of the target table, and whether the lookup pre-filters it.
|
||||
4. Visibility and Editable conditionals: every dynamic `Visible`/`Editable`, its driver, and whether the driver's `OnValidate` forces a `CurrPage.Update(false)` refresh.
|
||||
5. StandardDialog Mode pivots: every `PageType = StandardDialog` with a Mode selector, the visible-field set, OK side effect, and error per Mode.
|
||||
6. Subpage FK persistence: every `part(...)` with a `SubPageLink`, the FK propagated, and whether the explicit-push pattern or default-value behaviour alone is in use.
|
||||
7. State machine transitions: every status enum, its full from-by-to matrix marking allowed and disallowed transitions, and every code path mutating the status.
|
||||
8. Permission boundaries: every permission set with full RIMD per table and page or codeunit execute claims.
|
||||
9. Telemetry events: every custom-event log call site with event id, trigger, and payload keys.
|
||||
10. Mobile and tablet smoke: every top-level user-facing page.
|
||||
11. Cross-company isolation: every table with its `DataPerCompany` value, singletons called out.
|
||||
12. Upgrade paths: every upgrade codeunit and per-release schema delta with seed instructions.
|
||||
|
||||
Scope the inventory by `object-list` when supplied. A curated `testing`, `ux`, `security`, or `telemetry` file enters the worklist when its `keywords` intersect a category's tokens. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
Generate `DOCS/TEST_GUIDE.md` with exactly the twelve categories in order, each carrying a definition, a procedure, and a populated inventory drawn from the actual AL (no placeholders). Run the self-audit pass: every `TableRelation` lands in category 1 or 2, every dynamic `Visible`/`Editable` in category 4, every `SubPageLink` in category 6, every status enum in category 7, every permission set in category 8, every telemetry call site in category 9; if an inventory has fewer rows than the AL warrants, the guide is incomplete and the missing rows are added. Emit one finding carrying the generated guide. Where a curated file backs a category contract (a state-machine rule, a cross-company isolation rule, a telemetry payload rule), cite it: `id` equal to the file path, `references` carrying it, `severity` `info`, `confidence` `high` for an unambiguous match. Where no curated file applies, emit an agent finding: `references: []`, `id` slug prefixed `agent:` (for example `agent:generated-test-guide`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` stating the guide was generated, the categories that genuinely do not apply (stated explicitly, not stubbed), and the path to write it to. Put the generated markdown in `suggested-code`. Emit an `info` finding recommending the cross-link in `USER_GUIDE.md`. Hold every agent finding to the precision bar in `skills/do.md`.
|
||||
|
||||
Outcome selection: `completed` when the full inventory was built and the guide generated (including categories that do not apply, stated as such); `not-applicable` when the repository has no `app.json` and AL `src/`; `partial` when a token budget truncated the inventory (`summary.coverage` reflects the categories completed); `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. The generated guide with no curated backing is an agent finding (`references: []`, `agent:` id, severity capped at `minor`, markdown in `suggested-code`); findings citing a `testing`, `ux`, `security`, or `telemetry` file carry that file path as `id` and primary reference.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "bc-extension-test-guide", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 1 },
|
||||
"coverage": { "worklist-size": 12, "items-evaluated": 12 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:generated-test-guide",
|
||||
"severity": "minor",
|
||||
"message": "Generated DOCS/TEST_GUIDE.md with all 12 categories populated from src/ (47 TableRelation rows in cat 1/2, 3 status enums in cat 7, 2 permission sets in cat 8). Category 12 (Upgrade paths) is empty: the extension ships no upgrade codeunit, stated explicitly in the guide rather than stubbed.",
|
||||
"location": { "file": "DOCS/TEST_GUIDE.md" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": "# <Extension Name>, Test Guide\n\n## 0. Category index\n... (12 categories, each with definition, procedure, and inventory table) ..."
|
||||
},
|
||||
{
|
||||
"id": "agent:userguide-cross-link",
|
||||
"severity": "info",
|
||||
"message": "Add a cross-link in the USER_GUIDE.md header pointing to the new TEST_GUIDE.md so QA can find the audit companion.",
|
||||
"references": [],
|
||||
"confidence": "medium"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
93
custom/skills/testing/bc-webclient-runner.md
Normal file
93
custom/skills/testing/bc-webclient-runner.md
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: bc-webclient-runner
|
||||
version: 1
|
||||
title: BC web client runner
|
||||
description: Drives the rendered BC web client through a documented flow to catch UI residue AL TestPage cannot observe.
|
||||
inputs: [repository, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# BC web client runner
|
||||
|
||||
Drives a real Business Central web client through a documented user flow (typically `USER_GUIDE.md`), capturing screenshots and asserting on rendered UI state at every step. It catches the class of bug AL TestPage is structurally blind to: page layout, action enable and disable state, FactBox refresh timing, notification toasts, modal stacking, lookup usability, delayed-insert behaviour on subpages, and state-label drift between the guide and the enum. It executes scripted flows; it does not author AL. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `repository` (the extension source, so page and action names and the documented sandbox URL and company can be resolved) and a `file-path` (the user-guide markdown to walk). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `ux` or `testing` as the citable candidate set across every enabled layer: rendered-UI rules (delayed-insert, lookup usability, refresh-after-validate, state-label consistency) can back a finding the run surfaces. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. Where no curated rule covers an observed rendered-UI defect, this skill emits an agent finding within its own domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the repository `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the pages walked, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the flow to drive and the rendered checks per step. The environment must be non-production: refuse an on-prem host lacking `sandbox`, `dev`, `test`, or `staging`, and for a SaaS host on `businesscentral.dynamics.com` inspect the environment-name path segment and refuse if it matches `Production` or starts with `Prod`. Then build the worklist:
|
||||
|
||||
- Each top-level guide section (or the supplied subset), and within it each documented step.
|
||||
- Step-level state checks: read the documented outcome (status pill text, field value, subpage row count) after each action.
|
||||
- Action availability: confirm a button is enabled or disabled exactly as the guide states, reading `aria-disabled` from the accessibility tree.
|
||||
- Notification toasts: screenshot the toast region before auto-dismiss and read its content.
|
||||
- FactBox totals: read the numbers and compare to the documented arithmetic.
|
||||
- Lookup usability on every lookup-bearing field: open the lookup, confirm it lists records and a selection writes back.
|
||||
- Delayed-insert behaviour on every editable subpage: type into the first non-PK field, tab off, and watch for an out-of-filter banner, a blank PK column, or the row falling out of the parent filter.
|
||||
- Missing affordances the guide implies (a lookup drop-down a documented path needs).
|
||||
- State-label drift: compare the displayed status value and enum dropdown values against the names the guide uses.
|
||||
- Page-level errors: any `Error` notification, inline validation message, or console `ServerError`, captured even if the guide does not mention it.
|
||||
|
||||
A curated `ux` or `testing` file enters the worklist when its `keywords` intersect these tokens. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
Drive the web client through each worklisted step, screenshot the result, and assert on the documented outcome. The skill requires a Chrome automation surface in the calling session; if it is unavailable, do not fall back to anything else: emit `outcome: "failed"` with `outcome-reason` stating the surface is missing.
|
||||
|
||||
Emit a finding for every rendered-UI defect. Where a curated `ux` or `testing` file states the rule (for example a delayed-insert rule, a refresh-after-validate rule, or a state-label-consistency rule), emit a knowledge-backed finding citing it: `id` equal to the file path, `severity` up to `major`, `blocker` only when the file states a platform-level guarantee, `confidence` `high` for an unambiguous match. Where no curated file covers the observed defect, emit an agent finding within this skill's domain: `references: []`, `id` slug prefixed `agent:` (for example `agent:userguide-action-disabled`, `agent:subpage-missing-delayed-insert`, `agent:factbox-stale`, `agent:state-label-drift`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` carrying the section, the step, what was observed against what the guide promised, and the screenshot path. When the underlying impact would otherwise be major (a subpage missing `DelayedInsert = true` corrupting the parent FK, or state-label drift that breaks every downstream filter), keep the emitted `severity` at `minor` but say so plainly in the `message` and note the concern should be promoted to a curated rule before it can gate. Record the role the run used and the URL in the summary. Hold every agent finding to the precision bar in `skills/do.md`. The fix lives in AL, not in a renderable replacement, so omit `suggested-code` and set `suggested-code-omission-reason` to `fix is an AL change the developer applies after reading the report`.
|
||||
|
||||
Outcome selection: `completed` when every attempted step was driven and asserted (including a clean run with empty `findings`); `not-applicable` when the supplied path is not a user guide or the repository drives no rendered page; `partial` when a block stopped the run mid-flow and not every section was attempted (`summary.coverage` reflects the attempted subset); `failed` when the Chrome surface was unavailable or the run could not start, with `outcome-reason` required.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. Rendered-UI defects with no curated backing are agent findings (`references: []`, `agent:` id, severity capped at `minor`); findings citing a `ux` or `testing` file carry that file path as `id` and primary reference.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "bc-webclient-runner", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 2, "info": 0 },
|
||||
"coverage": { "worklist-size": 9, "items-evaluated": 9 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:userguide-action-disabled",
|
||||
"severity": "minor",
|
||||
"message": "Section 3.2: Release on the Freight Movement card is disabled although the guide says it should be enabled once the header is filled (aria-disabled=true on the command-bar item). Screenshot: screenshots/section-3-step-2-release-disabled.png. Verify the action's Enabled expression against the header-filled state.",
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "fix is an AL change the developer applies after reading the report"
|
||||
},
|
||||
{
|
||||
"id": "agent:subpage-missing-delayed-insert",
|
||||
"severity": "minor",
|
||||
"message": "Section 5.1: typing into the line subpage then tabbing off shows an out-of-filter banner and a blank No. column, indicating the subpage is missing DelayedInsert = true. Impact is major: OnInsert fires before the number series assigns the PK, so the row persists with a blank or wrong parent FK. Promote to a curated rule before it can gate.",
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code-omission-reason": "fix is an AL change the developer applies after reading the report"
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
79
custom/skills/testing/page-scripting-e2e.md
Normal file
79
custom/skills/testing/page-scripting-e2e.md
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
---
|
||||
kind: action-skill
|
||||
id: page-scripting-e2e
|
||||
version: 1
|
||||
title: Page Scripting e2e planner
|
||||
description: Decides what belongs in Page Scripting versus an AL test and produces a deterministic recording plan wired into the bc-replay harness.
|
||||
inputs: [repository, file-path]
|
||||
outputs: [findings-report]
|
||||
bc-version: [all]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
|
||||
# Page Scripting e2e planner
|
||||
|
||||
Builds a durable, repeatable browser-level test layer for a Business Central extension using BC's native Page Scripting (record and replay `.yml`) plus the `e2e-replay` (bc-replay) harness. It decides what belongs in Page Scripting versus an AL TestPage test, produces a deterministic recording plan a human follows, and wires the recordings so the whole set re-runs from one command. This is a generator-style skill: the recording plan markdown and the seed-factory scaffold are carried as findings' `suggested-code`, and the `## Action` step explains what is generated. This is a leaf action skill: it invokes no sub-skills.
|
||||
|
||||
An orchestrator invokes this skill with a `repository` (the extension source and any green AL suite) and a `file-path` (the `DOCS/TEST_GUIDE.md` or `USER_GUIDE.md` whose residue the plan covers). It produces a single JSON document conforming to the DO output contract.
|
||||
|
||||
## Source
|
||||
|
||||
Read the BCQuality knowledge index once (the `knowledge-index.json` Entry's preparation step regenerates over the live, already-filtered clone). Take the index entries whose `domain` is `testing` or `ux` as the citable candidate set across every enabled layer: rules about what a rendered client must verify (notification toasts, cue rendering, visibility refresh, factbox refresh) and about deterministic seeding can back a plan decision. Do not open individual article files at this step; open an article's full body only once it enters the Worklist below. The generated plan and the layer-allocation decisions, where no curated rule applies, are agent findings within this skill's domain (see Action).
|
||||
|
||||
## Relevance
|
||||
|
||||
Apply the frontmatter matching rules defined in READ against the task context:
|
||||
|
||||
- `bc-version`: the target BC version from the repository `app.json`, or `unknown` if unavailable.
|
||||
- `technologies`: `[al]`.
|
||||
- `countries`: the consuming app's declared countries, or `unknown`.
|
||||
- `application-area`: the application areas of the flows recorded, or `unknown`.
|
||||
|
||||
Discard files that are not applicable. Retain conditionally applicable files (any dimension `unknown`) only when configuration permits; findings derived from them have `confidence` no higher than `medium`, and the finding `message` names the unknown dimensions.
|
||||
|
||||
## Worklist
|
||||
|
||||
Narrow to the residue that needs a rendered client and the recordings that cover it, using the three-layer model: AL TestPage owns logic, state transitions, validation errors, action gates, proportional math, FlowField values, and permission RIMD; Page Scripting owns only what needs a rendered client; the manual checklist owns the irreducible (mobile, subjective look). Build the worklist:
|
||||
|
||||
- For each TEST_GUIDE category or USER_GUIDE flow, decide the layer. Anything verifiable by reading a record or asserting a field after invoking a codeunit stays layer 1 and is excluded here.
|
||||
- The residue items that need a rendered client: notification toasts firing, cue and tile rendering and Style, visibility and editability refresh after a field change, FactBox refresh on row change, dropdown and lookup population and filter-as-you-type narrowing, modal and dialog flow a user clicks through, real posting through standard codeunits driven from the UI.
|
||||
- One small single-purpose recording per residue item (`E2E-NN <flow>.yml`, the `NN` prefix sorting play order).
|
||||
- The deterministic precondition per recording: a Test Seed Factory codeunit gated behind an `Allow Test Data Seed` toggle that clears then seeds a fixed-prefix set, a reset No. Series so a recorded New yields a stable number, an `E2E-00 Clear and Seed.yml` head, and filter-as-you-type lookups that narrow to exactly one row.
|
||||
- The harness wiring: replay via `e2e-replay/run.ps1` or the VS Code task, accounting for the known constraints (bc-replay cannot run from a path with a space, credentials come from env vars not interactive login, the recordings globber rejects `..`).
|
||||
|
||||
A curated `testing` or `ux` file enters the worklist when its `keywords` intersect these tokens. Read its full body only after it makes the worklist. Resolve layer-precedence conflicts per READ and record dropped files in `suppressed`.
|
||||
|
||||
## Action
|
||||
|
||||
Generate `Page Scripting/E2E-PLAN.md`: a numbered chain of small recordings, each with the file name, the exact click-path the human follows while recording (Tell Me, page, field, value, action), the deterministic precondition, and the residue item it verifies. Generate the Test Seed Factory scaffold and the `E2E-00 Clear and Seed.yml` head as supporting artifacts. Emit one finding carrying the plan. Where a curated `testing` or `ux` file backs a residue allocation (a rule that a given behaviour needs a rendered client, or a determinism rule), cite it: `id` equal to the file path, `references` carrying it, `severity` `info`, `confidence` `high` for an unambiguous match. Where no curated file applies, emit an agent finding: `references: []`, `id` slug prefixed `agent:` (for example `agent:generated-e2e-plan`), `confidence` capped at `medium`, `severity` capped at `minor`, and a self-contained `message` listing the residue items, the recordings planned, and the path to write the plan to. Put the generated plan markdown in `suggested-code`. For any flow wrongly placed in layer 2 that an AL test could verify, emit a separate agent finding (`id` slug `agent:belongs-in-al-test`) recommending it stay in the TestPage suite; omit `suggested-code` and set `suggested-code-omission-reason` to `the recommendation is to keep the flow in the AL suite, not to generate a recording`. Hold every agent finding to the precision bar in `skills/do.md`.
|
||||
|
||||
Outcome selection: `completed` when the residue was identified and the plan generated (including when all flows are already covered by layer 1 and no recording is needed); `not-applicable` when the supplied path is not a test or user guide, or the repository has no rendered page to record; `no-knowledge` when no curated knowledge survived and no agent finding was raised; `partial` or `failed` per the DO contract with `outcome-reason`.
|
||||
|
||||
## Output
|
||||
|
||||
Output conforms to the DO output contract. The generated plan with no curated backing is an agent finding (`references: []`, `agent:` id, severity capped at `minor`, markdown in `suggested-code`); findings citing a `testing` or `ux` file carry that file path as `id` and primary reference.
|
||||
|
||||
```json
|
||||
{
|
||||
"skill": { "id": "page-scripting-e2e", "version": 1 },
|
||||
"outcome": "completed",
|
||||
"summary": {
|
||||
"counts": { "blocker": 0, "major": 0, "minor": 1, "info": 0 },
|
||||
"coverage": { "worklist-size": 4, "items-evaluated": 4 }
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"id": "agent:generated-e2e-plan",
|
||||
"severity": "minor",
|
||||
"message": "Generated Page Scripting/E2E-PLAN.md with 4 recordings for the rendered-UI residue the green AL suite cannot reach: notification toast on release, Attention cue turning red at zero, factbox refresh on row change, and the real posting flow driven from the UI. Each recording is anchored to the FE2E- seed prefix with a reset No. Series so replays are deterministic. Write the plan and the Test Seed Factory scaffold into the repo.",
|
||||
"location": { "file": "Page Scripting/E2E-PLAN.md" },
|
||||
"references": [],
|
||||
"confidence": "medium",
|
||||
"suggested-code": "# E2E Page Scripting plan\n\n## E2E-00 Clear and Seed\n... numbered recording chain, each with click-path, precondition, and verified residue ..."
|
||||
}
|
||||
],
|
||||
"suppressed": []
|
||||
}
|
||||
```
|
||||
Loading…
Add table
Add a link
Reference in a new issue