From 51597068b1bc2ac6dda10d3bb1103b0d7da4f4bd Mon Sep 17 00:00:00 2001 From: Jesper Schulz-Wedde Date: Fri, 11 Sep 2026 12:35:31 +0200 Subject: [PATCH 01/19] Add bounded knowledge retrieval (#179) * Add bounded knowledge retrieval Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0d8b7764-f15a-49ea-8d50-d9147334f8af * Fix bc-version overflow and pathless-row identity in bounded retrieval Catalog matching compared an Int32 -BCVersion against a bigint range bound. PowerShell coerces the right operand to the left operand's type, so a bound wider than Int32 threw a conversion error and failed the whole domain catalog rather than the single row. Metadata validation already accepts such bounds, so compare as bigint on both sides. The shared pager built its oversized-row message with $row.path, which throws under Set-StrictMode -Version Latest when a row carries no path, replacing the explicit bound failure with a property-lookup error. Resolve the path defensively for dictionary and object rows so the offset-based fallback is reachable. Both paths gain regression coverage that fails without these fixes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Jesper Schulz-Wedde Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0d8b7764-f15a-49ea-8d50-d9147334f8af --- .github/scripts/Test-KnowledgeIndex.ps1 | 3 + .../skills/review/al-appsource-review.md | 2 +- .../review/al-breaking-changes-review.md | 2 +- microsoft/skills/review/al-code-review.md | 14 +- .../skills/review/al-data-modeling-review.md | 2 +- .../skills/review/al-error-handling-review.md | 2 +- microsoft/skills/review/al-events-review.md | 2 +- .../skills/review/al-interfaces-review.md | 2 +- .../skills/review/al-performance-review.md | 2 +- microsoft/skills/review/al-privacy-review.md | 2 +- microsoft/skills/review/al-query-review.md | 2 +- microsoft/skills/review/al-security-review.md | 2 +- microsoft/skills/review/al-style-review.md | 2 +- .../skills/review/al-telemetry-review.md | 2 +- microsoft/skills/review/al-testing-review.md | 2 +- microsoft/skills/review/al-ui-review.md | 2 +- microsoft/skills/review/al-upgrade-review.md | 2 +- .../skills/review/al-web-services-review.md | 2 +- skills/do.md | 30 + skills/read.md | 55 ++ tools/Bounded-Results.ps1 | 117 +++ tools/Build-KnowledgeIndex.ps1 | 178 +++- tools/Get-KnowledgeArticles.ps1 | 187 +++++ tools/Knowledge-Retrieval.ps1 | 298 +++++++ tools/Search-Knowledge.ps1 | 244 ++++++ tools/Test-KnowledgeRetrieval.ps1 | 788 ++++++++++++++++++ 26 files changed, 1891 insertions(+), 55 deletions(-) create mode 100644 tools/Bounded-Results.ps1 create mode 100644 tools/Get-KnowledgeArticles.ps1 create mode 100644 tools/Knowledge-Retrieval.ps1 create mode 100644 tools/Search-Knowledge.ps1 create mode 100644 tools/Test-KnowledgeRetrieval.ps1 diff --git a/.github/scripts/Test-KnowledgeIndex.ps1 b/.github/scripts/Test-KnowledgeIndex.ps1 index 76896af..1e61064 100644 --- a/.github/scripts/Test-KnowledgeIndex.ps1 +++ b/.github/scripts/Test-KnowledgeIndex.ps1 @@ -16,6 +16,8 @@ 3. Selection-input integrity — every parsed article row carries the non-empty `domain` + `keywords` the worklist predicate selects on, and every article parses (an unparseable article is an invalid file). + 4. Bounded retrieval — delegates to tools/Test-KnowledgeRetrieval.ps1 for + lossless paging, exact-body round trips, and explicit failure cases. Exit code 0 = healthy; non-zero = a problem CI must block on. #> @@ -90,4 +92,5 @@ if ($problems.Count) { exit 1 } Write-Host "Knowledge-index check PASSED: $($rows.Count) articles, deterministic, full coverage, selection inputs intact." -ForegroundColor Green +& (Join-Path $Root 'tools/Test-KnowledgeRetrieval.ps1') -Root $Root exit 0 diff --git a/microsoft/skills/review/al-appsource-review.md b/microsoft/skills/review/al-appsource-review.md index ebbed36..c851ea2 100644 --- a/microsoft/skills/review/al-appsource-review.md +++ b/microsoft/skills/review/al-appsource-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `appsource` as this skill's candidate set across every enabled Microsoft, community, and custom layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/appsource/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain appsource`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-breaking-changes-review.md b/microsoft/skills/review/al-breaking-changes-review.md index cf962ba..7f30713 100644 --- a/microsoft/skills/review/al-breaking-changes-review.md +++ b/microsoft/skills/review/al-breaking-changes-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `breaking-changes` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/breaking-changes/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain breaking-changes`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-code-review.md b/microsoft/skills/review/al-code-review.md index 9972aca..6a577ba 100644 --- a/microsoft/skills/review/al-code-review.md +++ b/microsoft/skills/review/al-code-review.md @@ -65,7 +65,10 @@ The worklist is the list of sub-skills judged relevant by the previous step. Eve The Action step consists of **discrete leaf invocations**, not one combined generation. Invocation scheduling belongs to the orchestrator: independent leaves may run serially or concurrently, but their evaluation contexts and findings-reports remain isolated. Concretely this means: -- **Isolate leaf invocations when the host supports it.** For fast/small models, each sub-skill SHOULD run in a fresh model call or child context containing only the task input, READ/DO contracts, the leaf instructions, a domain-filtered slice of the current knowledge index, and articles that leaf worklists. Preserve each index row's exact `path`; the leaf must copy references from that slice. The coordinator then collects the resulting JSON. This is the preferred fast-model profile: it bounds context, prevents later leaves from being skipped as attention is exhausted, and removes any reason to synthesize article paths. +- **Isolate leaf invocations when the host supports it.** Each sub-skill SHOULD run in a fresh model call or child context containing only its assigned source paths, READ/DO contracts, the leaf instructions, the complete bounded domain catalog per READ, and articles that leaf worklists. Preserve each catalog row's exact `path`; the leaf must copy references from that catalog. +- **Keep run artifacts private.** Before dispatch, allocate a new GUID-named directory under the current session's artifact directory and a distinct scratch/report child directory for every leaf. Pass a leaf only its own assigned source paths and child directory, never the run root or sibling paths. A leaf MUST NOT discover, enumerate, read, modify, or delete sibling artifacts. Do not reuse a prior run directory, and do not clean up any run artifact until every leaf has finished and consolidation is complete. +- **Use the exact Task return as the report.** Capture each leaf's exact return as the primary transport and apply DO's consumer acceptance gate before rollup. Worker-side persistence of the same report in its private directory is optional and redundant; a missing report file does not invalidate an otherwise valid exact return. +- **Treat automatic output spills as host-owned.** If the host reports that a Task return was automatically spilled, the coordinator MAY read that file read-only only at the exact path returned by the tool. Never modify, delete, enumerate around, or reuse an automatic spill path. Never bypass a content-exclusion or access denial. - Treat each sub-skill in the worklist as its own pass: read the sub-skill's instructions, apply its Source → Relevance → Worklist → Action steps to the orchestrator-supplied inputs, and produce that sub-skill's complete findings-report independently. - Do not collapse multiple sub-skills into one shared reasoning step. Each sub-skill has a distinct knowledge subset and a distinct evaluation procedure; sharing one rolled-up scan dilutes per-skill attention and causes leaves to silently underreport (this has been observed in production: leaf skills returned empty `findings[]` while their standalone runs against the same diff produced multiple matches). - The agent self-review pass is its own final iteration. Begin it only after every sub-skill in the worklist has completed and its sub-result is recorded. @@ -77,8 +80,8 @@ The Action step consists of **discrete leaf invocations**, not one combined gene For each sub-skill in the worklist: 1. Invoke the sub-skill with the orchestrator's inputs, passing only the subset each sub-skill declares in its `inputs`. -2. Capture the sub-skill's complete findings-report verbatim and append it to `sub-results`. -3. If the sub-skill's `outcome` is `failed`, stop here for this sub-skill: its findings are not reliable per the DO contract and MUST NOT be copied into the super-skill's top-level `findings[]` or counted in `summary.counts`. +2. Capture the exact Task return and validate it against DO's consumer acceptance gate before accepting it. Preserve an invalid raw return unchanged in the leaf's private artifacts or host log; do not reconstruct or repair it. Record a separate failed validation result with no findings for rollup. +3. Append the accepted findings-report, or the separate failed validation result, to `sub-results`. If its `outcome` is `failed`, stop here for this sub-skill: its findings are not reliable per the DO contract and MUST NOT be copied into the super-skill's top-level `findings[]` or counted in `summary.counts`. 4. Otherwise, compare each entry from the sub-skill's `findings[]` with findings already rolled up. Two findings are duplicates when they point to the same file and overlapping line/range and prescribe materially the same correction, even when their knowledge-file IDs differ. Merge duplicates instead of appending both: keep the more specific domain owner, preserve that finding's optional `domain` field verbatim (including its absence), use its reference as `references[0]` and therefore as `id`, append the other references as supporting references, keep the highest severity and confidence justified by either report, and preserve one self-contained message. Article and leaf ownership notes decide specificity; do not choose by execution order. 5. Append each non-duplicate finding, setting `from-sub-skill` to the sub-skill's `skill.id` and preserving its optional `domain` field verbatim, including its absence. For non-citation findings (those whose `id` is a skill-defined slug rather than a reference path), prefix `id` with `:` to prevent collisions across sub-skills. Other finding fields are preserved. @@ -122,7 +125,10 @@ Calculate `summary.counts` from the final top-level `findings[]`, after failed s Derive `outcome` using the DO rollup rules. `outcome-reason` is populated for `partial` and `failed` and SHOULD summarize per-sub-skill state, for example: *"al-security-review failed (tool timeout); al-performance-review completed."* -Before emitting the rollup, apply DO's reference-integrity gate to every nested and top-level finding. Every knowledge-backed ID/reference path must exist in the live checkout, must have been opened by the producing leaf, and must be copied verbatim rather than synthesized. Treat a sub-result containing an unverifiable citation as failed and exclude its findings from the top-level rollup. +Before emitting the rollup, apply DO's consumer acceptance gate to every nested +and top-level finding. Treat an invalid sub-result as failed and exclude all of +its findings from the top-level rollup. Preserve its exact raw payload +separately; never reconstruct it into a success-shaped report. ## Output diff --git a/microsoft/skills/review/al-data-modeling-review.md b/microsoft/skills/review/al-data-modeling-review.md index 01a983a..05dcd0b 100644 --- a/microsoft/skills/review/al-data-modeling-review.md +++ b/microsoft/skills/review/al-data-modeling-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `data-modeling` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/data-modeling/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain data-modeling`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-error-handling-review.md b/microsoft/skills/review/al-error-handling-review.md index 63c4d5e..56bc0fe 100644 --- a/microsoft/skills/review/al-error-handling-review.md +++ b/microsoft/skills/review/al-error-handling-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `error-handling` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/error-handling/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain error-handling`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-events-review.md b/microsoft/skills/review/al-events-review.md index 559d036..68bcdb0 100644 --- a/microsoft/skills/review/al-events-review.md +++ b/microsoft/skills/review/al-events-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `events` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/events/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain events`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-interfaces-review.md b/microsoft/skills/review/al-interfaces-review.md index 885cd0b..f5d65cd 100644 --- a/microsoft/skills/review/al-interfaces-review.md +++ b/microsoft/skills/review/al-interfaces-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `interfaces` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/interfaces/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain interfaces`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-performance-review.md b/microsoft/skills/review/al-performance-review.md index f2fa80d..664f20d 100644 --- a/microsoft/skills/review/al-performance-review.md +++ b/microsoft/skills/review/al-performance-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `performance` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/performance/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain performance`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-privacy-review.md b/microsoft/skills/review/al-privacy-review.md index 469000c..17b4e7b 100644 --- a/microsoft/skills/review/al-privacy-review.md +++ b/microsoft/skills/review/al-privacy-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `privacy` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/privacy/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain privacy`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-query-review.md b/microsoft/skills/review/al-query-review.md index 3681b23..c52ddd8 100644 --- a/microsoft/skills/review/al-query-review.md +++ b/microsoft/skills/review/al-query-review.md @@ -18,7 +18,7 @@ Reviews AL source changes against the `query` knowledge domain in BCQuality. Thi ## Source -Read `knowledge-index.json` once and take entries whose `domain` is `query` across enabled layers. Open an article body only after it enters the Worklist. If the index is unavailable, discover `*/knowledge/query/*.md` by path. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain query`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-security-review.md b/microsoft/skills/review/al-security-review.md index e3e4049..00e8d10 100644 --- a/microsoft/skills/review/al-security-review.md +++ b/microsoft/skills/review/al-security-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `security` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/security/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain security`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-style-review.md b/microsoft/skills/review/al-style-review.md index 6c8e63c..2703c87 100644 --- a/microsoft/skills/review/al-style-review.md +++ b/microsoft/skills/review/al-style-review.md @@ -22,7 +22,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `style` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/style/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain style`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-telemetry-review.md b/microsoft/skills/review/al-telemetry-review.md index 4b50a9e..1c2046d 100644 --- a/microsoft/skills/review/al-telemetry-review.md +++ b/microsoft/skills/review/al-telemetry-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `telemetry` as this skill's candidate set across every enabled Microsoft, community, and custom layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/telemetry/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain telemetry`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-testing-review.md b/microsoft/skills/review/al-testing-review.md index ed50c46..c96ac83 100644 --- a/microsoft/skills/review/al-testing-review.md +++ b/microsoft/skills/review/al-testing-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `testing` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/testing/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain testing`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-ui-review.md b/microsoft/skills/review/al-ui-review.md index c04a30a..8ffd731 100644 --- a/microsoft/skills/review/al-ui-review.md +++ b/microsoft/skills/review/al-ui-review.md @@ -22,7 +22,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `ui` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/ui/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain ui`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-upgrade-review.md b/microsoft/skills/review/al-upgrade-review.md index 2bb1877..94851a3 100644 --- a/microsoft/skills/review/al-upgrade-review.md +++ b/microsoft/skills/review/al-upgrade-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `upgrade` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/upgrade/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain upgrade`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/microsoft/skills/review/al-web-services-review.md b/microsoft/skills/review/al-web-services-review.md index e818e46..19d0735 100644 --- a/microsoft/skills/review/al-web-services-review.md +++ b/microsoft/skills/review/al-web-services-review.md @@ -20,7 +20,7 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat ## Source -Read the BCQuality knowledge index once — the `knowledge-index.json` BCQuality builds at the root of the knowledge checkout (Entry's preparation step regenerates it over the live, already-filtered clone — see `skills/entry.md`). It lists every article that survived layer and allow/deny filtering and carries, per article, its `path`, `layer`, `domain`, frontmatter dimensions, `keywords`, `title`, and a one-line `description` hint — exactly the fields Relevance and Worklist consume. Take the index entries whose `domain` is `web-services` as this skill's candidate set across every enabled layer; do not open the individual article files at this step. Open an article's full body only once it enters the Worklist below, so a review reads the index plus the handful of worklisted articles instead of every file under `*/knowledge/web-services/**`. +Use READ's **Bounded retrieval for review skills** workflow with `-Domain web-services`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. ## Relevance diff --git a/skills/do.md b/skills/do.md index 4ac433b..f86509b 100644 --- a/skills/do.md +++ b/skills/do.md @@ -159,6 +159,36 @@ The emitted document MUST be strict, valid JSON per [RFC 8259](https://www.rfc-e AL source is the common failure case. Quoted identifiers (for example `Rec."No."`) and multi-line snippets routinely appear in `message`, `suggested-code`, and `suggested-code-omission-reason`, and each embedded quote or newline MUST be escaped when placed in a string value. A `suggested-code` payload that spans several lines is a single JSON string with `\n` separators, not a literal multi-line block. Emit the document as one JSON value with no trailing commentary, and do not rely on the consumer to repair unescaped output. +### Consumer acceptance gate + +The exact action-skill return is the primary report transport. Before accepting +it as a findings-report, a coordinator or host MUST validate it +deterministically: + +1. Parse the exact return as strict JSON and validate every required field, + enum, type, conditional requirement, summary count, coverage value, and + leaf/super-skill constraint against this output contract. +2. For every knowledge-backed finding, verify each `references[].path` is an + exact repo-relative knowledge path that exists in the live BCQuality + snapshot, and verify `findings[].id` exactly equals + `references[0].path`. Verify each path is also present in the coordinator's + recorded set of complete article bodies retrieved for that leaf; catalog + membership alone is insufficient. Keep optional `references[].sha` + separate: it is commit provenance, not an article content hash. +3. For every `location`, verify `file` is an exact source path in the supplied + review scope, the file exists in that source snapshot, and `line` and any + inclusive range identify existing lines with `start-line == line` and + `end-line >= start-line`. + +Validation failure invalidates the complete return; consumers MUST NOT salvage +individual findings, infer missing fields, reconstruct JSON, clamp ranges, +rewrite paths, or otherwise silently repair model output. Preserve the invalid +raw payload unchanged in private run artifacts or host logs. Record a separate +failed validation result for that leaf with no findings, and derive the +super-skill outcome as `partial` or `failed` using the normal rollup rules. +Worker-side report-file persistence is optional and never replaces validation +of the exact return. + ### Field semantics **`outcome`** (required) — diff --git a/skills/read.md b/skills/read.md index f2e9c1e..dddd7db 100644 --- a/skills/read.md +++ b/skills/read.md @@ -149,3 +149,58 @@ The standard workflow for finding applicable files: 4. Resolve conflicts via layer precedence. Steps 1–3 are deterministic; step 4 is applied only when conflicts are detected. + +### Bounded retrieval for review skills + +Resolve `$root` to the BCQuality root, not the reviewed source. Entry prepares +the index once before dispatch; that prepared index is the catalog snapshot and +leaves use it read-only. Catalog retrieval validates the complete index metadata +and returned paths without reopening or rehashing article bodies. Post-Entry +body changes therefore take effect only after Entry rebuilds the index; exact +body retrieval rejects a selected article whose content hash differs from its +prepared row. In one PowerShell tool session, invoke the helpers with `&` so +array arguments remain arrays: + +```powershell +& (Join-Path $root 'tools\Search-Knowledge.ps1') -Domain $domain -Technologies @('al') +& (Join-Path $root 'tools\Get-KnowledgeArticles.ps1') -Paths @($exactPath) +``` + +Pass enabled layers and only task dimensions that are actually known. Catalog +retrieval returns every domain and READ-applicable row: it does not rank, +sample, apply top-k, deduplicate by basename, or omit rows based on query text. +Consume every page by passing `continuation.offset` as `-Offset` and +`continuation.snapshot` as `-Snapshot` with the unchanged request until +`complete` is `true`. Each page repeats request context, defaults, and totals. +An omitted applicability field on a row inherits that page's `defaults`; it +does not mean unknown task context. Preserve every row's exact `path`, `layer`, +complete `keywords`, `title`, one-line `description`, non-default applicability +fields, explicit `applicability`, and `unknownDimensions`. + +Apply the leaf's existing Relevance and Worklist to the complete catalog union. +Split the resulting exact paths into stable chunks of at most eight; never pass +more paths than `-MaxArticles` (whose maximum is eight). Request article bodies +only by one such chunk. Consume every +returned `body`, then request `remainingPaths` with +`continuation.snapshot` as `-Snapshot` until `complete` is `true`, preserving +the other request settings. Continuation is confined to that chunk. Bodies are +original strict UTF-8 text with source byte counts and SHA-256 content hashes; +they are never summarized or truncated. Samples are not loaded unless +requested explicitly with `-Samples` and exact sibling paths; their sibling +article must match its prepared hash and contain the exact READ link. + +The default serialized response limit is 16,000 bytes including its output +newline. Never combine pages or bodies into an unbounded prompt. A malformed or +internally inconsistent prepared index, changed continuation snapshot, selected +article hash mismatch, invalid continuation, unsafe or missing path, invalid +UTF-8, broken sample link, oversized path chunk, or row/envelope that cannot fit +fails explicitly. Entry is the only index preparation point: a leaf does not +rebuild. If PowerShell, a helper, or a valid prepared index is unavailable, +discover exact paths across the enabled domain folders and use native bounded +reads through EOF, validating frontmatter per READ and never treating retrieval +failure as an empty result. + +The helpers' `sha256` and `bytes` fields describe the retrieved file content. +They are not citation provenance. Optional findings `references[].sha` is the +BCQuality commit SHA the skill reviewed; omit it when that provenance is not +available or would misrepresent uncommitted content. diff --git a/tools/Bounded-Results.ps1 b/tools/Bounded-Results.ps1 new file mode 100644 index 0000000..6def944 --- /dev/null +++ b/tools/Bounded-Results.ps1 @@ -0,0 +1,117 @@ +# Shared deterministic paging. Callers build the complete immutable result first. +#requires -Version 7.2 +Set-StrictMode -Version Latest + +function Get-ResultSnapshot { + param([Parameter(Mandatory)] $Value) + + $json = ConvertTo-Json -InputObject $Value -Depth 30 -Compress + return [Convert]::ToHexString( + [Security.Cryptography.SHA256]::HashData([Text.Encoding]::UTF8.GetBytes($json)) + ).ToLowerInvariant() +} + +function Get-SerializedByteCount { + param([Parameter(Mandatory)] [string] $Json) + + # PowerShell writes one platform newline after the returned JSON string. + return [Text.Encoding]::UTF8.GetByteCount($Json) + + [Text.Encoding]::UTF8.GetByteCount([Environment]::NewLine) +} + +function ConvertTo-BoundedPage { + param( + [Parameter(Mandatory)] [Collections.IDictionary] $Header, + [Parameter(Mandatory)] [Collections.IDictionary] $Groups, + [ValidateRange(0, 2147483647)] [int] $Offset = 0, + [string] $Snapshot, + [ValidateRange(1024, 16000)] [int] $MaxBytes = 16000 + ) + + $total = 0 + foreach ($name in $Groups.Keys) { + $total += $Groups[$name].Count + } + if (($total -eq 0 -and $Offset -ne 0) -or ($total -gt 0 -and $Offset -ge $total)) { + throw "Invalid Offset=$Offset for totalCount=$total; no rows were returned." + } + if ($Offset -gt 0 -and -not $Snapshot) { + throw 'Continuation requires Snapshot from the preceding page.' + } + if ($Snapshot -and $Snapshot -cne $Header.snapshot) { + throw 'Snapshot changed or continuation belongs to another request. Discard partial results and restart at Offset=0.' + } + + $page = [ordered]@{} + foreach ($key in $Header.Keys) { + $page[$key] = $Header[$key] + } + $page.offset = $Offset + $page.returnedCount = 0 + $page.totalCount = $total + $page.remainingCount = $total - $Offset + $page.complete = ($total -eq 0) + $page.continuation = if ($total) { + [ordered]@{ offset = $Offset; snapshot = $Header.snapshot } + } + else { + $null + } + foreach ($name in $Groups.Keys) { + $page[$name] = [Collections.Generic.List[object]]::new() + } + + $json = ConvertTo-Json -InputObject $page -Depth 30 -Compress + if ((Get-SerializedByteCount -Json $json) -gt $MaxBytes) { + throw "Page envelope exceeds MaxBytes=$MaxBytes. Use READ's path-discovery fallback; never truncate." + } + + $position = 0 + foreach ($name in $Groups.Keys) { + foreach ($row in $Groups[$name]) { + if ($position++ -lt $Offset) { + continue + } + + $page[$name].Add($row) + $page.returnedCount++ + $page.remainingCount-- + $page.complete = ($page.remainingCount -eq 0) + $page.continuation = if ($page.complete) { + $null + } + else { + [ordered]@{ + offset = $Offset + $page.returnedCount + snapshot = $Header.snapshot + } + } + + $next = ConvertTo-Json -InputObject $page -Depth 30 -Compress + if ((Get-SerializedByteCount -Json $next) -gt $MaxBytes) { + $page[$name].RemoveAt($page[$name].Count - 1) + $page.returnedCount-- + $page.remainingCount++ + $page.complete = $false + $page.continuation = [ordered]@{ + offset = $Offset + $page.returnedCount + snapshot = $Header.snapshot + } + if ($page.returnedCount -eq 0) { + $rowPath = $null + if ($row -is [Collections.IDictionary]) { + if ($row.Contains('path')) { $rowPath = $row['path'] } + } + elseif ($null -ne $row -and $row.PSObject.Properties['path']) { + $rowPath = $row.PSObject.Properties['path'].Value + } + $identity = if ($rowPath) { " at $rowPath" } else { " at Offset=$Offset" } + throw "One complete $name row plus envelope exceeds MaxBytes=$MaxBytes$identity. No row was clipped." + } + return $json + } + $json = $next + } + } + return $json +} diff --git a/tools/Build-KnowledgeIndex.ps1 b/tools/Build-KnowledgeIndex.ps1 index 5835e5d..d246ff3 100644 --- a/tools/Build-KnowledgeIndex.ps1 +++ b/tools/Build-KnowledgeIndex.ps1 @@ -30,6 +30,8 @@ expected to prune its clone to policy first). For provenance and to reproduce a consumer's exact view, pass -EnabledLayers to restrict the walk to those layers and to record the policy in the index header. + Invalid articles are omitted with a path-specific warning so one bad + optional layer article cannot block valid siblings. .PARAMETER BCQualityRoot Path to the BCQuality content root to index (typically a filtered clone). @@ -69,6 +71,17 @@ param( Set-StrictMode -Version Latest $ErrorActionPreference = 'Stop' +. (Join-Path $PSScriptRoot 'Knowledge-Retrieval.ps1') + +if ($PSBoundParameters.ContainsKey('EnabledLayers')) { + if ($null -eq $EnabledLayers) { + throw 'EnabledLayers must be an array; omit it to index all layers.' + } + if (@($EnabledLayers | Where-Object { $_ -cnotin @('microsoft', 'community', 'custom') }).Count -or + @($EnabledLayers | Group-Object -CaseSensitive | Where-Object Count -gt 1).Count) { + throw 'EnabledLayers must contain unique canonical lowercase layer names.' + } +} # Default to the clone root (parent of this script's tools/ folder) so the # agent's Entry preparation step can invoke this with no arguments from the @@ -93,6 +106,45 @@ function Get-RelativePath { return ($rel -replace '\\', '/') } +function Get-BytesSha256 { + param([byte[]] $Bytes) + $sha = [Security.Cryptography.SHA256]::Create() + try { + return ([BitConverter]::ToString($sha.ComputeHash($Bytes)) -replace '-', '').ToLowerInvariant() + } + finally { + $sha.Dispose() + } +} + +function Read-ArticleSource { + param([string] $Path) + $bytes = [IO.File]::ReadAllBytes($Path) + try { + $text = [Text.UTF8Encoding]::new($false, $true).GetString($bytes) + } + catch [Text.DecoderFallbackException] { + throw [IO.InvalidDataException]::new('invalid UTF-8', $_.Exception) + } + return [pscustomobject]@{ + bytes = $bytes + text = $text + sha256 = Get-BytesSha256 -Bytes $bytes + } +} + +function Get-ValueSha256 { + param([Parameter(Mandatory)] $Value) + $bytes = [Text.Encoding]::UTF8.GetBytes((ConvertTo-Json -InputObject $Value -Depth 8 -Compress)) + $sha = [Security.Cryptography.SHA256]::Create() + try { + return ([BitConverter]::ToString($sha.ComputeHash($bytes)) -replace '-', '').ToLowerInvariant() + } + finally { + $sha.Dispose() + } +} + # Trims a Description to a single short line (<= $Max chars) for the lean # index. Takes the first sentence; truncates on a word boundary if still long. function Get-LeanDescription { @@ -116,9 +168,12 @@ function ConvertFrom-ArticleFrontmatter { # Pattern) is included; the index is a lossless substitute for the # frontmatter + Description the worklist predicate reads, not a # substitute for the article's normative guidance. - param([string] $Path) + param( + [string] $Path, + [string] $Text + ) - $lines = Get-Content -LiteralPath $Path -ErrorAction Stop + $lines = [regex]::Split($Text.TrimStart([char]0xfeff), '\r\n|\n|\r') # Frontmatter is the first '---'-delimited block. if ($lines.Count -lt 1 -or $lines[0].Trim() -ne '---') { return $null } @@ -129,19 +184,42 @@ function ConvertFrom-ArticleFrontmatter { if ($fmEnd -lt 0) { return $null } $fm = @{} + $arrayFields = @('bc-version', 'keywords', 'technologies', 'countries', 'application-area') for ($i = 1; $i -lt $fmEnd; $i++) { $line = $lines[$i] if ($line -match '^\s*([a-zA-Z][\w-]*)\s*:\s*(.*)$') { $key = $Matches[1] $val = $Matches[2].Trim() - if ($val -match '^\[(.*)\]$') { + if ($key -in $arrayFields) { + if ($val -notmatch '^\[(.*)\]$') { + throw [IO.InvalidDataException]::new( + "frontmatter field '$key' must use non-empty bracket-array syntax" + ) + } $inner = $Matches[1].Trim() - if ($inner -eq '') { $fm[$key] = @() } - else { $fm[$key] = @($inner -split '\s*,\s*' | ForEach-Object { $_.Trim() }) } + if ($inner -eq '') { + throw [IO.InvalidDataException]::new( + "frontmatter field '$key' must use non-empty bracket-array syntax" + ) + } + $values = @($inner -split '\s*,\s*' | ForEach-Object { $_.Trim() }) + if (@($values | Where-Object { [string]::IsNullOrWhiteSpace($_) }).Count) { + throw [IO.InvalidDataException]::new( + "frontmatter field '$key' must use non-empty bracket-array syntax" + ) + } + $fm[$key] = $values } elseif ($val -ne '') { $fm[$key] = $val } } } + foreach ($field in $arrayFields) { + if (-not $fm.ContainsKey($field) -or $fm[$field] -isnot [array] -or -not $fm[$field].Count) { + throw [IO.InvalidDataException]::new( + "frontmatter field '$field' must use non-empty bracket-array syntax" + ) + } + } # Body parsing: H1 title and the full Description section. The Description # is the article's primary retrieval target per READ and is captured @@ -185,42 +263,71 @@ $indexArticles = [System.Collections.Generic.List[object]]::new() foreach ($layerDir in @('microsoft', 'community', 'custom')) { $kbRoot = Join-Path $BCQualityRoot (Join-Path $layerDir 'knowledge') if (-not (Test-Path $kbRoot)) { continue } - if ($EnabledLayers -and ($EnabledLayers -notcontains $layerDir)) { continue } + if ($EnabledLayers -and ($EnabledLayers -cnotcontains $layerDir)) { continue } - Get-ChildItem -LiteralPath $kbRoot -Recurse -File -Filter '*.md' -ErrorAction SilentlyContinue | - Sort-Object FullName | - ForEach-Object { - $rel = Get-RelativePath -Root $BCQualityRoot -Full $_.FullName - $parsed = $null - try { $parsed = ConvertFrom-ArticleFrontmatter -Path $_.FullName } catch { $parsed = $null } + $files = @( + Get-ChildItem -LiteralPath $kbRoot -Recurse -File -Filter '*.md' -ErrorAction SilentlyContinue | + Sort-Object FullName + ) + foreach ($file in $files) { + $rel = Get-RelativePath -Root $BCQualityRoot -Full $file.FullName + try { + $source = Read-ArticleSource -Path $file.FullName + $parsed = ConvertFrom-ArticleFrontmatter -Path $file.FullName -Text $source.text if (-not $parsed) { - # Invalid/unparseable file: list path + domain-from-path so it - # is never silently dropped from discovery. Consumers fall back - # to reading it in full. - $domainFromPath = if ($rel -match '/knowledge/([^/]+)/') { $Matches[1] } else { '' } - $indexArticles.Add([pscustomobject]@{ - path = $rel; layer = $layerDir; domain = $domainFromPath - 'bc-version' = @(); technologies = @(); countries = @(); 'application-area' = @() - keywords = @(); title = ''; description = ''; parsed = $false - }) | Out-Null - return + throw [IO.InvalidDataException]::new('missing or unterminated frontmatter') + } + foreach ($required in @( + @('domain', $parsed.domain), + @('H1 title', $parsed.title), + @('Description', $parsed.description) + )) { + if ([string]::IsNullOrWhiteSpace([string]$required[1])) { + throw [IO.InvalidDataException]::new("missing $($required[0])") + } } - $indexArticles.Add([pscustomobject]@{ - path = $rel - layer = $layerDir - domain = $parsed.domain - 'bc-version' = @($parsed.'bc-version') - technologies = @($parsed.technologies) - countries = @($parsed.countries) - 'application-area' = @($parsed.'application-area') - keywords = @($parsed.keywords) - title = $parsed.title - description = if ($FullIndex) { $parsed.description } else { Get-LeanDescription -Text $parsed.description } - parsed = $true - }) | Out-Null } + catch [IO.InvalidDataException] { + Write-Warning "Skipping invalid knowledge article '$rel': $($_.Exception.Message)." + continue + } + + $article = [ordered]@{ + path = $rel + layer = $layerDir + domain = $parsed.domain + 'bc-version' = @($parsed.'bc-version') + technologies = @($parsed.technologies) + countries = @($parsed.countries) + 'application-area' = @($parsed.'application-area') + keywords = @($parsed.keywords) + title = $parsed.title + description = if ($FullIndex) { $parsed.description } else { Get-LeanDescription -Text $parsed.description } + parsed = $true + sourceSha256 = $source.sha256 + } + $problem = Get-KnowledgeMetadataProblem -Row $article + if ($problem) { + Write-Warning "Skipping invalid knowledge article '$rel': $problem." + continue + } + $indexArticles.Add($article) | Out-Null + } } +$articlesByPath = [Collections.Generic.Dictionary[string, object]]::new([StringComparer]::Ordinal) +foreach ($article in $indexArticles) { + if (-not $articlesByPath.TryAdd($article.path, $article)) { + throw "Duplicate knowledge path while building source snapshot: $($article.path)" + } +} +$sourcePaths = [string[]]@($articlesByPath.Keys) +[Array]::Sort($sourcePaths, [StringComparer]::Ordinal) +$sourceManifest = @( + foreach ($path in $sourcePaths) { + [ordered]@{ path = $path; sha256 = $articlesByPath[$path].sourceSha256 } + } +) $index = [pscustomobject]@{ version = 1 generatedAt = (Get-Date).ToUniversalTime().ToString('o') @@ -228,6 +335,7 @@ $index = [pscustomobject]@{ knowledgeAllow= @($KnowledgeAllow) knowledgeDeny = @($KnowledgeDeny) articleCount = $indexArticles.Count + sourceSnapshot= Get-ValueSha256 -Value $sourceManifest articles = @($indexArticles) } diff --git a/tools/Get-KnowledgeArticles.ps1 b/tools/Get-KnowledgeArticles.ps1 new file mode 100644 index 0000000..cdd112a --- /dev/null +++ b/tools/Get-KnowledgeArticles.ps1 @@ -0,0 +1,187 @@ +<# +.SYNOPSIS + Reads a bounded prefix of exact article or sample paths without altering bodies. +.DESCRIPTION + The UTF-8 byte size bound covers the complete serialized JSON plus its output + newline. A body that cannot fit fails explicitly; it is never summarized or + truncated. Samples are loaded only with -Samples and must be linked by their + sibling article using READ's exact link convention. +#> +#requires -Version 7.2 +[CmdletBinding()] +param( + [ValidateNotNullOrEmpty()] [string] $BCQualityRoot = (Split-Path $PSScriptRoot -Parent), + [Parameter(Mandatory)] [ValidateNotNullOrEmpty()] [string[]] $Paths, + [ValidateRange(1, 8)] [int] $MaxArticles = 8, + [ValidateRange(1024, 16000)] [int] $MaxBytes = 16000, + [ValidateSet('microsoft', 'community', 'custom')] + [AllowEmptyCollection()] [string[]] $EnabledLayers = @('microsoft', 'community', 'custom'), + [string] $IndexPath, + [ValidatePattern('^[a-f0-9]{64}$')] [string] $Snapshot, + [switch] $Samples +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +. (Join-Path $PSScriptRoot 'Knowledge-Retrieval.ps1') +. (Join-Path $PSScriptRoot 'Bounded-Results.ps1') + +$BCQualityRoot = Resolve-KnowledgeRoot $BCQualityRoot +if (-not $IndexPath) { + $IndexPath = Join-Path $BCQualityRoot 'knowledge-index.json' +} +if ($Paths.Count -gt $MaxArticles) { + throw "Paths count $($Paths.Count) exceeds MaxArticles=$MaxArticles. Split the worklist into stable chunks of at most $MaxArticles exact paths." +} +if ($null -eq $EnabledLayers) { + throw 'EnabledLayers must be an array.' +} +if (@($EnabledLayers | Where-Object { $_ -cnotin @('microsoft', 'community', 'custom') }).Count -or + @($EnabledLayers | Group-Object -CaseSensitive | Where-Object Count -gt 1).Count) { + throw 'EnabledLayers must contain unique canonical lowercase layer names.' +} + +$recovery = "Run Entry preparation once before dispatch, or use READ's bounded native-file fallback. Do not rebuild in a leaf." +$preparedIndex = Read-PreparedKnowledgeIndex -IndexPath $IndexPath -Recovery $recovery +$index = $preparedIndex.index +$byPath = $preparedIndex.byPath +$unrestricted = $index.enabledLayers.Count -eq 0 -or + ($index.enabledLayers.Count -eq 1 -and $null -eq $index.enabledLayers[0]) +$indexedLayers = @( + if ($unrestricted) { 'microsoft', 'community', 'custom' } else { $index.enabledLayers } +) +if (@($EnabledLayers | Where-Object { $_ -cnotin $indexedLayers }).Count) { + throw "Index layer coverage does not cover EnabledLayers. $recovery" +} + +$resolved = [Collections.Generic.List[string]]::new() +$records = [Collections.Generic.List[object]]::new() +$seen = [Collections.Generic.HashSet[string]]::new([StringComparer]::Ordinal) +$articleTexts = [Collections.Generic.Dictionary[string, string]]::new([StringComparer]::Ordinal) +$sampleContents = [Collections.Generic.Dictionary[string, object]]::new([StringComparer]::Ordinal) +foreach ($path in $Paths) { + $kind = if ($Samples) { 'sample' } else { 'article' } + $fullPath = Resolve-KnowledgePath -Root $BCQualityRoot -Path $path -Kind $kind + if ($path.Split('/')[0] -cnotin $EnabledLayers) { + throw "Layer disabled for path: $path" + } + if (-not $seen.Add($path)) { + throw "Duplicate requested path: $path" + } + if ($Samples) { + $articlePath = $path -replace '\.(good|bad)\.[a-z0-9]+$', '.md' + if (-not $byPath.ContainsKey($articlePath)) { + throw "Sample article is absent from the prepared index: $articlePath" + } + $fullArticlePath = Resolve-KnowledgePath -Root $BCQualityRoot -Path $articlePath + if (-not $articleTexts.ContainsKey($articlePath)) { + $articleContent = Read-KnowledgeText -Path $fullArticlePath + if ($articleContent.sha256 -cne $byPath[$articlePath].sourceSha256) { + throw "Selected article hash does not match the prepared index: $articlePath" + } + $articleTexts.Add($articlePath, $articleContent.text) + } + Assert-SampleLink -ArticleText $articleTexts[$articlePath] -SamplePath $fullPath + $sampleContent = Read-KnowledgeText -Path $fullPath + $sampleContents.Add($path, $sampleContent) + $records.Add([ordered]@{ + path = $path + articlePath = $articlePath + articleSha256 = $byPath[$articlePath].sourceSha256 + sampleSha256 = $sampleContent.sha256 + sampleBytes = $sampleContent.bytes + }) + } + else { + if (-not $byPath.ContainsKey($path)) { + throw "Selected article is absent from the prepared index: $path" + } + $records.Add([ordered]@{ + path = $path + expectedSha256 = $byPath[$path].sourceSha256 + }) + } + $resolved.Add($fullPath) +} + +$requestSnapshot = Get-ResultSnapshot -Value ([ordered]@{ + root = $BCQualityRoot + preparedIndexSha256 = $preparedIndex.content.sha256 + kind = if ($Samples) { 'samples' } else { 'articles' } + enabledLayers = @($EnabledLayers) + files = @($records) +}) +if ($Snapshot -and $Snapshot -cne $requestSnapshot) { + throw 'Article snapshot changed or continuation belongs to another exact path batch. Discard partial results and restart.' +} + +$articles = [Collections.Generic.List[object]]::new() +function ConvertTo-BatchJson { + param([int] $ReadCount) + + $remaining = @( + if ($ReadCount -lt $Paths.Count) { + $Paths[$ReadCount..($Paths.Count - 1)] + } + ) + $remainingRecords = @( + if ($ReadCount -lt $records.Count) { + $records[$ReadCount..($records.Count - 1)] + } + ) + $continuation = if ($remaining.Count) { + [ordered]@{ + snapshot = Get-ResultSnapshot -Value ([ordered]@{ + root = $BCQualityRoot + preparedIndexSha256 = $preparedIndex.content.sha256 + kind = if ($Samples) { 'samples' } else { 'articles' } + enabledLayers = @($EnabledLayers) + files = $remainingRecords + }) + } + } + else { + $null + } + return [ordered]@{ + version = 1 + kind = if ($Samples) { 'samples' } else { 'articles' } + snapshot = $requestSnapshot + requestedCount = $Paths.Count + returnedCount = $ReadCount + complete = ($ReadCount -eq $Paths.Count) + articles = @($articles) + remainingPaths = $remaining + continuation = $continuation + } | ConvertTo-Json -Depth 8 -Compress +} + +$json = '' +for ($i = 0; $i -lt [Math]::Min($MaxArticles, $Paths.Count); $i++) { + $content = if ($Samples) { + $sampleContents[$Paths[$i]] + } + else { + Read-KnowledgeText -Path $resolved[$i] + } + if (-not $Samples -and $content.sha256 -cne $records[$i].expectedSha256) { + throw "Selected article hash does not match the prepared index: $($Paths[$i])" + } + $articles.Add([ordered]@{ + path = $Paths[$i] + bytes = $content.bytes + sha256 = $content.sha256 + body = $content.text + }) + $next = ConvertTo-BatchJson -ReadCount ($i + 1) + if ((Get-SerializedByteCount -Json $next) -gt $MaxBytes) { + $articles.RemoveAt($articles.Count - 1) + if ($i -eq 0) { + throw "No complete body plus continuation fits MaxBytes=$MaxBytes at $($Paths[$i]). Use a smaller exact path batch or READ's bounded native-file fallback; never truncate." + } + break + } + $json = $next +} + +$json diff --git a/tools/Knowledge-Retrieval.ps1 b/tools/Knowledge-Retrieval.ps1 new file mode 100644 index 0000000..7007868 --- /dev/null +++ b/tools/Knowledge-Retrieval.ps1 @@ -0,0 +1,298 @@ +# Shared filesystem guards for catalog and exact article retrieval. +Set-StrictMode -Version Latest + +function Resolve-KnowledgeRoot { + param([string] $Root) + + $item = Get-Item -LiteralPath $Root -Force -ErrorAction Stop + if ($item.PSProvider.Name -ne 'FileSystem' -or -not $item.PSIsContainer) { + throw "BCQuality root must be a filesystem directory: $Root" + } + if ($item.Attributes -band [IO.FileAttributes]::ReparsePoint) { + throw "Linked BCQuality roots are not supported: $Root" + } + return $item.FullName +} + +function Assert-KnowledgePath { + param( + [string] $Path, + [ValidateSet('article', 'sample')] [string] $Kind = 'article' + ) + + if ([string]::IsNullOrWhiteSpace($Path) -or + $Path -cnotmatch '^(microsoft|community|custom)/knowledge/[^/]+/.+' -or + $Path -match '[\\:*?"<>|\x00-\x1f]' -or + @($Path.Split('/') | Where-Object { $_ -in '', '.', '..' -or $_ -match '[. ]$' }).Count) { + throw "Invalid knowledge path: $Path" + } + if (($Kind -eq 'article' -and -not $Path.EndsWith('.md', [StringComparison]::Ordinal)) -or + ($Kind -eq 'sample' -and $Path -cnotmatch '\.(good|bad)\.[a-z0-9]+$')) { + throw "Expected an exact $Kind path: $Path" + } +} + +function Resolve-KnowledgePath { + param( + [string] $Root, + [string] $Path, + [ValidateSet('article', 'sample')] [string] $Kind = 'article' + ) + + Assert-KnowledgePath -Path $Path -Kind $Kind + $current = $Root + foreach ($part in $Path.Split('/')) { + $items = @( + Get-ChildItem -LiteralPath $current -Filter $part -Force -ErrorAction Stop | + Where-Object Name -CEQ $part + ) + if ($items.Count -ne 1) { + throw "Knowledge path does not exist with exact casing: $Path" + } + $item = $items[0] + if ($item.Attributes -band [IO.FileAttributes]::ReparsePoint) { + throw "Linked knowledge paths are not supported: $Path" + } + $current = $item.FullName + } + if ($item.PSIsContainer) { + throw "Knowledge path is not a file: $Path" + } + return $item.FullName +} + +function Read-KnowledgeText { + param([string] $Path) + + $bytes = [IO.File]::ReadAllBytes($Path) + try { + $text = [Text.UTF8Encoding]::new($false, $true).GetString($bytes) + } + catch { + throw "Knowledge file is not valid strict UTF-8: $Path" + } + return [pscustomobject]@{ + text = $text + bytes = $bytes.Length + sha256 = [Convert]::ToHexString( + [Security.Cryptography.SHA256]::HashData($bytes) + ).ToLowerInvariant() + } +} + +function Get-NormalizedKnowledgeVersions { + param([string[]] $Values) + + foreach ($value in $Values) { + if ($value -match '^"([^"]*)"$' -or $value -match "^'([^']*)'$") { + $Matches[1] + } + else { + $value + } + } +} + +function Get-KnowledgeMetadataProblem { + param([Collections.IDictionary] $Row) + + if ($Row['parsed'] -isnot [bool] -or -not $Row['parsed']) { + return 'unparsed frontmatter' + } + if ($Row['domain'] -isnot [string] -or + $Row['domain'] -cnotmatch '^[a-z0-9]+(-[a-z0-9]+)*$') { + return 'missing/invalid domain' + } + foreach ($field in @('bc-version', 'technologies', 'countries', 'application-area', 'keywords')) { + if ($Row[$field] -isnot [array] -or -not $Row[$field].Count) { + return "missing/invalid $field" + } + foreach ($value in $Row[$field]) { + if ($value -isnot [string] -or [string]::IsNullOrWhiteSpace($value)) { + return "invalid $field value" + } + } + } + foreach ($field in @('title', 'description')) { + if ($Row[$field] -isnot [string] -or + [string]::IsNullOrWhiteSpace($Row[$field]) -or + $Row[$field] -match '[\r\n]') { + return "missing/invalid $field" + } + } + + $versions = @(Get-NormalizedKnowledgeVersions -Values $Row['bc-version']) + if ($versions -ccontains 'all') { + if ($versions.Count -ne 1) { + return 'mixed bc-version sentinel' + } + } + elseif ($versions.Count -eq 1 -and $versions[0] -match '^(\d+)\.\.(\d+)?$') { + $start = [bigint]::Parse($Matches[1]) + if ($start -le 0 -or ($Matches[2] -and [bigint]::Parse($Matches[2]) -le 0)) { + return 'invalid bc-version range bound' + } + if ($Matches[2] -and $start -gt [bigint]::Parse($Matches[2])) { + return 'descending bc-version range' + } + } + else { + foreach ($version in $versions) { + if ($version -notmatch '^\d+$' -or [bigint]::Parse($version) -le 0) { + return 'invalid bc-version' + } + } + } + if (@($Row.technologies | Where-Object { $_ -cnotmatch '^[a-z0-9]+(-[a-z0-9]+)*$' }).Count) { + return 'invalid technologies' + } + if ($Row.technologies -ccontains 'all') { + return 'invalid technologies sentinel' + } + if ($Row.countries -ccontains 'w1') { + if ($Row.countries.Count -ne 1) { + return 'mixed countries sentinel' + } + } + elseif (@($Row.countries | Where-Object { $_ -cnotmatch '^[a-z]{2}$' }).Count) { + return 'invalid countries' + } + if (@($Row['application-area'] | Where-Object { $_ -cnotmatch '^(all|[a-z0-9]+(-[a-z0-9]+)*)$' }).Count) { + return 'invalid application-area' + } + if ($Row['application-area'] -ccontains 'all' -and $Row['application-area'].Count -ne 1) { + return 'mixed application-area sentinel' + } + if (@($Row.keywords | Where-Object { $_ -cnotmatch '^[a-z0-9]+(-[a-z0-9]+)*$' }).Count) { + return 'invalid keywords' + } + return '' +} + +function Get-PreparedManifestSha256 { + param( + [string[]] $Paths, + [Collections.Generic.Dictionary[string, object]] $ByPath + ) + + $hash = [Security.Cryptography.IncrementalHash]::CreateHash( + [Security.Cryptography.HashAlgorithmName]::SHA256 + ) + try { + $hash.AppendData([byte[]][char]'[') + for ($i = 0; $i -lt $Paths.Count; $i++) { + if ($i) { + $hash.AppendData([byte[]][char]',') + } + $row = [ordered]@{ + path = $Paths[$i] + sha256 = $ByPath[$Paths[$i]].sourceSha256 + } + $hash.AppendData([Text.Encoding]::UTF8.GetBytes( + (ConvertTo-Json -InputObject $row -Depth 8 -Compress) + )) + } + $hash.AppendData([byte[]][char]']') + return [Convert]::ToHexString($hash.GetHashAndReset()).ToLowerInvariant() + } + finally { + $hash.Dispose() + } +} + +function Read-PreparedKnowledgeIndex { + param( + [string] $IndexPath, + [string] $Recovery + ) + + if (-not (Test-Path -LiteralPath $IndexPath -PathType Leaf)) { + throw "Knowledge index missing: $IndexPath. $Recovery" + } + $indexItem = Get-Item -LiteralPath $IndexPath -Force -ErrorAction Stop + if ($indexItem.PSProvider.Name -ne 'FileSystem' -or + ($indexItem.Attributes -band [IO.FileAttributes]::ReparsePoint)) { + throw "Knowledge index must be an unlinked filesystem file: $IndexPath. $Recovery" + } + + $content = Read-KnowledgeText -Path $indexItem.FullName + try { + $index = $content.text.TrimStart([char]0xfeff) | + ConvertFrom-Json -AsHashtable -ErrorAction Stop + } + catch { + throw "Malformed knowledge index JSON: $($_.Exception.Message). $Recovery" + } + if ($index -isnot [Collections.IDictionary] -or + $index.version -ne 1 -or + $index.articles -isnot [array] -or + $index.articleCount -ne $index.articles.Count -or + $index.enabledLayers -isnot [array] -or + $index.knowledgeAllow -isnot [array] -or + $index.knowledgeDeny -isnot [array] -or + $index.sourceSnapshot -isnot [string] -or + $index.sourceSnapshot -cnotmatch '^[a-f0-9]{64}$') { + throw "Invalid knowledge index envelope. $Recovery" + } + + $generatedAt = [DateTimeOffset]::MinValue + if ($index.generatedAt -is [DateTime]) { + $generatedAt = [DateTimeOffset]$index.generatedAt + } + elseif (-not [DateTimeOffset]::TryParse( + [string]$index.generatedAt, + [Globalization.CultureInfo]::InvariantCulture, + [Globalization.DateTimeStyles]::RoundtripKind, + [ref]$generatedAt + )) { + throw "Invalid knowledge index generatedAt. $Recovery" + } + if ($generatedAt -gt [DateTimeOffset]::UtcNow.AddMinutes(1)) { + throw "Invalid knowledge index generatedAt. $Recovery" + } + + $byPath = [Collections.Generic.Dictionary[string, object]]::new([StringComparer]::Ordinal) + foreach ($row in $index.articles) { + if ($row -isnot [Collections.IDictionary] -or $row.path -isnot [string]) { + throw "Index row has no exact path. $Recovery" + } + Assert-KnowledgePath -Path $row.path + if ($row.layer -cne $row.path.Split('/')[0] -or + $row.layer -cnotin @('microsoft', 'community', 'custom')) { + throw "Invalid index layer: $($row.path). $Recovery" + } + if ($row.sourceSha256 -isnot [string] -or + $row.sourceSha256 -cnotmatch '^[a-f0-9]{64}$') { + throw "Invalid source hash in knowledge index: $($row.path). $Recovery" + } + if (-not $byPath.TryAdd($row.path, $row)) { + throw "Duplicate index path: $($row.path). $Recovery" + } + } + + $paths = [string[]]@($byPath.Keys) + [Array]::Sort($paths, [StringComparer]::Ordinal) + if ((Get-PreparedManifestSha256 -Paths $paths -ByPath $byPath) -cne $index.sourceSnapshot) { + throw "Stale or internally inconsistent prepared index snapshot. $Recovery" + } + + return [pscustomobject]@{ + index = $index + content = $content + byPath = $byPath + paths = $paths + } +} + +function Assert-SampleLink { + param( + [string] $ArticleText, + [string] $SamplePath + ) + + $sampleName = [IO.Path]::GetFileName($SamplePath) + $expected = '[`' + $sampleName + '`](' + $sampleName + ')' + if (-not $ArticleText.Contains($expected, [StringComparison]::Ordinal)) { + throw "Sample is not linked by its article using the READ convention: $sampleName" + } +} diff --git a/tools/Search-Knowledge.ps1 b/tools/Search-Knowledge.ps1 new file mode 100644 index 0000000..ade280e --- /dev/null +++ b/tools/Search-Knowledge.ps1 @@ -0,0 +1,244 @@ +<# +.SYNOPSIS + Returns bounded pages of every domain/layer/READ-applicable catalog row. +.DESCRIPTION + Consumes Entry's prepared index read-only. Results are never ranked, sampled, + top-k limited, deduplicated by basename, or narrowed by query text. Omit an + unknown task dimension; an explicit empty array is a known empty set. +#> +#requires -Version 7.2 +[CmdletBinding()] +param( + [ValidateNotNullOrEmpty()] [string] $BCQualityRoot = (Split-Path $PSScriptRoot -Parent), + [Parameter(Mandatory)] [ValidateNotNullOrEmpty()] + [ValidateScript({ -not [string]::IsNullOrWhiteSpace($_) })] [string] $Domain, + [ValidateSet('microsoft', 'community', 'custom')] + [AllowEmptyCollection()] [string[]] $EnabledLayers = @('microsoft', 'community', 'custom'), + [ValidateRange(1, 2147483647)] [int] $BCVersion, + [AllowEmptyCollection()] [string[]] $Technologies, + [AllowEmptyCollection()] [string[]] $Countries, + [AllowEmptyCollection()] [string[]] $ApplicationAreas, + [switch] $ExcludeConditional, + [string] $IndexPath, + [ValidateRange(1024, 16000)] [int] $MaxBytes = 16000, + [ValidateRange(0, 2147483647)] [int] $Offset = 0, + [ValidatePattern('^[a-f0-9]{64}$')] [string] $Snapshot +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +. (Join-Path $PSScriptRoot 'Knowledge-Retrieval.ps1') +. (Join-Path $PSScriptRoot 'Bounded-Results.ps1') + +$BCQualityRoot = Resolve-KnowledgeRoot $BCQualityRoot +if (-not $IndexPath) { + $IndexPath = Join-Path $BCQualityRoot 'knowledge-index.json' +} +if ($null -eq $EnabledLayers) { + throw 'EnabledLayers must be an array; use an empty array to disable all layers.' +} +if (@($EnabledLayers | Where-Object { $_ -cnotin @('microsoft', 'community', 'custom') }).Count -or + @($EnabledLayers | Group-Object -CaseSensitive | Where-Object Count -gt 1).Count) { + throw 'EnabledLayers must contain unique canonical lowercase layer names.' +} + +$context = [ordered]@{} +foreach ($pair in @( + @('BCVersion', 'bc-version'), + @('Technologies', 'technologies'), + @('Countries', 'countries'), + @('ApplicationAreas', 'application-area') +)) { + if (-not $PSBoundParameters.ContainsKey($pair[0])) { + continue + } + $value = $PSBoundParameters[$pair[0]] + if ($null -eq $value) { + throw "Omit unknown context; do not pass null for $($pair[0])." + } + if ($pair[0] -ne 'BCVersion') { + foreach ($entry in $value) { + if ([string]::IsNullOrWhiteSpace($entry) -or $entry -cne $entry.Trim()) { + throw "Invalid context value for $($pair[0]): '$entry'" + } + } + } + $context[$pair[1]] = $value +} +if ($context.Contains('technologies') -and $context['technologies'] -ccontains 'all') { + throw "Technologies has no 'all' sentinel. Omit unknown context." +} + +$recovery = "Run Entry preparation once before dispatch, or use READ's path-discovery fallback. Do not rebuild in a leaf." +$preparedIndex = Read-PreparedKnowledgeIndex -IndexPath $IndexPath -Recovery $recovery +$index = $preparedIndex.index +$indexContent = $preparedIndex.content +$byPath = $preparedIndex.byPath +$paths = $preparedIndex.paths + +# The v1 generator historically serialized an omitted EnabledLayers parameter as [null]. +$unrestricted = $index.enabledLayers.Count -eq 0 -or + ($index.enabledLayers.Count -eq 1 -and $null -eq $index.enabledLayers[0]) +$indexedLayers = @( + if ($unrestricted) { + 'microsoft', 'community', 'custom' + } + else { + $index.enabledLayers + } +) +if (@($indexedLayers | Where-Object { $_ -cnotin @('microsoft', 'community', 'custom') }).Count -or + @($indexedLayers | Group-Object -CaseSensitive | Where-Object Count -gt 1).Count -or + @($EnabledLayers | Where-Object { $_ -cnotin $indexedLayers }).Count) { + throw "Index layer coverage does not cover EnabledLayers. $recovery" +} + +foreach ($path in $paths) { + $row = $byPath[$path] + if ($row.layer -cnotin $indexedLayers) { + throw "Index row layer is outside index coverage: $path. $recovery" + } + $problem = Get-KnowledgeMetadataProblem -Row $row + if ($problem) { + throw "Malformed knowledge index row at ${path}: $problem. $recovery" + } +} + +$defaults = [ordered]@{ + 'bc-version' = @('all') + technologies = @('al') + countries = @('w1') + 'application-area' = @('all') +} +$candidates = [Collections.Generic.List[object]]::new() +$excluded = [Collections.Generic.List[object]]::new() +foreach ($path in $paths) { + $row = $byPath[$path] + if ($row.domain -cne $Domain) { + continue + } + + $unknown = [Collections.Generic.List[string]]::new() + $matchesContext = $true + foreach ($field in $defaults.Keys) { + $values = $row[$field] + if ($field -eq 'bc-version') { + $values = @(Get-NormalizedKnowledgeVersions -Values $values) + } + $sentinel = switch ($field) { + 'bc-version' { 'all' } + 'countries' { 'w1' } + 'application-area' { 'all' } + default { '' } + } + if ($sentinel -and $values -ccontains $sentinel) { + continue + } + if (-not $context.Contains($field)) { + $unknown.Add($field) + continue + } + + $target = $context[$field] + $matched = $false + if ($field -eq 'bc-version') { + # Compare as bigint on both sides: metadata validation accepts bounds + # wider than Int32, and an int left operand would coerce them down. + $targetVersion = [bigint]$target + if ($values.Count -eq 1 -and $values[0] -match '^(\d+)\.\.(\d+)?$') { + $matched = $targetVersion -ge [bigint]::Parse($Matches[1]) -and + (-not $Matches[2] -or $targetVersion -le [bigint]::Parse($Matches[2])) + } + else { + $matched = @($values | Where-Object { [bigint]::Parse($_) -eq $targetVersion }).Count -gt 0 + } + } + else { + $matched = @($values | Where-Object { $target -ccontains $_ }).Count -gt 0 + } + if (-not $matched) { + $matchesContext = $false + break + } + } + if (-not $matchesContext -or ($ExcludeConditional -and $unknown.Count)) { + continue + } + $null = Resolve-KnowledgePath -Root $BCQualityRoot -Path $path + + $candidate = [ordered]@{ + path = $path + layer = $row.layer + keywords = $row.keywords + title = $row.title + description = $row.description + } + foreach ($field in $defaults.Keys) { + if (($row[$field] -join "`0") -cne ($defaults[$field] -join "`0")) { + $candidate[$field] = $row[$field] + } + } + $candidate.applicability = if ($unknown.Count) { 'conditional' } else { 'applicable' } + $candidate.unknownDimensions = @($unknown) + if ($row.layer -cin $EnabledLayers) { + $candidates.Add($candidate) + } + else { + $excluded.Add($candidate) + } +} + +$header = [ordered]@{ + version = 2 + domain = $Domain + context = $context + enabledLayers = @($EnabledLayers) + indexedLayers = @($indexedLayers) + excludeConditional = [bool]$ExcludeConditional + defaults = $defaults + candidateCount = $candidates.Count + excludedByConfigurationCount = $excluded.Count +} + +function Get-CatalogSnapshot { + param( + [string] $PreparedIndexSha256, + [Collections.IDictionary] $Request, + [Collections.IDictionary] $Groups + ) + + $hash = [Security.Cryptography.IncrementalHash]::CreateHash( + [Security.Cryptography.HashAlgorithmName]::SHA256 + ) + try { + foreach ($value in @( + $PreparedIndexSha256, + (ConvertTo-Json -InputObject $Request -Depth 8 -Compress) + )) { + $hash.AppendData([Text.Encoding]::UTF8.GetBytes($value)) + $hash.AppendData([byte[]](10)) + } + foreach ($groupName in $Groups.Keys) { + $hash.AppendData([Text.Encoding]::UTF8.GetBytes("[$groupName]")) + $hash.AppendData([byte[]](10)) + foreach ($row in $Groups[$groupName]) { + $hash.AppendData([Text.Encoding]::UTF8.GetBytes( + (ConvertTo-Json -InputObject $row -Depth 8 -Compress) + )) + $hash.AppendData([byte[]](10)) + } + } + return [Convert]::ToHexString($hash.GetHashAndReset()).ToLowerInvariant() + } + finally { + $hash.Dispose() + } +} + +$groups = [ordered]@{ + candidates = $candidates + excludedByConfiguration = $excluded +} +$header.snapshot = Get-CatalogSnapshot -PreparedIndexSha256 $indexContent.sha256 -Request $header -Groups $groups + +ConvertTo-BoundedPage -Header $header -Groups $groups -Offset $Offset -Snapshot $Snapshot -MaxBytes $MaxBytes diff --git a/tools/Test-KnowledgeRetrieval.ps1 b/tools/Test-KnowledgeRetrieval.ps1 new file mode 100644 index 0000000..ef65812 --- /dev/null +++ b/tools/Test-KnowledgeRetrieval.ps1 @@ -0,0 +1,788 @@ +<# +.SYNOPSIS + Validates lossless bounded catalog and exact-body retrieval. +#> +#requires -Version 7.2 +[CmdletBinding()] +param( + [string] $Root = (Resolve-Path (Join-Path $PSScriptRoot '..')) +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +$Root = (Resolve-Path -LiteralPath $Root).Path + +$generator = Join-Path $Root 'tools/Build-KnowledgeIndex.ps1' +$search = Join-Path $Root 'tools/Search-Knowledge.ps1' +$getArticles = Join-Path $Root 'tools/Get-KnowledgeArticles.ps1' +$utf8 = [Text.UTF8Encoding]::new($false, $true) + +function Assert-True { + param([bool] $Condition, [string] $Message) + if (-not $Condition) { + throw "Assertion failed: $Message" + } +} + +function Assert-Equal { + param($Actual, $Expected, [string] $Message) + if ($Actual -cne $Expected) { + throw "Assertion failed: $Message. Expected '$Expected', got '$Actual'." + } +} + +function Assert-Sequence { + param($Actual, $Expected, [string] $Message) + $actualJson = ConvertTo-Json -InputObject @($Actual) -Compress + $expectedJson = ConvertTo-Json -InputObject @($Expected) -Compress + if ($actualJson -cne $expectedJson) { + throw "Assertion failed: $Message. Expected $expectedJson, got $actualJson." + } +} + +function Assert-Throws { + param([scriptblock] $Action, [string] $Pattern, [string] $Message) + try { + & $Action + } + catch { + if ($_.Exception.Message -notmatch $Pattern) { + throw "Assertion failed: $Message. Wrong error: $($_.Exception.Message)" + } + return + } + throw "Assertion failed: $Message. No error was thrown." +} + +function Get-OutputByteCount { + param([string] $Text) + return [Text.Encoding]::UTF8.GetByteCount($Text) + + [Text.Encoding]::UTF8.GetByteCount([Environment]::NewLine) +} + +function Invoke-CatalogPages { + param( + [hashtable] $Arguments, + [int] $MaxBytes = 4096 + ) + + $allCandidates = [Collections.Generic.List[object]]::new() + $allExcluded = [Collections.Generic.List[object]]::new() + $offset = 0 + $snapshot = '' + $shared = '' + $pageCount = 0 + $lastPage = $null + do { + $pageArguments = @{} + $Arguments + $pageArguments.MaxBytes = $MaxBytes + $pageArguments.Offset = $offset + if ($snapshot) { + $pageArguments.Snapshot = $snapshot + } + $raw = & $search @pageArguments + Assert-True ($raw -is [string]) 'catalog helper emitted exactly one JSON string' + Assert-True ((Get-OutputByteCount -Text $raw) -le $MaxBytes) 'catalog page includes its newline in MaxBytes' + $page = $raw | ConvertFrom-Json + $pageCount++ + Assert-True ($pageCount -le 1000) 'catalog continuation terminates' + Assert-Equal $page.offset $offset 'catalog offset is exact' + Assert-Equal $page.returnedCount (@($page.candidates).Count + @($page.excludedByConfiguration).Count) 'page returnedCount matches rows' + Assert-Equal $page.remainingCount ($page.totalCount - $offset - $page.returnedCount) 'page remainingCount is exact' + + $currentShared = [ordered]@{ + version = $page.version + domain = $page.domain + context = $page.context + enabledLayers = $page.enabledLayers + indexedLayers = $page.indexedLayers + excludeConditional = $page.excludeConditional + defaults = $page.defaults + candidateCount = $page.candidateCount + excludedByConfigurationCount = $page.excludedByConfigurationCount + snapshot = $page.snapshot + totalCount = $page.totalCount + } | ConvertTo-Json -Depth 8 -Compress + if (-not $shared) { + $shared = $currentShared + $snapshot = $page.snapshot + } + else { + Assert-Equal $currentShared $shared 'catalog pages repeat shared context, defaults, totals, and snapshot' + } + + foreach ($row in @($page.candidates)) { + $allCandidates.Add($row) + } + foreach ($row in @($page.excludedByConfiguration)) { + $allExcluded.Add($row) + } + if (-not $page.complete) { + Assert-True ($null -ne $page.continuation) 'incomplete page has continuation' + Assert-Equal $page.continuation.snapshot $snapshot 'continuation is snapshot-bound' + Assert-True ($page.continuation.offset -gt $offset) 'continuation makes progress' + $offset = $page.continuation.offset + } + $lastPage = $page + } while (-not $page.complete) + + Assert-True ($null -eq $lastPage.continuation) 'final page has no continuation' + Assert-Equal $allCandidates.Count $lastPage.candidateCount 'candidate total survives paging' + Assert-Equal $allExcluded.Count $lastPage.excludedByConfigurationCount 'excluded total survives paging' + return [pscustomobject]@{ + candidates = @($allCandidates) + excluded = @($allExcluded) + pages = $pageCount + snapshot = $snapshot + lastPage = $lastPage + } +} + +function Test-BodyRoundTrip { + param( + [string[]] $Paths, + [string] $IndexPath, + [switch] $Samples + ) + + $seen = [Collections.Generic.HashSet[string]]::new([StringComparer]::Ordinal) + for ($start = 0; $start -lt $Paths.Count; $start += 8) { + $end = [Math]::Min($start + 7, $Paths.Count - 1) + $remaining = @($Paths[$start..$end]) + $snapshot = '' + do { + $arguments = @{ + BCQualityRoot = $Root + IndexPath = $IndexPath + Paths = $remaining + MaxArticles = 8 + MaxBytes = 16000 + } + if ($Samples) { + $arguments.Samples = $true + } + if ($snapshot) { + $arguments.Snapshot = $snapshot + } + $raw = & $getArticles @arguments + Assert-True ($raw -is [string]) 'article helper emitted exactly one JSON string' + Assert-True ((Get-OutputByteCount -Text $raw) -le 16000) 'article batch includes its newline in MaxBytes' + $batch = $raw | ConvertFrom-Json + Assert-True ($batch.returnedCount -gt 0) 'article batching makes progress' + Assert-Equal $batch.returnedCount @($batch.articles).Count 'article returnedCount matches rows' + Assert-Equal $batch.complete (@($batch.remainingPaths).Count -eq 0) 'article completion matches remaining paths' + if ($batch.complete) { + Assert-True ($null -eq $batch.continuation) 'complete article batch has no continuation' + } + else { + Assert-True ($batch.continuation.snapshot -match '^[a-f0-9]{64}$') 'article continuation is snapshot-bound' + } + + foreach ($article in @($batch.articles)) { + Assert-True ($seen.Add($article.path)) "body returned once: $($article.path)" + $fullPath = Join-Path $Root ($article.path.Replace('/', [IO.Path]::DirectorySeparatorChar)) + $bytes = [IO.File]::ReadAllBytes($fullPath) + $text = $utf8.GetString($bytes) + $hash = [Convert]::ToHexString( + [Security.Cryptography.SHA256]::HashData($bytes) + ).ToLowerInvariant() + Assert-Equal $article.bytes $bytes.Length "byte count round-trips: $($article.path)" + Assert-Equal $article.sha256 $hash "SHA-256 round-trips: $($article.path)" + Assert-Equal $article.body $text "body round-trips: $($article.path)" + } + $remaining = @($batch.remainingPaths) + $snapshot = if ($batch.complete) { '' } else { $batch.continuation.snapshot } + } while ($remaining.Count) + } + Assert-Equal $seen.Count $Paths.Count 'every requested body round-trips without loss' +} + +function New-NeutralArticle { + param( + [string] $FixtureRoot, + [string] $Layer, + [string] $Slug, + [string] $Version = 'all', + [string] $Technology = 'al', + [string] $Country = 'w1', + [string] $Area = 'all', + [string] $Title = 'Neutral retrieval example', + [string] $Description = 'Neutral retrieval metadata for deterministic tests.' + ) + + $directory = Join-Path $FixtureRoot "$Layer\knowledge\neutral" + New-Item -ItemType Directory -Force -Path $directory | Out-Null + $content = @" +--- +bc-version: [$Version] +domain: neutral +keywords: [neutral, retrieval, deterministic] +technologies: [$Technology] +countries: [$Country] +application-area: [$Area] +--- + +# $Title + +## Description + +$Description +"@ + Set-Content -LiteralPath (Join-Path $directory "$Slug.md") -Value $content -Encoding utf8NoBOM +} + +function Test-InvalidSourceIndexing { + param( + [string] $FixtureRoot, + [string] $Field, + [string] $ValidValue, + [string] $InvalidValue + ) + + New-NeutralArticle -FixtureRoot $FixtureRoot -Layer microsoft -Slug valid-source + New-NeutralArticle -FixtureRoot $FixtureRoot -Layer community -Slug invalid-source + $articlePath = Join-Path $FixtureRoot 'community\knowledge\neutral\invalid-source.md' + $text = [IO.File]::ReadAllText($articlePath, $utf8) + $text = $text.Replace("$Field`: $ValidValue", "$Field`: $InvalidValue") + [IO.File]::WriteAllText($articlePath, $text, $utf8) + + $indexPath = Join-Path (Split-Path $FixtureRoot -Parent) ("$Field-index.json") + $generation = @(& $generator -BCQualityRoot $FixtureRoot -IndexPath $indexPath 3>&1) + $warnings = @($generation | Where-Object { $_ -is [Management.Automation.WarningRecord] }) + Assert-Equal $warnings.Count 1 "scalar $Field source emits one omission warning" + Assert-True ( + $warnings[0].Message -match + "Skipping invalid knowledge article 'community/knowledge/neutral/invalid-source\.md': frontmatter field '$([regex]::Escape($Field))' must use non-empty bracket-array syntax\." + ) "scalar $Field warning identifies the exact path and reason" + $prepared = Get-Content -LiteralPath $indexPath -Raw -Encoding utf8 | ConvertFrom-Json + Assert-Equal $prepared.articleCount 1 "scalar $Field source is omitted while its valid sibling is indexed" + Assert-Sequence $prepared.articles.path @('microsoft/knowledge/neutral/valid-source.md') "scalar $Field index contains only the valid sibling" + $catalog = & $search -BCQualityRoot $FixtureRoot -IndexPath $indexPath -Domain neutral | + ConvertFrom-Json + Assert-Sequence $catalog.candidates.path @('microsoft/knowledge/neutral/valid-source.md') "scalar $Field catalog retrieves the valid sibling" + $valid = & $getArticles -BCQualityRoot $FixtureRoot -IndexPath $indexPath ` + -Paths 'microsoft/knowledge/neutral/valid-source.md' | + ConvertFrom-Json + Assert-True $valid.complete "scalar $Field valid sibling body retrieves completely" + Assert-Throws { + & $getArticles -BCQualityRoot $FixtureRoot -IndexPath $indexPath ` + -Paths 'community/knowledge/neutral/invalid-source.md' + } 'Selected article is absent from the prepared index' "scalar $Field omitted source cannot be retrieved" +} + +function Test-InvalidSemanticIndexing { + param( + [string] $FixtureRoot, + [string] $CaseName, + [string] $Field, + [string] $ValidValue, + [string] $InvalidValue, + [string] $ExpectedReason + ) + + New-NeutralArticle -FixtureRoot $FixtureRoot -Layer microsoft -Slug valid-source + New-NeutralArticle -FixtureRoot $FixtureRoot -Layer community -Slug invalid-source + $articlePath = Join-Path $FixtureRoot 'community\knowledge\neutral\invalid-source.md' + $text = [IO.File]::ReadAllText($articlePath, $utf8) + $text = $text.Replace("$Field`: $ValidValue", "$Field`: $InvalidValue") + [IO.File]::WriteAllText($articlePath, $text, $utf8) + + $indexPath = Join-Path (Split-Path $FixtureRoot -Parent) ("$CaseName-index.json") + $generation = @(& $generator -BCQualityRoot $FixtureRoot -IndexPath $indexPath 3>&1) + $warnings = @($generation | Where-Object { $_ -is [Management.Automation.WarningRecord] }) + Assert-Equal $warnings.Count 1 "$CaseName emits one omission warning" + Assert-Equal $warnings[0].Message "Skipping invalid knowledge article 'community/knowledge/neutral/invalid-source.md': $ExpectedReason." "$CaseName warning identifies exact path and reason" + + $prepared = Get-Content -LiteralPath $indexPath -Raw -Encoding utf8 | ConvertFrom-Json + Assert-Equal $prepared.articleCount 1 "$CaseName omits invalid source and retains valid sibling" + Assert-Sequence $prepared.articles.path @('microsoft/knowledge/neutral/valid-source.md') "$CaseName index contains only valid sibling" + Assert-True ($prepared.sourceSnapshot -match '^[a-f0-9]{64}$') "$CaseName source snapshot remains valid" + $validRow = $prepared.articles[0] + $manifest = @( + [ordered]@{ path = $validRow.path; sha256 = $validRow.sourceSha256 } + ) + $manifestBytes = [Text.Encoding]::UTF8.GetBytes( + (ConvertTo-Json -InputObject $manifest -Depth 8 -Compress) + ) + $expectedSnapshot = [Convert]::ToHexString( + [Security.Cryptography.SHA256]::HashData($manifestBytes) + ).ToLowerInvariant() + Assert-Equal $prepared.sourceSnapshot $expectedSnapshot "$CaseName source snapshot covers only retained rows" + + $catalog = & $search -BCQualityRoot $FixtureRoot -IndexPath $indexPath -Domain neutral | + ConvertFrom-Json + Assert-Sequence $catalog.candidates.path @('microsoft/knowledge/neutral/valid-source.md') "$CaseName catalog retains valid sibling" + $valid = & $getArticles -BCQualityRoot $FixtureRoot -IndexPath $indexPath ` + -Paths 'microsoft/knowledge/neutral/valid-source.md' | + ConvertFrom-Json + Assert-True $valid.complete "$CaseName valid sibling body retrieves" + Assert-Throws { + & $getArticles -BCQualityRoot $FixtureRoot -IndexPath $indexPath ` + -Paths 'community/knowledge/neutral/invalid-source.md' + } 'Selected article is absent from the prepared index' "$CaseName invalid source cannot be retrieved" +} + +function Test-InvalidEnabledLayers { + param( + [string] $FixtureRoot, + [string] $CaseName, + $Layers, + [string] $ExpectedPattern + ) + + $indexPath = Join-Path (Split-Path $FixtureRoot -Parent) ("layers-$CaseName.json") + $arguments = @{ + BCQualityRoot = $FixtureRoot + IndexPath = $indexPath + EnabledLayers = $Layers + } + Assert-Throws { + & $generator @arguments + } $ExpectedPattern "$CaseName EnabledLayers fails" + Assert-True (-not (Test-Path -LiteralPath $indexPath)) "$CaseName fails before index creation" +} + +$tmp = Join-Path ([IO.Path]::GetTempPath()) ("bcquality_retrieval_" + [guid]::NewGuid().ToString('N')) +New-Item -ItemType Directory -Force -Path $tmp | Out-Null +try { + $indexPath = Join-Path $tmp 'knowledge-index.json' + & $generator -BCQualityRoot $Root -IndexPath $indexPath | Out-Null + $index = Get-Content -LiteralPath $indexPath -Raw -Encoding utf8 | ConvertFrom-Json + $diskArticlePaths = @( + foreach ($layer in 'microsoft', 'community', 'custom') { + $knowledge = Join-Path $Root "$layer\knowledge" + if (Test-Path -LiteralPath $knowledge) { + Get-ChildItem -LiteralPath $knowledge -Recurse -File -Filter '*.md' | + ForEach-Object { + [IO.Path]::GetRelativePath($Root, $_.FullName).Replace('\', '/') + } + } + } + ) | Sort-Object + Assert-Equal $index.articleCount $diskArticlePaths.Count 'index covers every current article' + Assert-True ($index.sourceSnapshot -match '^[a-f0-9]{64}$') 'index carries an exact source snapshot' + + $allCatalogRows = [Collections.Generic.List[object]]::new() + $domains = @($index.articles.domain | Sort-Object -Unique) + foreach ($domain in $domains) { + $catalog = Invoke-CatalogPages -Arguments @{ + BCQualityRoot = $Root + IndexPath = $indexPath + Domain = $domain + } + Assert-Equal $catalog.excluded.Count 0 "all layers enabled for $domain" + foreach ($row in $catalog.candidates) { + $allCatalogRows.Add($row) + } + } + + $expectedRows = @($index.articles | Sort-Object path) + $actualRows = @($allCatalogRows | Sort-Object path) + Assert-Equal $actualRows.Count $expectedRows.Count 'paged union has no top-k or query-based loss' + Assert-Sequence ($actualRows.path) ($expectedRows.path) 'paged union equals all READ-filtered candidates' + Assert-Equal @($actualRows.path | Sort-Object -Unique).Count $actualRows.Count 'catalog does not deduplicate distinct paths' + + $defaults = [ordered]@{ + 'bc-version' = @('all') + technologies = @('al') + countries = @('w1') + 'application-area' = @('all') + } + for ($i = 0; $i -lt $actualRows.Count; $i++) { + $actual = $actualRows[$i] + $expected = $expectedRows[$i] + Assert-Equal $actual.path $expected.path 'catalog preserves exact path' + Assert-Equal $actual.layer $expected.layer 'catalog preserves layer' + Assert-Sequence $actual.keywords $expected.keywords 'catalog preserves full keywords' + Assert-Equal $actual.title $expected.title 'catalog preserves title' + Assert-Equal $actual.description $expected.description 'catalog preserves one-line description' + + $unknown = [Collections.Generic.List[string]]::new() + foreach ($field in $defaults.Keys) { + $expectedValues = @($expected.$field) + $sentinel = switch ($field) { + 'bc-version' { 'all' } + 'countries' { 'w1' } + 'application-area' { 'all' } + default { '' } + } + if (-not $sentinel -or $expectedValues -notcontains $sentinel) { + $unknown.Add($field) + } + $hasField = $actual.PSObject.Properties.Name -ccontains $field + if (($expectedValues -join "`0") -ceq (@($defaults[$field]) -join "`0")) { + Assert-True (-not $hasField) "default field is inherited from page: $field" + } + else { + Assert-True $hasField "non-default field survives paging: $field" + Assert-Sequence $actual.$field $expectedValues "non-default field is exact: $field" + } + } + Assert-Equal $actual.applicability ($(if ($unknown.Count) { 'conditional' } else { 'applicable' })) 'applicability verdict is explicit' + Assert-Sequence $actual.unknownDimensions @($unknown) 'unknown dimensions are explicit' + } + + $performanceFirst = & $search -BCQualityRoot $Root -IndexPath $indexPath -Domain performance -MaxBytes 4096 | + ConvertFrom-Json + Assert-True (-not $performanceFirst.complete) 'large domain produces deterministic continuation' + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $indexPath -Domain performance -MaxBytes 4096 -Offset $performanceFirst.continuation.offset + } 'Continuation requires Snapshot' 'continuation without snapshot fails' + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $indexPath -Domain performance -Offset $performanceFirst.totalCount + } 'Invalid Offset' 'offset at total fails' + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $indexPath -Domain performance -Offset 1 -Snapshot ('0' * 64) + } 'Snapshot changed' 'wrong snapshot fails' + + $changedRawIndex = Join-Path $tmp 'changed-raw-index.json' + $changedRaw = Get-Content -LiteralPath $indexPath -Raw -Encoding utf8 | ConvertFrom-Json + $changedRaw.generatedAt = [DateTimeOffset]::UtcNow.ToString('O') + $changedRaw | ConvertTo-Json -Depth 8 -Compress | + Set-Content -LiteralPath $changedRawIndex -Encoding utf8NoBOM -NoNewline + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $changedRawIndex -Domain performance ` + -MaxBytes 4096 -Offset $performanceFirst.continuation.offset ` + -Snapshot $performanceFirst.continuation.snapshot + } 'Snapshot changed' 'continuation is bound to the exact prepared index bytes' + + $malformedIndex = Join-Path $tmp 'malformed.json' + Set-Content -LiteralPath $malformedIndex -Value '{not-json' -Encoding utf8NoBOM + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $malformedIndex -Domain performance + } 'Malformed knowledge index JSON' 'malformed JSON fails' + + $unsafeIndex = Join-Path $tmp 'unsafe.json' + $unsafe = Get-Content -LiteralPath $indexPath -Raw -Encoding utf8 | ConvertFrom-Json + $unsafe.articles[0].path = '../outside.md' + $unsafe | ConvertTo-Json -Depth 8 -Compress | + Set-Content -LiteralPath $unsafeIndex -Encoding utf8NoBOM + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $unsafeIndex -Domain performance + } 'Invalid knowledge path' 'unsafe indexed path fails' + + $invalidRowIndex = Join-Path $tmp 'invalid-row.json' + $invalidRow = Get-Content -LiteralPath $indexPath -Raw -Encoding utf8 | ConvertFrom-Json + $invalidRow.articles[0].keywords = @() + $invalidRow | ConvertTo-Json -Depth 8 -Compress | + Set-Content -LiteralPath $invalidRowIndex -Encoding utf8NoBOM + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $invalidRowIndex -Domain performance + } 'Malformed knowledge index row' 'malformed index row fails' + + $semanticCorruptIndex = Join-Path $tmp 'semantic-corrupt-row.json' + $semanticCorrupt = Get-Content -LiteralPath $indexPath -Raw -Encoding utf8 | ConvertFrom-Json + $semanticCorrupt.articles[0].countries = @('usa') + $semanticCorrupt | ConvertTo-Json -Depth 8 -Compress | + Set-Content -LiteralPath $semanticCorruptIndex -Encoding utf8NoBOM + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $semanticCorruptIndex -Domain performance + } 'Malformed knowledge index row.*invalid countries' 'search rejects semantically invalid external index rows' + + $corruptSemanticCases = @( + @{ name = 'uppercase-all'; field = 'bc-version'; value = @('ALL'); reason = 'invalid bc-version' }, + @{ name = 'uppercase-w1'; field = 'countries'; value = @('W1'); reason = 'invalid countries' }, + @{ name = 'zero-open-range'; field = 'bc-version'; value = @('"0.."'); reason = 'invalid bc-version range bound' }, + @{ name = 'zero-closed-range'; field = 'bc-version'; value = @('"0..0"'); reason = 'invalid bc-version range bound' } + ) + foreach ($case in $corruptSemanticCases) { + $corruptPath = Join-Path $tmp ("corrupt-$($case.name).json") + $corrupt = Get-Content -LiteralPath $indexPath -Raw -Encoding utf8 | ConvertFrom-Json + $corrupt.articles[0].PSObject.Properties[$case.field].Value = $case.value + $corrupt | ConvertTo-Json -Depth 8 -Compress | + Set-Content -LiteralPath $corruptPath -Encoding utf8NoBOM + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $corruptPath -Domain performance + } "Malformed knowledge index row.*$([regex]::Escape($case.reason))" "search rejects $($case.name) in an external index" + } + + $invalidUtf8Root = Join-Path $tmp 'invalid-utf8-source' + New-NeutralArticle -FixtureRoot $invalidUtf8Root -Layer microsoft -Slug valid-catalog + New-NeutralArticle -FixtureRoot $invalidUtf8Root -Layer community -Slug invalid-utf8-source + $invalidUtf8Article = Join-Path $invalidUtf8Root 'community\knowledge\neutral\invalid-utf8-source.md' + $validBytes = [IO.File]::ReadAllBytes($invalidUtf8Article) + [IO.File]::WriteAllBytes($invalidUtf8Article, [byte[]]@($validBytes + @(0xc3, 0x28))) + $invalidUtf8Index = Join-Path $tmp 'invalid-utf8-index.json' + $generation = @(& $generator -BCQualityRoot $invalidUtf8Root -IndexPath $invalidUtf8Index 3>&1) + $warnings = @($generation | Where-Object { $_ -is [Management.Automation.WarningRecord] }) + Assert-Equal $warnings.Count 1 'malformed UTF-8 source emits one omission warning' + Assert-Equal $warnings[0].Message "Skipping invalid knowledge article 'community/knowledge/neutral/invalid-utf8-source.md': invalid UTF-8." 'malformed UTF-8 warning identifies the exact path and reason' + $invalidUtf8Prepared = Get-Content -LiteralPath $invalidUtf8Index -Raw -Encoding utf8 | + ConvertFrom-Json + Assert-Equal $invalidUtf8Prepared.articleCount 1 'malformed UTF-8 source is omitted while its valid sibling is indexed' + Assert-Sequence $invalidUtf8Prepared.articles.path @('microsoft/knowledge/neutral/valid-catalog.md') 'malformed UTF-8 index contains only the valid sibling' + $validCatalog = & $search -BCQualityRoot $invalidUtf8Root -IndexPath $invalidUtf8Index -Domain neutral | + ConvertFrom-Json + Assert-Sequence $validCatalog.candidates.path @('microsoft/knowledge/neutral/valid-catalog.md') 'catalog retrieves the valid sibling after malformed UTF-8 omission' + $validBody = & $getArticles -BCQualityRoot $invalidUtf8Root -IndexPath $invalidUtf8Index ` + -Paths 'microsoft/knowledge/neutral/valid-catalog.md' | + ConvertFrom-Json + Assert-True $validBody.complete 'valid sibling body retrieves after malformed UTF-8 omission' + Assert-Throws { + & $getArticles -BCQualityRoot $invalidUtf8Root -IndexPath $invalidUtf8Index ` + -Paths 'community/knowledge/neutral/invalid-utf8-source.md' + } 'Selected article is absent from the prepared index' 'omitted malformed UTF-8 source cannot be retrieved' + + $scalarCases = @( + @{ field = 'bc-version'; valid = '[all]'; invalid = 'all' }, + @{ field = 'keywords'; valid = '[neutral, retrieval, deterministic]'; invalid = 'neutral' }, + @{ field = 'technologies'; valid = '[al]'; invalid = 'al' }, + @{ field = 'countries'; valid = '[w1]'; invalid = 'w1' }, + @{ field = 'application-area'; valid = '[all]'; invalid = 'all' } + ) + foreach ($case in $scalarCases) { + Test-InvalidSourceIndexing -FixtureRoot (Join-Path $tmp "scalar-$($case.field)") ` + -Field $case.field -ValidValue $case.valid -InvalidValue $case.invalid + } + + $semanticCases = @( + @{ name = 'mixed-version-sentinel'; field = 'bc-version'; valid = '[all]'; invalid = '[all, 27]'; reason = 'mixed bc-version sentinel' }, + @{ name = 'invalid-country'; field = 'countries'; valid = '[w1]'; invalid = '[usa]'; reason = 'invalid countries' }, + @{ name = 'descending-version-range'; field = 'bc-version'; valid = '[all]'; invalid = '["28..27"]'; reason = 'descending bc-version range' }, + @{ name = 'malformed-version-range'; field = 'bc-version'; valid = '[all]'; invalid = '[twenty-seven]'; reason = 'invalid bc-version' }, + @{ name = 'malformed-keyword'; field = 'keywords'; valid = '[neutral, retrieval, deterministic]'; invalid = '[neutral, Bad_Token, deterministic]'; reason = 'invalid keywords' }, + @{ name = 'malformed-technology'; field = 'technologies'; valid = '[al]'; invalid = '[AL]'; reason = 'invalid technologies' }, + @{ name = 'malformed-application-area'; field = 'application-area'; valid = '[all]'; invalid = '[finance_]'; reason = 'invalid application-area' }, + @{ name = 'uppercase-version-sentinel'; field = 'bc-version'; valid = '[all]'; invalid = '[ALL]'; reason = 'invalid bc-version' }, + @{ name = 'uppercase-country-sentinel'; field = 'countries'; valid = '[w1]'; invalid = '[W1]'; reason = 'invalid countries' }, + @{ name = 'zero-open-version-range'; field = 'bc-version'; valid = '[all]'; invalid = '["0.."]'; reason = 'invalid bc-version range bound' }, + @{ name = 'zero-closed-version-range'; field = 'bc-version'; valid = '[all]'; invalid = '["0..0"]'; reason = 'invalid bc-version range bound' } + ) + foreach ($case in $semanticCases) { + Test-InvalidSemanticIndexing -FixtureRoot (Join-Path $tmp "semantic-$($case.name)") ` + -CaseName $case.name -Field $case.field -ValidValue $case.valid ` + -InvalidValue $case.invalid -ExpectedReason $case.reason + } + + Assert-Throws { + & $search -BCQualityRoot $Root -IndexPath $indexPath -Domain ('x' * 2000) -MaxBytes 1024 + } 'Page envelope exceeds' 'oversized page envelope fails' + + $articlePaths = @($index.articles.path | Sort-Object) + Assert-Sequence $articlePaths $diskArticlePaths 'exact article path union matches disk' + Assert-Throws { + & $getArticles -BCQualityRoot $Root -IndexPath $indexPath -Paths @($articlePaths[0..8]) + } 'exceeds MaxArticles=8' 'exact retrieval rejects path batches larger than eight' + $samplePaths = @( + foreach ($layer in 'microsoft', 'community', 'custom') { + $knowledge = Join-Path $Root "$layer\knowledge" + if (Test-Path -LiteralPath $knowledge) { + Get-ChildItem -LiteralPath $knowledge -Recurse -File | + Where-Object Name -Match '\.(good|bad)\.[a-z0-9]+$' | + ForEach-Object { + [IO.Path]::GetRelativePath($Root, $_.FullName).Replace('\', '/') + } + } + } + ) | Sort-Object + Test-BodyRoundTrip -Paths $articlePaths -IndexPath $indexPath + Test-BodyRoundTrip -Paths $samplePaths -IndexPath $indexPath -Samples + + $fixtureRoot = Join-Path $tmp 'neutral' + New-NeutralArticle -FixtureRoot $fixtureRoot -Layer microsoft -Slug default + New-NeutralArticle -FixtureRoot $fixtureRoot -Layer community -Slug versioned -Version '"27.."' -Technology javascript -Country dk -Area finance -Title 'Versioned neutral example' + New-NeutralArticle -FixtureRoot $fixtureRoot -Layer custom -Slug localized -Version 28 -Technology al -Country de -Area service -Title 'Localized neutral example' + $fixtureIndex = Join-Path $tmp 'neutral-index.json' + & $generator -BCQualityRoot $fixtureRoot -IndexPath $fixtureIndex | Out-Null + + foreach ($case in @( + @{ name = 'uppercase'; layers = @('Microsoft'); pattern = 'unique canonical lowercase layer names' }, + @{ name = 'duplicate'; layers = @('microsoft', 'microsoft'); pattern = 'unique canonical lowercase layer names' }, + @{ name = 'unknown'; layers = @('partner'); pattern = 'unique canonical lowercase layer names' }, + @{ name = 'null'; layers = $null; pattern = 'must be an array' } + )) { + Test-InvalidEnabledLayers -FixtureRoot $fixtureRoot -CaseName $case.name ` + -Layers $case.layers -ExpectedPattern $case.pattern + } + $subsetIndex = Join-Path $tmp 'community-only-index.json' + & $generator -BCQualityRoot $fixtureRoot -IndexPath $subsetIndex ` + -EnabledLayers @('community') | Out-Null + $subset = & $search -BCQualityRoot $fixtureRoot -IndexPath $subsetIndex ` + -Domain neutral -EnabledLayers @('community') | + ConvertFrom-Json + Assert-Equal $subset.candidateCount 1 'valid EnabledLayers subset builds and is consumable' + Assert-Sequence $subset.candidates.path @('community/knowledge/neutral/versioned.md') 'valid subset contains only its exact layer' + + $applicable = Invoke-CatalogPages -Arguments @{ + BCQualityRoot = $fixtureRoot + IndexPath = $fixtureIndex + Domain = 'neutral' + BCVersion = 28 + Technologies = @('al', 'javascript') + Countries = @('dk', 'de') + ApplicationAreas = @('finance', 'service') + } -MaxBytes 16000 + Assert-Equal $applicable.candidates.Count 3 'neutral layer/version rows all survive matching context' + Assert-True (@($applicable.candidates | Where-Object applicability -CEQ applicable).Count -eq 3) 'matching rows are applicable' + $versioned = $applicable.candidates | Where-Object path -CEQ 'community/knowledge/neutral/versioned.md' + Assert-Equal $versioned.layer community 'non-default layer survives' + Assert-Sequence $versioned.'bc-version' @('"27.."') 'original version metadata survives' + Assert-Equal $versioned.applicability applicable 'lowercase sentinels and positive open range remain applicable' + Assert-Sequence $versioned.technologies @('javascript') 'non-default technology survives' + Assert-Sequence $versioned.countries @('dk') 'non-default country survives' + Assert-Sequence $versioned.'application-area' @('finance') 'non-default application area survives' + + # Metadata validation accepts range bounds wider than Int32, so version + # matching must compare as bigint rather than coercing the bound down. + $wideRoot = Join-Path $tmp 'wide-version' + New-NeutralArticle -FixtureRoot $wideRoot -Layer microsoft -Slug wide-closed -Version '"1..99999999999"' + New-NeutralArticle -FixtureRoot $wideRoot -Layer microsoft -Slug wide-open -Version '"99999999999.."' + $wideIndex = Join-Path $tmp 'wide-version-index.json' + & $generator -BCQualityRoot $wideRoot -IndexPath $wideIndex | Out-Null + $wide = Invoke-CatalogPages -Arguments @{ + BCQualityRoot = $wideRoot + IndexPath = $wideIndex + Domain = 'neutral' + BCVersion = 28 + } -MaxBytes 16000 + Assert-Sequence $wide.candidates.path @('microsoft/knowledge/neutral/wide-closed.md') 'bc-version bounds beyond Int32 compare without overflow' + + $conditional = Invoke-CatalogPages -Arguments @{ + BCQualityRoot = $fixtureRoot + IndexPath = $fixtureIndex + Domain = 'neutral' + BCVersion = 28 + Technologies = @('al', 'javascript') + } -MaxBytes 16000 + $conditionalVersioned = $conditional.candidates | + Where-Object path -CEQ 'community/knowledge/neutral/versioned.md' + Assert-Equal $conditionalVersioned.applicability conditional 'unknown context produces conditional verdict' + Assert-Sequence $conditionalVersioned.unknownDimensions @('countries', 'application-area') 'unknown dimensions survive' + + $layerFiltered = Invoke-CatalogPages -Arguments @{ + BCQualityRoot = $fixtureRoot + IndexPath = $fixtureIndex + Domain = 'neutral' + EnabledLayers = @('microsoft') + } -MaxBytes 16000 + Assert-Equal $layerFiltered.candidates.Count 1 'enabled layer remains a candidate' + Assert-Equal $layerFiltered.excluded.Count 2 'disabled layers remain explicit' + Assert-Sequence ($layerFiltered.excluded.layer | Sort-Object) @('community', 'custom') 'excluded rows preserve layer' + + $oldSnapshot = $conditional.snapshot + Add-Content -LiteralPath (Join-Path $fixtureRoot 'community\knowledge\neutral\versioned.md') -Value ' ' -Encoding utf8NoBOM + $preparedCatalog = & $search -BCQualityRoot $fixtureRoot -IndexPath $fixtureIndex -Domain neutral | + ConvertFrom-Json + Assert-Equal $preparedCatalog.candidateCount 3 'catalog uses the prepared index without rehashing article bodies' + Assert-Throws { + & $getArticles -BCQualityRoot $fixtureRoot -IndexPath $fixtureIndex ` + -Paths 'community/knowledge/neutral/versioned.md' + } 'Selected article hash does not match the prepared index' 'exact retrieval detects selected article changes' + & $generator -BCQualityRoot $fixtureRoot -IndexPath $fixtureIndex | Out-Null + Assert-Throws { + & $search -BCQualityRoot $fixtureRoot -IndexPath $fixtureIndex -Domain neutral -Offset 1 -Snapshot $oldSnapshot + } 'Snapshot changed' 'continuation cannot cross rebuilt snapshots' + + $largeRoot = Join-Path $tmp 'large-catalog' + New-NeutralArticle -FixtureRoot $largeRoot -Layer microsoft -Slug huge-title -Title ('T' * 3000) + $largeIndex = Join-Path $tmp 'large-index.json' + & $generator -BCQualityRoot $largeRoot -IndexPath $largeIndex | Out-Null + Assert-Throws { + & $search -BCQualityRoot $largeRoot -IndexPath $largeIndex -Domain neutral -MaxBytes 1024 + } 'One complete candidates row|Page envelope exceeds' 'oversized catalog row fails without clipping' + + # The shared pager reports the oversized row's identity for any row shape; + # a row without a path must still reach its explicit offset-based failure. + . (Join-Path $Root 'tools/Bounded-Results.ps1') + $pagerHeader = [ordered]@{ version = 2; snapshot = ('0' * 64) } + foreach ($shape in @( + @{ name = 'dictionary'; row = [ordered]@{ blob = ('x' * 3000) } }, + @{ name = 'object'; row = [pscustomobject]@{ blob = ('x' * 3000) } } + )) { + Assert-Throws { + ConvertTo-BoundedPage -Header $pagerHeader ` + -Groups ([ordered]@{ rows = @($shape.row) }) -MaxBytes 1024 + } 'One complete rows row plus envelope exceeds MaxBytes=1024 at Offset=0' "oversized pathless $($shape.name) row fails with its offset identity" + } + + $bodyRoot = Join-Path $tmp 'body-failures' + New-NeutralArticle -FixtureRoot $bodyRoot -Layer microsoft -Slug huge-body -Description ('x' * 3000) + New-NeutralArticle -FixtureRoot $bodyRoot -Layer microsoft -Slug broken-link + New-NeutralArticle -FixtureRoot $bodyRoot -Layer microsoft -Slug continuation-one -Description ('a' * 300) + New-NeutralArticle -FixtureRoot $bodyRoot -Layer microsoft -Slug continuation-two -Description ('b' * 300) + New-NeutralArticle -FixtureRoot $bodyRoot -Layer microsoft -Slug invalid-utf8 + New-NeutralArticle -FixtureRoot $bodyRoot -Layer microsoft -Slug sample-one + New-NeutralArticle -FixtureRoot $bodyRoot -Layer microsoft -Slug sample-two + Add-Content -LiteralPath (Join-Path $bodyRoot 'microsoft\knowledge\neutral\sample-one.md') ` + -Value '[`sample-one.good.al`](sample-one.good.al)' -Encoding utf8NoBOM + Add-Content -LiteralPath (Join-Path $bodyRoot 'microsoft\knowledge\neutral\sample-two.md') ` + -Value '[`sample-two.good.al`](sample-two.good.al)' -Encoding utf8NoBOM + Set-Content -LiteralPath (Join-Path $bodyRoot 'microsoft\knowledge\neutral\sample-one.good.al') ` + -Value ('a' * 900) -Encoding utf8NoBOM + Set-Content -LiteralPath (Join-Path $bodyRoot 'microsoft\knowledge\neutral\sample-two.good.al') ` + -Value ('b' * 900) -Encoding utf8NoBOM + $bodyIndex = Join-Path $tmp 'body-index.json' + & $generator -BCQualityRoot $bodyRoot -IndexPath $bodyIndex | Out-Null + Assert-Throws { + & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex ` + -Paths 'microsoft/knowledge/neutral/huge-body.md' -MaxBytes 1024 + } 'No complete body plus continuation fits' 'oversized body fails without truncation' + Assert-Throws { + & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex -Paths '../outside.md' + } 'Invalid knowledge path' 'unsafe requested path fails' + Assert-Throws { + & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex ` + -Paths 'microsoft/knowledge/neutral/huge-body.md' -EnabledLayers community + } 'Layer disabled' 'disabled article layer fails' + Assert-Throws { + & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex -Paths @( + 'microsoft/knowledge/neutral/huge-body.md', + 'microsoft/knowledge/neutral/huge-body.md' + ) + } 'Duplicate requested path' 'duplicate exact paths fail' + + $brokenSample = Join-Path $bodyRoot 'microsoft\knowledge\neutral\broken-link.good.al' + Set-Content -LiteralPath $brokenSample -Value 'codeunit 1 Neutral { }' -Encoding utf8NoBOM + Assert-Throws { + & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex ` + -Paths 'microsoft/knowledge/neutral/broken-link.good.al' -Samples + } 'Sample is not linked' 'unlinked sample fails' + + $sampleContinuationPaths = @( + 'microsoft/knowledge/neutral/sample-one.good.al', + 'microsoft/knowledge/neutral/sample-two.good.al' + ) + $firstSamplePage = & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex ` + -Paths $sampleContinuationPaths -Samples -MaxBytes 1600 | + ConvertFrom-Json + Assert-True (-not $firstSamplePage.complete) 'bounded sample batch produces continuation' + Assert-Sequence $firstSamplePage.remainingPaths @('microsoft/knowledge/neutral/sample-two.good.al') 'sample continuation preserves pending path' + Add-Content -LiteralPath (Join-Path $bodyRoot 'microsoft\knowledge\neutral\sample-two.good.al') ` + -Value 'changed' -Encoding utf8NoBOM + Assert-Throws { + & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex ` + -Paths @($firstSamplePage.remainingPaths) -Samples ` + -Snapshot $firstSamplePage.continuation.snapshot + } 'Article snapshot changed' 'sample continuation rejects a changed pending sample' + + $continuationPaths = @( + 'microsoft/knowledge/neutral/continuation-one.md', + 'microsoft/knowledge/neutral/continuation-two.md' + ) + $firstBodyPage = & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex ` + -Paths $continuationPaths -MaxBytes 1300 | + ConvertFrom-Json + Assert-True (-not $firstBodyPage.complete) 'bounded article batch produces continuation' + Assert-Throws { + & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex ` + -Paths @($firstBodyPage.remainingPaths) -Snapshot ('0' * 64) + } 'Article snapshot changed' 'wrong article continuation snapshot fails' + Add-Content -LiteralPath (Join-Path $bodyRoot 'microsoft\knowledge\neutral\continuation-two.md') -Value 'changed' -Encoding utf8NoBOM + Assert-Throws { + & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex ` + -Paths @($firstBodyPage.remainingPaths) -Snapshot $firstBodyPage.continuation.snapshot + } 'Selected article hash does not match the prepared index' 'article continuation rejects a changed remaining body' + + $invalidUtf8 = Join-Path $bodyRoot 'microsoft\knowledge\neutral\invalid-utf8.md' + $indexedBytes = [IO.File]::ReadAllBytes($invalidUtf8) + [IO.File]::WriteAllBytes($invalidUtf8, [byte[]]@($indexedBytes + @(0xc3, 0x28))) + Assert-Throws { + & $getArticles -BCQualityRoot $bodyRoot -IndexPath $bodyIndex ` + -Paths 'microsoft/knowledge/neutral/invalid-utf8.md' + } 'Knowledge file is not valid strict UTF-8' 'invalid UTF-8 fails' + + Write-Host "Knowledge retrieval check PASSED: $($articlePaths.Count) articles and $($samplePaths.Count) samples round-tripped; catalog union was lossless and bounded." -ForegroundColor Green +} +finally { + Remove-Item -LiteralPath $tmp -Recurse -Force -ErrorAction SilentlyContinue +} From 35d0966a8d45e8d4a7c2b0238afbffe338a31d2b Mon Sep 17 00:00:00 2001 From: Jesper Schulz-Wedde Date: Fri, 11 Sep 2026 15:24:20 +0200 Subject: [PATCH 02/19] Normalize recoverable leaf finding ranges (#180) * Normalize recoverable leaf ranges Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Run contract checks with review fixtures Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Jesper Schulz-Wedde Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/contributing.md | 7 +- docs/standalone-runner.md | 20 ++- microsoft/skills/review/al-code-review.md | 13 +- skills/do.md | 57 ++++++-- tools/Test-ReviewContract.ps1 | 171 ++++++++++++++++++++++ tools/Test-ReviewFixtures.ps1 | 1 + 6 files changed, 249 insertions(+), 20 deletions(-) create mode 100644 tools/Test-ReviewContract.ps1 diff --git a/docs/contributing.md b/docs/contributing.md index 27da177..51f6d0f 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -161,12 +161,15 @@ If PyYAML is not installed in your development environment, install it with ```powershell python .github\scripts\validate_frontmatter.py --root . pwsh .\tools\Test-ReviewFixtures.ps1 -Root . +pwsh .\tools\Test-ReviewContract.ps1 -Root . ``` The first command checks schema, sections, naming, sample references, and skill registration. The second checks that every review leaf has a valid -positive/clean sample pair. Neither proves a model will find every defect. -See [evaluation](../evaluation/README.md) for optional model-based scoring. +positive/clean sample pair. The third checks the cross-surface findings-report +contract and its bounded range-normalization cases. None proves a model will +find every defect. See [evaluation](../evaluation/README.md) for optional +model-based scoring. In the PR description, explain the mistake being prevented, supporting evidence, applicable BC versions, and why the chosen domain owns it. For a diff --git a/docs/standalone-runner.md b/docs/standalone-runner.md index 61ecb55..5829e4e 100644 --- a/docs/standalone-runner.md +++ b/docs/standalone-runner.md @@ -52,10 +52,17 @@ only result. 4. When an action skill declares `sub-skills`, execute every relevant leaf as a discrete invocation. Leaves are independent and may be scheduled serially or concurrently. -5. Collect each complete findings-report into `sub-results` in the declared +5. Capture the exact Task return as the immutable raw audit payload and primary + transport. Preserve it unchanged in private artifacts or host logs. Before + the full DO acceptance gate, create a normalized candidate only for DO's + bounded optional-range case, record that normalization separately in private + telemetry, and accept the candidate only if the entire copy passes the + unchanged strict gate. The accepted report contains no undeclared telemetry + fields. +6. Collect each accepted findings-report into `sub-results` in the declared `sub-skills` order, not completion order. Run the super-skill self-review only after all leaves have finished. -6. Apply the DO composition, failure, deduplication, reference-integrity, and +7. Apply the DO composition, failure, deduplication, reference-integrity, and outcome rules. Return strict JSON before rendering it for people or another system. @@ -86,6 +93,15 @@ A compatible runner: - invokes every worklisted leaf exactly once unless a documented retry replaces a failed attempt; - keeps leaf contexts isolated and passes only the inputs they declare; +- preserves each raw Task return unchanged for audit and distinguishes it from + any normalized accepted copy; +- removes only an optional range whose positive integer bounds contain the + primary line but start before it, and only when the complete report has no + other defect and the finding has no `suggested-code`; +- records normalization only in private runner telemetry and never adds fields + to the findings-report; +- rejects reversed, invalid, or out-of-bounds ranges, range mismatches attached + to `suggested-code`, and every repair outside DO's bounded exception; - preserves every leaf report, including failed reports, in `sub-results`; - excludes unreliable findings from failed leaves and returns `partial` when only part of the review is reliable; diff --git a/microsoft/skills/review/al-code-review.md b/microsoft/skills/review/al-code-review.md index 6a577ba..9239220 100644 --- a/microsoft/skills/review/al-code-review.md +++ b/microsoft/skills/review/al-code-review.md @@ -67,7 +67,7 @@ The Action step consists of **discrete leaf invocations**, not one combined gene - **Isolate leaf invocations when the host supports it.** Each sub-skill SHOULD run in a fresh model call or child context containing only its assigned source paths, READ/DO contracts, the leaf instructions, the complete bounded domain catalog per READ, and articles that leaf worklists. Preserve each catalog row's exact `path`; the leaf must copy references from that catalog. - **Keep run artifacts private.** Before dispatch, allocate a new GUID-named directory under the current session's artifact directory and a distinct scratch/report child directory for every leaf. Pass a leaf only its own assigned source paths and child directory, never the run root or sibling paths. A leaf MUST NOT discover, enumerate, read, modify, or delete sibling artifacts. Do not reuse a prior run directory, and do not clean up any run artifact until every leaf has finished and consolidation is complete. -- **Use the exact Task return as the report.** Capture each leaf's exact return as the primary transport and apply DO's consumer acceptance gate before rollup. Worker-side persistence of the same report in its private directory is optional and redundant; a missing report file does not invalidate an otherwise valid exact return. +- **Keep raw Task transport distinct from the accepted copy.** Capture the exact Task return as the immutable raw audit payload and primary transport. Preserve it unchanged in the leaf's private artifacts or host log. Then apply DO's bounded pre-gate range normalization, when eligible, and its full consumer acceptance gate. The report accepted for rollup is the exact return when no normalization occurred, or the normalized candidate copy when DO permits it; worker-side persistence of another report file is optional and redundant. - **Treat automatic output spills as host-owned.** If the host reports that a Task return was automatically spilled, the coordinator MAY read that file read-only only at the exact path returned by the tool. Never modify, delete, enumerate around, or reuse an automatic spill path. Never bypass a content-exclusion or access denial. - Treat each sub-skill in the worklist as its own pass: read the sub-skill's instructions, apply its Source → Relevance → Worklist → Action steps to the orchestrator-supplied inputs, and produce that sub-skill's complete findings-report independently. - Do not collapse multiple sub-skills into one shared reasoning step. Each sub-skill has a distinct knowledge subset and a distinct evaluation procedure; sharing one rolled-up scan dilutes per-skill attention and causes leaves to silently underreport (this has been observed in production: leaf skills returned empty `findings[]` while their standalone runs against the same diff produced multiple matches). @@ -80,7 +80,7 @@ The Action step consists of **discrete leaf invocations**, not one combined gene For each sub-skill in the worklist: 1. Invoke the sub-skill with the orchestrator's inputs, passing only the subset each sub-skill declares in its `inputs`. -2. Capture the exact Task return and validate it against DO's consumer acceptance gate before accepting it. Preserve an invalid raw return unchanged in the leaf's private artifacts or host log; do not reconstruct or repair it. Record a separate failed validation result with no findings for rollup. +2. Capture the exact Task return as the immutable raw audit payload and primary transport. Preserve it unchanged in the leaf's private artifacts or host log before deriving a candidate. Apply only DO's bounded pre-gate normalization: when the complete raw report has no other defect, a finding has positive-integer `line`, `start-line`, and `end-line`, `start-line <= line <= end-line`, `start-line != line`, and no `suggested-code` field, copy the complete report and remove only that finding's optional `location.range`. Record the normalization separately in private run telemetry or artifacts, never in the findings-report. Validate the entire candidate through DO's existing strict acceptance gate. Accept the exact return when unchanged or the normalized candidate when it passes; otherwise record a separate failed validation result with no findings for rollup. Do not reconstruct JSON, infer fields, alter paths or references, clamp lines, normalize reversed or out-of-bounds ranges, remove a range associated with `suggested-code`, or salvage individual findings. 3. Append the accepted findings-report, or the separate failed validation result, to `sub-results`. If its `outcome` is `failed`, stop here for this sub-skill: its findings are not reliable per the DO contract and MUST NOT be copied into the super-skill's top-level `findings[]` or counted in `summary.counts`. 4. Otherwise, compare each entry from the sub-skill's `findings[]` with findings already rolled up. Two findings are duplicates when they point to the same file and overlapping line/range and prescribe materially the same correction, even when their knowledge-file IDs differ. Merge duplicates instead of appending both: keep the more specific domain owner, preserve that finding's optional `domain` field verbatim (including its absence), use its reference as `references[0]` and therefore as `id`, append the other references as supporting references, keep the highest severity and confidence justified by either report, and preserve one self-contained message. Article and leaf ownership notes decide specificity; do not choose by execution order. 5. Append each non-duplicate finding, setting `from-sub-skill` to the sub-skill's `skill.id` and preserving its optional `domain` field verbatim, including its absence. For non-citation findings (those whose `id` is a skill-defined slug rather than a reference path), prefix `id` with `:` to prevent collisions across sub-skills. Other finding fields are preserved. @@ -126,9 +126,12 @@ Calculate `summary.counts` from the final top-level `findings[]`, after failed s Derive `outcome` using the DO rollup rules. `outcome-reason` is populated for `partial` and `failed` and SHOULD summarize per-sub-skill state, for example: *"al-security-review failed (tool timeout); al-performance-review completed."* Before emitting the rollup, apply DO's consumer acceptance gate to every nested -and top-level finding. Treat an invalid sub-result as failed and exclude all of -its findings from the top-level rollup. Preserve its exact raw payload -separately; never reconstruct it into a success-shaped report. +and top-level finding. A leaf's nested report is its accepted exact return or +its accepted normalized candidate copy; its exact Task return remains the +separate immutable raw audit payload. Treat an invalid sub-result as failed and +exclude all of its findings from the top-level rollup. Never reconstruct it +into a success-shaped report or perform normalization beyond DO's bounded +exception. ## Output diff --git a/skills/do.md b/skills/do.md index f86509b..9abdf58 100644 --- a/skills/do.md +++ b/skills/do.md @@ -161,13 +161,49 @@ AL source is the common failure case. Quoted identifiers (for example `Rec."No." ### Consumer acceptance gate -The exact action-skill return is the primary report transport. Before accepting -it as a findings-report, a coordinator or host MUST validate it -deterministically: +Capture the exact Task return as the immutable raw audit payload and primary +transport. Preserve it unchanged in private run artifacts or host logs before +creating any derived value. The accepted findings-report is either that exact +return or the bounded normalized candidate described below; the raw audit +payload never changes. -1. Parse the exact return as strict JSON and validate every required field, - enum, type, conditional requirement, summary count, coverage value, and - leaf/super-skill constraint against this output contract. +Before the full acceptance gate, a coordinator MAY create a normalized +candidate copy only through this deterministic procedure: + +1. Parse the exact return as strict JSON and provisionally check the complete + report without mutating it. Every acceptance rule below MUST already pass + except for one or more findings whose optional `location.range` has + `start-line != line`. +2. Each such finding is eligible only when `location.line`, + `location.range.start-line`, and `location.range.end-line` are positive + integers, `start-line <= line <= end-line`, and the finding does not contain + the `suggested-code` field. Field presence disqualifies normalization even + if its value is empty because suggested code may be bound to the reported + range. +3. Deep-copy the complete parsed report. In the candidate copy, remove only + `location.range` from every eligible finding. Retain `location.line` and + every other value unchanged. Do not add normalization metadata to the + findings-report. +4. Record each removed range separately in private run telemetry or artifacts, + associated with the immutable raw audit payload. This record is + runner-owned and is not part of the declared report schema. +5. Validate the entire normalized candidate with the existing full consumer + acceptance gate below. Only a candidate that passes every rule becomes the + accepted copy used for rollup. If any other validation defect exists, or + full validation fails, discard the candidate, preserve the raw payload, and + fail the complete leaf as before. + +This exception does not infer missing fields, alter references or paths, clamp +line numbers, repair JSON, normalize a reversed or out-of-bounds range, remove +a range from a finding containing `suggested-code`, or salvage arbitrary +individual findings. + +Before accepting either the exact return or an eligible normalized candidate +as a findings-report, a coordinator or host MUST validate it deterministically: + +1. Validate every required field, enum, type, conditional requirement, summary + count, coverage value, and leaf/super-skill constraint against this output + contract. 2. For every knowledge-backed finding, verify each `references[].path` is an exact repo-relative knowledge path that exists in the live BCQuality snapshot, and verify `findings[].id` exactly equals @@ -183,11 +219,10 @@ deterministically: Validation failure invalidates the complete return; consumers MUST NOT salvage individual findings, infer missing fields, reconstruct JSON, clamp ranges, rewrite paths, or otherwise silently repair model output. Preserve the invalid -raw payload unchanged in private run artifacts or host logs. Record a separate -failed validation result for that leaf with no findings, and derive the -super-skill outcome as `partial` or `failed` using the normal rollup rules. -Worker-side report-file persistence is optional and never replaces validation -of the exact return. +raw payload unchanged. Record a separate failed validation result for that leaf +with no findings, and derive the super-skill outcome as `partial` or `failed` +using the normal rollup rules. Worker-side report-file persistence is optional +and never replaces validation of the accepted exact or normalized copy. ### Field semantics diff --git a/tools/Test-ReviewContract.ps1 b/tools/Test-ReviewContract.ps1 new file mode 100644 index 0000000..1d40058 --- /dev/null +++ b/tools/Test-ReviewContract.ps1 @@ -0,0 +1,171 @@ +<# +.SYNOPSIS + Validates the bounded leaf-range normalization contract. + +.DESCRIPTION + BCQuality has no executable findings-report consumer. These assertions keep + the normative DO contract, AL coordinator, and standalone runner aligned + while exercising the exact normalization predicate against representative + safe and ambiguous inputs. +#> +[CmdletBinding()] +param( + [string] $Root = (Resolve-Path (Join-Path $PSScriptRoot '..')) +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' + +$Root = (Resolve-Path -LiteralPath $Root).Path + +function Assert-True { + param( + [bool] $Condition, + [string] $Message + ) + + if (-not $Condition) { + throw "Assertion failed: $Message" + } +} + +function Assert-Contains { + param( + [string] $Text, + [string] $Expected, + [string] $Message + ) + + Assert-True $Text.Contains($Expected) $Message +} + +function Test-PositiveInteger { + param([object] $Value) + + if (($null -eq $Value) -or ($Value -is [bool]) -or ($Value -isnot [ValueType])) { + return $false + } + + $number = [double]$Value + return [double]::IsFinite($number) -and ($number -gt 0) -and ([math]::Truncate($number) -eq $number) +} + +function Test-RangeNormalizationEligibility { + param([pscustomobject] $Finding) + + if ($Finding.PSObject.Properties.Name -contains 'suggested-code') { + return $false + } + if (-not ($Finding.PSObject.Properties.Name -contains 'location')) { + return $false + } + if (-not ($Finding.location.PSObject.Properties.Name -contains 'line')) { + return $false + } + if (-not ($Finding.location.PSObject.Properties.Name -contains 'range')) { + return $false + } + + $range = $Finding.location.range + if (-not ($range.PSObject.Properties.Name -contains 'start-line') -or + -not ($range.PSObject.Properties.Name -contains 'end-line')) { + return $false + } + + $line = $Finding.location.line + $startLine = $range.'start-line' + $endLine = $range.'end-line' + if (-not (Test-PositiveInteger $line) -or + -not (Test-PositiveInteger $startLine) -or + -not (Test-PositiveInteger $endLine)) { + return $false + } + + return ($startLine -le $line) -and ($line -le $endLine) -and ($startLine -ne $line) +} + +$transportSentence = 'Capture the exact Task return as the immutable raw audit payload and primary transport.' +$doContract = Get-Content -LiteralPath (Join-Path $Root 'skills/do.md') -Raw +$coordinatorContract = Get-Content -LiteralPath (Join-Path $Root 'microsoft/skills/review/al-code-review.md') -Raw +$runnerContract = Get-Content -LiteralPath (Join-Path $Root 'docs/standalone-runner.md') -Raw + +foreach ($surface in @( + [pscustomobject]@{ Name = 'DO'; Text = ($doContract -replace '\s+', ' ') } + [pscustomobject]@{ Name = 'AL coordinator'; Text = ($coordinatorContract -replace '\s+', ' ') } + [pscustomobject]@{ Name = 'standalone runner'; Text = ($runnerContract -replace '\s+', ' ') } +)) { + Assert-Contains $surface.Text $transportSentence "$($surface.Name) preserves exact Task transport wording" +} + +$normalizedDoContract = $doContract -replace '\s+', ' ' +foreach ($expected in @( + 'positive integers', + 'start-line <= line <= end-line', + 'does not contain the `suggested-code` field', + 'remove only', + 'private run telemetry or artifacts', + 'Validate the entire normalized candidate', + 'If any other validation defect exists', + 'salvage arbitrary individual findings' +)) { + Assert-Contains $normalizedDoContract $expected "DO documents '$expected'" +} + +$cases = @( + [pscustomobject]@{ + Name = 'contained mismatched range without suggested code' + Expected = $true + Finding = '{"message":"keep me","location":{"file":"src/codeunit.al","line":37,"range":{"start-line":36,"end-line":38}}}' | ConvertFrom-Json + } + [pscustomobject]@{ + Name = 'aligned range' + Expected = $false + Finding = '{"location":{"line":37,"range":{"start-line":37,"end-line":38}}}' | ConvertFrom-Json + } + [pscustomobject]@{ + Name = 'suggested code present' + Expected = $false + Finding = '{"location":{"line":37,"range":{"start-line":36,"end-line":38}},"suggested-code":""}' | ConvertFrom-Json + } + [pscustomobject]@{ + Name = 'line outside range' + Expected = $false + Finding = '{"location":{"line":39,"range":{"start-line":36,"end-line":38}}}' | ConvertFrom-Json + } + [pscustomobject]@{ + Name = 'reversed range' + Expected = $false + Finding = '{"location":{"line":37,"range":{"start-line":38,"end-line":36}}}' | ConvertFrom-Json + } + [pscustomobject]@{ + Name = 'zero bound' + Expected = $false + Finding = '{"location":{"line":1,"range":{"start-line":0,"end-line":2}}}' | ConvertFrom-Json + } + [pscustomobject]@{ + Name = 'fractional primary line' + Expected = $false + Finding = '{"location":{"line":37.5,"range":{"start-line":36,"end-line":38}}}' | ConvertFrom-Json + } + [pscustomobject]@{ + Name = 'missing end line' + Expected = $false + Finding = '{"location":{"line":37,"range":{"start-line":36}}}' | ConvertFrom-Json + } +) + +foreach ($case in $cases) { + $actual = Test-RangeNormalizationEligibility $case.Finding + Assert-True ($actual -eq $case.Expected) "$($case.Name) eligibility is $($case.Expected)" +} + +$rawFinding = $cases[0].Finding +$candidateFinding = $rawFinding | ConvertTo-Json -Depth 10 | ConvertFrom-Json +$candidateFinding.location.PSObject.Properties.Remove('range') + +Assert-True ($rawFinding.location.PSObject.Properties.Name -contains 'range') 'raw finding remains unchanged' +Assert-True (-not ($candidateFinding.location.PSObject.Properties.Name -contains 'range')) 'candidate removes only the optional range' +Assert-True ($candidateFinding.location.line -eq $rawFinding.location.line) 'candidate preserves the primary line' +Assert-True ($candidateFinding.message -ceq $rawFinding.message) 'candidate preserves all other finding content' + +Write-Output "Review contract validation passed ($($cases.Count) normalization cases)." diff --git a/tools/Test-ReviewFixtures.ps1 b/tools/Test-ReviewFixtures.ps1 index a9912b7..c4b0bf1 100644 --- a/tools/Test-ReviewFixtures.ps1 +++ b/tools/Test-ReviewFixtures.ps1 @@ -434,6 +434,7 @@ if ($PrepareDirectory) { } if (-not $ResultsPath -and -not $ResultsDirectory) { + & (Join-Path $PSScriptRoot 'Test-ReviewContract.ps1') -Root $Root Write-Host "Review fixture validation PASSED: $($cases.Count) cases cover $($leafDomains.Count) leaf domains." -ForegroundColor Green exit 0 } From 45ac371e7a8252f2ce7176060a640ea9506b320c Mon Sep 17 00:00:00 2001 From: Stefano Demiliani <33155438+demiliani@users.noreply.github.com> Date: Mon, 14 Sep 2026 12:42:45 +0200 Subject: [PATCH 03/19] Add AL-focused AppSource validation guidance (#142) * knowledge(appsource): add AL validation guidance * Address Marketplace review feedback * Address remaining Marketplace review feedback * Align AppSource review applicability outcome --- .../define-profiles-as-al-objects.bad.al | 12 +++++++++ .../define-profiles-as-al-objects.good.al | 6 +++++ .../define-profiles-as-al-objects.md | 26 +++++++++++++++++++ .../do-not-hard-code-time-zone-offsets.bad.al | 7 +++++ ...do-not-hard-code-time-zone-offsets.good.al | 7 +++++ .../do-not-hard-code-time-zone-offsets.md | 26 +++++++++++++++++++ ...-web-service-paths-free-of-ui-calls.bad.al | 21 +++++++++++++++ ...web-service-paths-free-of-ui-calls.good.al | 15 +++++++++++ ...keep-web-service-paths-free-of-ui-calls.md | 26 +++++++++++++++++++ ...on-actions-with-addfirst-or-addlast.bad.al | 15 +++++++++++ ...n-actions-with-addfirst-or-addlast.good.al | 15 +++++++++++ ...ension-actions-with-addfirst-or-addlast.md | 26 +++++++++++++++++++ ...category-on-searchable-entry-points.bad.al | 20 ++++++++++++++ ...ategory-on-searchable-entry-points.good.al | 21 +++++++++++++++ ...sagecategory-on-searchable-entry-points.md | 26 +++++++++++++++++++ .../use-invariant-date-literals.bad.al | 10 +++++++ .../use-invariant-date-literals.good.al | 7 +++++ .../appsource/use-invariant-date-literals.md | 26 +++++++++++++++++++ .../skills/review/al-appsource-review.md | 14 +++++++--- 19 files changed, 322 insertions(+), 4 deletions(-) create mode 100644 community/knowledge/appsource/define-profiles-as-al-objects.bad.al create mode 100644 community/knowledge/appsource/define-profiles-as-al-objects.good.al create mode 100644 community/knowledge/appsource/define-profiles-as-al-objects.md create mode 100644 community/knowledge/appsource/do-not-hard-code-time-zone-offsets.bad.al create mode 100644 community/knowledge/appsource/do-not-hard-code-time-zone-offsets.good.al create mode 100644 community/knowledge/appsource/do-not-hard-code-time-zone-offsets.md create mode 100644 community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.bad.al create mode 100644 community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.good.al create mode 100644 community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.md create mode 100644 community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.bad.al create mode 100644 community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.good.al create mode 100644 community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.md create mode 100644 community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.bad.al create mode 100644 community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.good.al create mode 100644 community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.md create mode 100644 community/knowledge/appsource/use-invariant-date-literals.bad.al create mode 100644 community/knowledge/appsource/use-invariant-date-literals.good.al create mode 100644 community/knowledge/appsource/use-invariant-date-literals.md diff --git a/community/knowledge/appsource/define-profiles-as-al-objects.bad.al b/community/knowledge/appsource/define-profiles-as-al-objects.bad.al new file mode 100644 index 0000000..20438c8 --- /dev/null +++ b/community/knowledge/appsource/define-profiles-as-al-objects.bad.al @@ -0,0 +1,12 @@ +codeunit 50100 "Rental Profile Install" +{ + Subtype = Install; + + trigger OnInstallAppPerDatabase() + var + RentalProfile: Record Profile; + begin + RentalProfile.Init(); + RentalProfile.Insert(true); + end; +} \ No newline at end of file diff --git a/community/knowledge/appsource/define-profiles-as-al-objects.good.al b/community/knowledge/appsource/define-profiles-as-al-objects.good.al new file mode 100644 index 0000000..23ed6bf --- /dev/null +++ b/community/knowledge/appsource/define-profiles-as-al-objects.good.al @@ -0,0 +1,6 @@ +profile "RENTAL MANAGER" +{ + Caption = 'Rental Manager'; + Description = 'Manages rental agreements and equipment availability.'; + RoleCenter = "Business Manager Role Center"; +} \ No newline at end of file diff --git a/community/knowledge/appsource/define-profiles-as-al-objects.md b/community/knowledge/appsource/define-profiles-as-al-objects.md new file mode 100644 index 0000000..6f34aee --- /dev/null +++ b/community/knowledge/appsource/define-profiles-as-al-objects.md @@ -0,0 +1,26 @@ +--- +bc-version: [all] +domain: appsource +keywords: [profile-object, profile-table, install-codeunit, role-center, page-customization] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Define profiles as AL objects + +## Description + +Profiles delivered by a Marketplace extension must be declared as AL `profile` objects. A profile object is validated with its Role Center and page customizations when the extension is compiled and is registered through extension synchronization. Inserting profile-table records from install or setup code bypasses that object lifecycle. + +## Best Practice + +Declare each app-owned profile with the `profile` object and set its `RoleCenter`, user-facing caption, and optional customizations in AL. Let installation and synchronization register the object. + +See sample: [`define-profiles-as-al-objects.good.al`](define-profiles-as-al-objects.good.al). + +## Anti Pattern + +Install, upgrade, or setup code that creates an app-owned profile by inserting a `Profile` table record. Detection signal: a `Record Profile` variable followed by `Insert` in profile provisioning code. + +See sample: [`define-profiles-as-al-objects.bad.al`](define-profiles-as-al-objects.bad.al). \ No newline at end of file diff --git a/community/knowledge/appsource/do-not-hard-code-time-zone-offsets.bad.al b/community/knowledge/appsource/do-not-hard-code-time-zone-offsets.bad.al new file mode 100644 index 0000000..4ec258b --- /dev/null +++ b/community/knowledge/appsource/do-not-hard-code-time-zone-offsets.bad.al @@ -0,0 +1,7 @@ +codeunit 50100 "Rental Audit" +{ + procedure SetCreatedAt(var RentalAgreement: Record "Rental Agreement") + begin + RentalAgreement."Created At" := CurrentDateTime() + 7200000; + end; +} \ No newline at end of file diff --git a/community/knowledge/appsource/do-not-hard-code-time-zone-offsets.good.al b/community/knowledge/appsource/do-not-hard-code-time-zone-offsets.good.al new file mode 100644 index 0000000..c4e155e --- /dev/null +++ b/community/knowledge/appsource/do-not-hard-code-time-zone-offsets.good.al @@ -0,0 +1,7 @@ +codeunit 50100 "Rental Audit" +{ + procedure SetCreatedAt(var RentalAgreement: Record "Rental Agreement") + begin + RentalAgreement."Created At" := CurrentDateTime(); + end; +} \ No newline at end of file diff --git a/community/knowledge/appsource/do-not-hard-code-time-zone-offsets.md b/community/knowledge/appsource/do-not-hard-code-time-zone-offsets.md new file mode 100644 index 0000000..935b4d5 --- /dev/null +++ b/community/knowledge/appsource/do-not-hard-code-time-zone-offsets.md @@ -0,0 +1,26 @@ +--- +bc-version: [all] +domain: appsource +keywords: [datetime, time-zone, utc, currentdatetime, locale, regional-settings] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Do not hard-code time-zone offsets + +## Description + +Marketplace extensions run for users and services in many time zones. Adding a fixed offset to a `DateTime` assumes one locale, ignores daylight-saving transitions, and changes an absolute timestamp into an incorrect value for other regions. + +## Best Practice + +Store and compare `DateTime` values without a manually applied regional offset. Business Central stores `DateTime` values in UTC and presents them according to the client time zone. Keep service contracts time-zone explicit and perform a conversion only when the business requirement identifies a particular zone. + +See sample: [`do-not-hard-code-time-zone-offsets.good.al`](do-not-hard-code-time-zone-offsets.good.al). + +## Anti Pattern + +Adding or subtracting a fixed duration solely to convert `CurrentDateTime` or another timestamp to an assumed local time. Detection signals include fixed hour-sized millisecond values near `DateTime` assignments and comments naming a specific time zone; confirm the duration is an offset rather than a legitimate deadline or schedule interval. + +See sample: [`do-not-hard-code-time-zone-offsets.bad.al`](do-not-hard-code-time-zone-offsets.bad.al). \ No newline at end of file diff --git a/community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.bad.al b/community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.bad.al new file mode 100644 index 0000000..d062837 --- /dev/null +++ b/community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.bad.al @@ -0,0 +1,21 @@ +codeunit 50100 "Rental Service" +{ + [ServiceEnabled] + procedure CloseAgreement(AgreementNo: Code[20]): Boolean + var + RentalAgreement: Record "Rental Agreement"; + begin + if not Confirm(CloseAgreementQst, false, AgreementNo) then + exit(false); + + RentalAgreement.Get(AgreementNo); + RentalAgreement.Closed := true; + RentalAgreement.Modify(true); + Message(AgreementClosedMsg, AgreementNo); + exit(true); + end; + + var + CloseAgreementQst: Label 'Close rental agreement %1?'; + AgreementClosedMsg: Label 'Rental agreement %1 was closed.'; +} \ No newline at end of file diff --git a/community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.good.al b/community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.good.al new file mode 100644 index 0000000..c990850 --- /dev/null +++ b/community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.good.al @@ -0,0 +1,15 @@ +codeunit 50100 "Rental Service" +{ + [ServiceEnabled] + procedure CloseAgreement(AgreementNo: Code[20]): Boolean + var + RentalAgreement: Record "Rental Agreement"; + begin + if not RentalAgreement.Get(AgreementNo) then + exit(false); + + RentalAgreement.Closed := true; + RentalAgreement.Modify(true); + exit(true); + end; +} \ No newline at end of file diff --git a/community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.md b/community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.md new file mode 100644 index 0000000..45b06b2 --- /dev/null +++ b/community/knowledge/appsource/keep-web-service-paths-free-of-ui-calls.md @@ -0,0 +1,26 @@ +--- +bc-version: [all] +domain: appsource +keywords: [web-service, serviceenabled, guiallowed, message, confirm, strmenu] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Keep web-service paths free of UI calls + +## Description + +Pages and codeunits exposed as web services run without an interactive client. Calls that require a UI callback, including `Confirm`, `StrMenu`, and modal pages, can terminate the service request instead of completing the operation. `Message` does not raise the callback error: the message is suppressed and logged, making it ineffective for communicating a service result. + +## Best Practice + +Keep service entry points and every procedure they call free of interactive UI. Return data through the service contract and report validation failures with service-safe error handling. When a procedure is shared with an interactive client, guard UI-only behavior with `GuiAllowed` while preserving the underlying operation. + +See sample: [`keep-web-service-paths-free-of-ui-calls.good.al`](keep-web-service-paths-free-of-ui-calls.good.al). + +## Anti Pattern + +A web-service-exposed page or codeunit calls an interactive UI method directly or indirectly. Detection signals include `Message`, `Confirm`, `StrMenu`, `Page.RunModal`, and confirmation-dialog pages on a service call path. Treat `Message` as suppressed and ineffective, not as a callback failure. Do not flag a controlled `Error` solely because it returns a service fault. + +See sample: [`keep-web-service-paths-free-of-ui-calls.bad.al`](keep-web-service-paths-free-of-ui-calls.bad.al). \ No newline at end of file diff --git a/community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.bad.al b/community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.bad.al new file mode 100644 index 0000000..81e8f83 --- /dev/null +++ b/community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.bad.al @@ -0,0 +1,15 @@ +pageextension 50100 "Rental Customer List" extends "Customer List" +{ + actions + { + addafter("Customer Ledger Entries") + { + action(OpenRentalAgreements) + { + ApplicationArea = All; + Caption = 'Rental Agreements'; + RunObject = page "Rental Agreement List"; + } + } + } +} \ No newline at end of file diff --git a/community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.good.al b/community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.good.al new file mode 100644 index 0000000..155a211 --- /dev/null +++ b/community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.good.al @@ -0,0 +1,15 @@ +pageextension 50100 "Rental Customer List" extends "Customer List" +{ + actions + { + addlast(Processing) + { + action(OpenRentalAgreements) + { + ApplicationArea = All; + Caption = 'Rental Agreements'; + RunObject = page "Rental Agreement List"; + } + } + } +} \ No newline at end of file diff --git a/community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.md b/community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.md new file mode 100644 index 0000000..4967645 --- /dev/null +++ b/community/knowledge/appsource/place-page-extension-actions-with-addfirst-or-addlast.md @@ -0,0 +1,26 @@ +--- +bc-version: [all] +domain: appsource +keywords: [pageextension, actions, addfirst, addlast, addbefore, addafter] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Place page extension actions with addfirst or addlast + +## Description + +Place new page-extension actions at the beginning or end of an existing action group with `addfirst` or `addlast`. Anchoring a new action relative to a specific base-app action with `addbefore` or `addafter` couples the extension to an implementation detail that can move or disappear between Business Central releases. + +## Best Practice + +Choose the semantic action area or group and append or prepend the extension's actions. This keeps placement deterministic without depending on the continued existence of one neighboring action. + +See sample: [`place-page-extension-actions-with-addfirst-or-addlast.good.al`](place-page-extension-actions-with-addfirst-or-addlast.good.al). + +## Anti Pattern + +Using `addbefore` or `addafter` to place newly added actions next to a specific action from another app. The syntax is valid AL, but the placement anchor is brittle for a Marketplace extension. + +See sample: [`place-page-extension-actions-with-addfirst-or-addlast.bad.al`](place-page-extension-actions-with-addfirst-or-addlast.bad.al). \ No newline at end of file diff --git a/community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.bad.al b/community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.bad.al new file mode 100644 index 0000000..d648f4a --- /dev/null +++ b/community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.bad.al @@ -0,0 +1,20 @@ +page 50100 "Rental Agreement List" +{ + PageType = List; + SourceTable = "Rental Agreement"; + ApplicationArea = All; + + layout + { + area(Content) + { + repeater(Agreements) + { + field("No."; Rec."No.") + { + ApplicationArea = All; + } + } + } + } +} \ No newline at end of file diff --git a/community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.good.al b/community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.good.al new file mode 100644 index 0000000..97fe848 --- /dev/null +++ b/community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.good.al @@ -0,0 +1,21 @@ +page 50100 "Rental Agreement List" +{ + PageType = List; + SourceTable = "Rental Agreement"; + ApplicationArea = All; + UsageCategory = Lists; + + layout + { + area(Content) + { + repeater(Agreements) + { + field("No."; Rec."No.") + { + ApplicationArea = All; + } + } + } + } +} \ No newline at end of file diff --git a/community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.md b/community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.md new file mode 100644 index 0000000..9241236 --- /dev/null +++ b/community/knowledge/appsource/set-usagecategory-on-searchable-entry-points.md @@ -0,0 +1,26 @@ +--- +bc-version: [all] +domain: appsource +keywords: [usagecategory, tell-me, search, page, report, discoverability] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Set UsageCategory on searchable entry points + +## Description + +Pages and reports that users are expected to open directly must set `UsageCategory`. Without it, the object is absent from Tell Me and users cannot bookmark it from the web client. Supporting objects such as list parts, dialogs, API pages, and objects reached only through another page do not need to be searchable entry points. + +## Best Practice + +Set `UsageCategory` to the category that matches the entry point, such as `Lists`, `Tasks`, `ReportsAndAnalysis`, or `Documents`. Also set the appropriate object-level `ApplicationArea` so search results respect feature visibility. + +See sample: [`set-usagecategory-on-searchable-entry-points.good.al`](set-usagecategory-on-searchable-entry-points.good.al). + +## Anti Pattern + +A user-facing page or report intended for direct discovery omits `UsageCategory` or sets it to `None`. Do not infer intent from the object type alone; require evidence that the object is a direct user entry point. + +See sample: [`set-usagecategory-on-searchable-entry-points.bad.al`](set-usagecategory-on-searchable-entry-points.bad.al). \ No newline at end of file diff --git a/community/knowledge/appsource/use-invariant-date-literals.bad.al b/community/knowledge/appsource/use-invariant-date-literals.bad.al new file mode 100644 index 0000000..437083d --- /dev/null +++ b/community/knowledge/appsource/use-invariant-date-literals.bad.al @@ -0,0 +1,10 @@ +codeunit 50100 "Rental Period Defaults" +{ + procedure GetPolicyStartDate(): Date + var + PolicyStartDate: Date; + begin + Evaluate(PolicyStartDate, '01/31/2025'); + exit(PolicyStartDate); + end; +} \ No newline at end of file diff --git a/community/knowledge/appsource/use-invariant-date-literals.good.al b/community/knowledge/appsource/use-invariant-date-literals.good.al new file mode 100644 index 0000000..07b19c4 --- /dev/null +++ b/community/knowledge/appsource/use-invariant-date-literals.good.al @@ -0,0 +1,7 @@ +codeunit 50100 "Rental Period Defaults" +{ + procedure GetPolicyStartDate(): Date + begin + exit(20250131D); + end; +} \ No newline at end of file diff --git a/community/knowledge/appsource/use-invariant-date-literals.md b/community/knowledge/appsource/use-invariant-date-literals.md new file mode 100644 index 0000000..c38476c --- /dev/null +++ b/community/knowledge/appsource/use-invariant-date-literals.md @@ -0,0 +1,26 @@ +--- +bc-version: [all] +domain: appsource +keywords: [date-literal, invariant-date, dateformula, localization, appsourcecop] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Use invariant date literals + +## Description + +Write fixed dates in AL with the invariant `yyyymmddD` syntax. A locale-dependent text value parsed with `Evaluate` can change meaning or fail under another user's regional settings, which makes the Marketplace extension unreliable across markets. + +## Best Practice + +Represent a fixed date directly as an AL date literal, such as `20250131D`. Use `CalcDate` with a date formula when the value is relative rather than fixed. + +See sample: [`use-invariant-date-literals.good.al`](use-invariant-date-literals.good.al). + +## Anti Pattern + +Building a fixed date by passing localized text such as `01/02/2025` to `Evaluate`. Detection signal: `Evaluate` converting a hard-coded or label-backed formatted string into a `Date`. + +See sample: [`use-invariant-date-literals.bad.al`](use-invariant-date-literals.bad.al). \ No newline at end of file diff --git a/microsoft/skills/review/al-appsource-review.md b/microsoft/skills/review/al-appsource-review.md index c851ea2..3f6a221 100644 --- a/microsoft/skills/review/al-appsource-review.md +++ b/microsoft/skills/review/al-appsource-review.md @@ -16,7 +16,7 @@ application-area: [all] Reviews AL source and app metadata changes against the `appsource` knowledge domain in BCQuality and emits a findings report. This is a leaf action skill: it invokes no sub-skills. It is one of the skills composed by `al-code-review`. -An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-path`. AppSource findings are narrow by design — they apply to AppSource-facing metadata and complete permission coverage that requires repository context. Mechanical AppSourceCop diagnostics are intentionally outside this skill. The skill returns `not-applicable` when none of those surfaces apply. +An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-path`. AppSource findings are narrow by design — they apply to Marketplace-facing metadata, complete permission coverage that requires repository context, and contextual AL constructs covered by Marketplace submission requirements. Mechanical compiler and analyzer diagnostics are intentionally outside this skill. The skill returns `not-applicable` when none of those surfaces apply. ## Source @@ -37,14 +37,20 @@ Discard files that are not applicable. Retain conditionally applicable files (an Narrow the relevant files to the subset that applies to the changes under review. For each relevant file, compute overlap against: -- The changed files and AL object types — especially `app.json`, permission-set objects, setup and usage entry points, and AppSource-facing help metadata. -- Tokens extracted from the diff that relate to AppSource (`permissionset`, `Assignable`, `Permissions`, `SUPER`, `tabledata`, `execute`, `app.json`, `help`, `ContextSensitiveHelpPage`, `Copilot`, `https`). +- The changed files and AL object types — especially `app.json`, permission-set and profile objects, setup and usage entry points, service-enabled procedures, user-facing pages and reports, and AppSource-facing help metadata. +- Tokens extracted from the diff that relate to AppSource (`permissionset`, `Assignable`, `Permissions`, `SUPER`, `tabledata`, `execute`, `profile`, `Record Profile`, `Evaluate`, `Date`, `DateTime`, `CurrentDateTime`, `UsageCategory`, `PageType`, `addfirst`, `addlast`, `addbefore`, `addafter`, `ServiceEnabled`, `GuiAllowed`, `Message`, `Confirm`, `StrMenu`, `RunModal`, `app.json`, `help`, `ContextSensitiveHelpPage`, `Copilot`, `https`). A file enters the candidate worklist when its `keywords` intersect the extracted tokens or its topic (derived from the index entry's `path`, `title`, and `description`) matches a changed object type. Read an article's full file — its `## Best Practice` / `## Anti Pattern` bodies — only after it makes the worklist; candidate selection uses the index alone. When the diff contains no AppSource-related source or metadata changes by any of the above signals, return `outcome: "not-applicable"` without evaluating files. The following targeted checks cover every current `appsource` article across the Microsoft and community layers. Treat each as a candidate-selection cue: when the signal appears in changed code, add the named article to the worklist and evaluate it in Action. - The app has no assignable permission set covering its setup and usage paths, omits visible object/tabledata grants, or requires `SUPER` for normal operation — `permission-sets-cover-setup-and-usage-without-super`. Require repository-level app context; one isolated permission-set object cannot prove complete coverage. +- Install, upgrade, or setup code provisions an app-owned profile through `Record Profile` and `Insert` instead of declaring a `profile` object — `define-profiles-as-al-objects`. +- A hard-coded or label-backed formatted string is converted to `Date` with `Evaluate` — `use-invariant-date-literals`. Do not select this article for variable external input whose format must be validated at runtime. +- A page extension uses `addbefore` or `addafter` to place a newly added action relative to a specific action owned by another app — `place-page-extension-actions-with-addfirst-or-addlast`. Do not flag those keywords in layouts or placement relative to an action owned by the same extension. +- A page or codeunit web-service entry point, including a `[ServiceEnabled]` procedure, contains or reaches `Message`, `Confirm`, `StrMenu`, `Page.RunModal`, or a confirmation-dialog page without an effective non-GUI guard — `keep-web-service-paths-free-of-ui-calls`. Treat `Message` as suppressed and logged, making it ineffective as a service response; treat the other UI calls as callback-failure risks. Do not treat a controlled `Error` as interactive UI solely because it returns a service fault. +- A page or report that repository context identifies as a direct user entry point omits `UsageCategory` or sets it to `None` — `set-usagecategory-on-searchable-entry-points`. Do not select this article based only on object type; exclude supporting parts, dialogs, API pages, and objects intentionally reached through another page. +- A `DateTime` assignment adds or subtracts a fixed duration to represent an assumed regional offset — `do-not-hard-code-time-zone-offsets`. Require contextual evidence such as an hour-sized constant, offset-oriented name, or time-zone comment; do not flag deadlines, schedules, or elapsed-time calculations. - For BC v27 or later, `app.json` adds or changes the `help` URL to a path deeper than two levels, or a changed Copilot/context-sensitive help arrangement would ground the app under an overly broad truncated parent — `keep-copilot-help-url-to-two-path-levels`. Once the candidate worklist is known, resolve layer-precedence conflicts per READ. Drop lower-precedence files whose normative guidance (`## Best Practice` or `## Anti Pattern`) directly contradicts a higher-precedence candidate, and record each dropped file in `suppressed` with `reason: "layer-precedence"`. Files that would have been candidates but are hidden because their layer is disabled in consumer configuration are recorded with `reason: "configuration"`. Files that never became candidates are NOT recorded in `suppressed`. @@ -75,7 +81,7 @@ Outcome selection: - `completed` — the skill evaluated every worklist item. - `no-knowledge` — no applicable AppSource knowledge survived filtering. -- `not-applicable` — the diff touches no AppSource permission or app-metadata surface. +- `not-applicable` — the diff touches no Marketplace-related source, permission, or app-metadata surface. - `partial` — a budget was hit before the worklist was exhausted. - `failed` — an unrecoverable error occurred. From 8c26ba4e7640fd613e60a40734c744f0f3bda40b Mon Sep 17 00:00:00 2001 From: Stefano Demiliani <33155438+demiliani@users.noreply.github.com> Date: Mon, 14 Sep 2026 12:44:30 +0200 Subject: [PATCH 04/19] Add Job Queue reliability and scheduling guidance (#148) * knowledge(performance): add job queue reliability guidance * Address Job Queue review feedback * Address Job Queue routing review feedback * Encode job queue conflict in fixtures --- evaluation/README.md | 4 +- evaluation/review-fixtures.json | 10 ++- ...nt-inside-write-transaction-holds-locks.md | 2 +- ...ry-code-serializes-conflicting-jobs.bad.al | 11 +++ ...y-code-serializes-conflicting-jobs.good.al | 18 +++++ ...tegory-code-serializes-conflicting-jobs.md | 28 +++++++ ...external-effects-must-be-idempotent.bad.al | 57 ++++++++++++++ ...xternal-effects-must-be-idempotent.good.al | 65 ++++++++++++++++ ...eue-external-effects-must-be-idempotent.md | 30 ++++++++ ...-queue-handlers-must-not-require-ui.bad.al | 17 ++++ ...queue-handlers-must-not-require-ui.good.al | 14 ++++ .../job-queue-handlers-must-not-require-ui.md | 28 +++++++ ...ue-handlers-must-propagate-failures.bad.al | 23 ++++++ ...e-handlers-must-propagate-failures.good.al | 16 ++++ ...-queue-handlers-must-propagate-failures.md | 28 +++++++ ...-on-hold-does-not-stop-running-work.bad.al | 18 +++++ ...on-hold-does-not-stop-running-work.good.al | 49 ++++++++++++ ...ueue-on-hold-does-not-stop-running-work.md | 28 +++++++ ...ncompanyopen-subscribers-must-not-do-io.md | 2 +- ...ed-task-id-to-avoid-duplicate-tasks.bad.al | 15 ++++ ...d-task-id-to-avoid-duplicate-tasks.good.al | 23 ++++++ ...eduled-task-id-to-avoid-duplicate-tasks.md | 28 +++++++ .../skills/review/al-performance-review.md | 8 +- tools/Test-ReviewFixtures.ps1 | 77 ++++++++++++++----- 24 files changed, 574 insertions(+), 25 deletions(-) create mode 100644 microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.bad.al create mode 100644 microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.good.al create mode 100644 microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.md create mode 100644 microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.bad.al create mode 100644 microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.good.al create mode 100644 microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.md create mode 100644 microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.bad.al create mode 100644 microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.good.al create mode 100644 microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.md create mode 100644 microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.bad.al create mode 100644 microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.good.al create mode 100644 microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.md create mode 100644 microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.bad.al create mode 100644 microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.good.al create mode 100644 microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.md create mode 100644 microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.bad.al create mode 100644 microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.good.al create mode 100644 microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.md diff --git a/evaluation/README.md b/evaluation/README.md index 7be55b4..2125ce3 100644 --- a/evaluation/README.md +++ b/evaluation/README.md @@ -2,7 +2,7 @@ The evaluation is convention-driven. The harness discovers every `/skills/review/al--review.md` leaf across the enabled `microsoft`, `community`, and `custom` layers. Duplicate domains resolve with `custom > community > microsoft` precedence. For each selected leaf, the harness finds paired knowledge across the same layers, applies the same precedence to duplicate article slugs, selects the first article (by filename) with both `.bad.al` and `.good.al` companions, and derives the expected positive and clean control automatically. Adding a conforming leaf requires no scoring-contract edit. -`review-fixtures.json` contains only global thresholds and optional exceptional overrides. An override may select a different article or add context when the generic convention cannot express a scenario. It should remain empty in the normal case. +`review-fixtures.json` contains only global thresholds and optional exceptional overrides. An override may select a different `article`, add context when the generic convention cannot express a scenario, or use an `articles` array when one domain needs explicit regression coverage for several paired articles. Specify either `article` or `articles`, not both. The first selected article retains the stable `-bad` and `-good` manifest IDs; additional articles use slug-qualified IDs. Overrides should remain empty in the normal case. Model-facing preparation hashes case IDs, neutralizes `Good`/`Bad` object-name tokens, and removes full-line sample comments so neither the article slug, domain, nor expected outcome reveals the answer. @@ -26,7 +26,7 @@ This credential-free check proves every selected leaf maps to a same-named knowl 2. For a fast/small model, use one fresh invocation per `request-case-*.json`. Each request embeds the exact leaf instructions, that domain's candidate index rows with authoritative paths, and one opaque case. The model opens only matching articles and copies finding IDs from `candidateArticles[].path`. Save each response with the matching `result-case-*.json` name in the same directory. - `request-.json` files provide optional two-case leaf batches and identify the selected layer-owned skill path; save those as `result-.json`. Directory scoring prefers `result-case-*.json` when present and otherwise falls back to `result-*.json`. `review-request.json` is an optional all-domains stress test for larger models. Neither batch form is the preferred fast-model profile. + `request-.json` files provide optional leaf batches containing every selected case for that domain and identify the selected layer-owned skill path; save those as `result-.json`. A normal convention-selected domain has one bad/good pair, while an `articles` override contributes one pair per listed article. Directory scoring prefers `result-case-*.json` when present and otherwise falls back to `result-*.json`. `review-request.json` is an optional all-domains stress test for larger models. Neither batch form is the preferred fast-model profile. 3. Save only this result shape: diff --git a/evaluation/review-fixtures.json b/evaluation/review-fixtures.json index e11508a..352fccf 100644 --- a/evaluation/review-fixtures.json +++ b/evaluation/review-fixtures.json @@ -17,7 +17,15 @@ "article": "set-defaultimplementation-on-enum" }, "performance": { - "article": "use-isempty-for-existence-check" + "articles": [ + "use-isempty-for-existence-check", + "job-queue-category-code-serializes-conflicting-jobs", + "job-queue-external-effects-must-be-idempotent", + "job-queue-handlers-must-not-require-ui", + "job-queue-handlers-must-propagate-failures", + "job-queue-on-hold-does-not-stop-running-work", + "store-scheduled-task-id-to-avoid-duplicate-tasks" + ] }, "privacy": { "article": "no-pii-in-telemetry-message-string" diff --git a/microsoft/knowledge/performance/httpclient-inside-write-transaction-holds-locks.md b/microsoft/knowledge/performance/httpclient-inside-write-transaction-holds-locks.md index 21a2c04..88d0ff7 100644 --- a/microsoft/knowledge/performance/httpclient-inside-write-transaction-holds-locks.md +++ b/microsoft/knowledge/performance/httpclient-inside-write-transaction-holds-locks.md @@ -17,7 +17,7 @@ The first database write opens an AL write transaction that the runtime holds un ## Best Practice -Defer the HTTP call to a separate session. When the external operation must correspond to a committed database change, insert an outbox work item in the same transaction as that change and process committed outbox rows with a recurring job queue entry. The change and work item then commit or roll back together, and the worker performs HTTP before deleting the item so it holds no write lock during the call. Make the external operation idempotent because a failure after a successful HTTP response can cause the work item to be retried. +Defer the HTTP call to a separate session. When the external operation must correspond to a committed database change, insert an outbox work item in the same transaction as that change and process committed outbox rows with a recurring job queue entry. The change and work item then commit or roll back together, and the worker performs HTTP before deleting the item so it holds no write lock during the call. The separate retry-safety requirement is covered by `job-queue-external-effects-must-be-idempotent.md`. A directly created scheduled task is suitable only when its work is independent of the caller's commit. An immediately ready task can run concurrently with the caller, so it must not assume that the caller's writes are already committed. Do **not** use `Commit()` as a general remedy: it irrevocably commits all prior writes in the current transaction, so any subsequent failure cannot roll them back. `Commit()` is appropriate only at top-level entry points where partial persistence is intentional and understood. diff --git a/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.bad.al b/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.bad.al new file mode 100644 index 0000000..fbd0b39 --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.bad.al @@ -0,0 +1,11 @@ +codeunit 50113 "Job Queue Category Bad" +{ + procedure ConfigureJobsForSharedExclusiveResource(var SalesPostingJob: Record "Job Queue Entry"; var PurchasePostingJob: Record "Job Queue Entry"; ExclusiveResourceId: Text[250]) + begin + // Both jobs update the same posting resources, but nothing prevents overlap. + SalesPostingJob.Validate("Parameter String", ExclusiveResourceId); + PurchasePostingJob.Validate("Parameter String", ExclusiveResourceId); + SalesPostingJob.Validate("Job Queue Category Code", ''); + PurchasePostingJob.Validate("Job Queue Category Code", ''); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.good.al b/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.good.al new file mode 100644 index 0000000..f77200b --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.good.al @@ -0,0 +1,18 @@ +codeunit 50113 "Job Queue Category Good" +{ + procedure ConfigureJobsForSharedExclusiveResource(var SalesPostingJob: Record "Job Queue Entry"; var PurchasePostingJob: Record "Job Queue Entry"; ExclusiveResourceId: Text[250]) + var + JobQueueCategory: Record "Job Queue Category"; + begin + if not JobQueueCategory.Get('POSTING') then begin + JobQueueCategory.Code := 'POSTING'; + JobQueueCategory.Insert(); + end; + + // The shared category lets only one conflicting posting job run at a time. + SalesPostingJob.Validate("Parameter String", ExclusiveResourceId); + PurchasePostingJob.Validate("Parameter String", ExclusiveResourceId); + SalesPostingJob.Validate("Job Queue Category Code", 'POSTING'); + PurchasePostingJob.Validate("Job Queue Category Code", 'POSTING'); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.md b/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.md new file mode 100644 index 0000000..d92647e --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.md @@ -0,0 +1,28 @@ +--- +bc-version: [all] +domain: performance +keywords: [job-queue, category-code, concurrency, waiting, serialization, locking] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Use a job queue category to serialize conflicting jobs + +> Contributions welcome — open a PR to refine or extend this article. + +## Description + +Different job queue entries can run at the same time. When two jobs update the same exclusive resource, concurrent execution can cause lock contention, deadlocks, or conflicting results. Within one company, entries with the same Job Queue Category Code are serialized: while one runs, another entry in that category waits. + +## Best Practice + +Assign the same non-empty Job Queue Category Code to job queue entries in the same company that must not overlap, regardless of which codeunit they run. Define categories around the shared resource or exclusivity requirement, not merely around object names. Leave independent jobs in different categories so they can still run concurrently. A category does not serialize work across companies or environments, or coordinate workers outside the job queue dispatcher. Protect shared external or cross-company resources with a separate application-level locking mechanism. + +See sample: `job-queue-category-code-serializes-conflicting-jobs.good.al`. + +## Anti Pattern + +Creating or configuring multiple job queue entries that update the same exclusive resource while leaving their Job Queue Category Code empty or different. Do not flag jobs merely because they touch the same tables; the rule applies when their operation requires mutual exclusion. + +See sample: `job-queue-category-code-serializes-conflicting-jobs.bad.al`. \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.bad.al b/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.bad.al new file mode 100644 index 0000000..8f92f10 --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.bad.al @@ -0,0 +1,57 @@ +table 50112 "Queued Export Bad" +{ + DataClassification = CustomerContent; + + fields + { + field(1; "Entry No."; Integer) + { + AutoIncrement = true; + } + field(2; Payload; Text[250]) + { + } + } + + keys + { + key(PK; "Entry No.") + { + Clustered = true; + } + } +} + +codeunit 50112 "Queued Export Worker Bad" +{ + TableNo = "Job Queue Entry"; + + trigger OnRun() + var + QueuedExport: Record "Queued Export Bad"; + Client: HttpClient; + Content: HttpContent; + Response: HttpResponseMessage; + begin + if not QueuedExport.FindFirst() then + exit; + + Content.WriteFrom(QueuedExport.Payload); + Client.Post('https://example.local/exports', Content, Response); + if not Response.IsSuccessStatusCode() then + Error('Export failed with HTTP status %1.', Response.HttpStatusCode()); + + // If this local step fails, the external export exists but this row is retried. + UpdateLocalStatus(); + FinalizeExport(QueuedExport); + end; + + local procedure UpdateLocalStatus() + begin + end; + + local procedure FinalizeExport(var QueuedExport: Record "Queued Export Bad") + begin + QueuedExport.Delete(); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.good.al b/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.good.al new file mode 100644 index 0000000..d161089 --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.good.al @@ -0,0 +1,65 @@ +table 50112 "Queued Export Good" +{ + DataClassification = CustomerContent; + + fields + { + field(1; "Entry No."; Integer) + { + AutoIncrement = true; + } + field(2; Payload; Text[250]) + { + } + } + + keys + { + key(PK; "Entry No.") + { + Clustered = true; + } + } + +} + +codeunit 50112 "Queued Export Worker Good" +{ + TableNo = "Job Queue Entry"; + + trigger OnRun() + var + QueuedExport: Record "Queued Export Good"; + Client: HttpClient; + Content: HttpContent; + ContentHeaders: HttpHeaders; + JsonPayload: JsonObject; + RequestBody: Text; + Response: HttpResponseMessage; + begin + if not QueuedExport.FindFirst() then + exit; + + JsonPayload.Add('idempotencyKey', Format(QueuedExport.SystemId)); + JsonPayload.Add('payload', QueuedExport.Payload); + JsonPayload.WriteTo(RequestBody); + + Content.WriteFrom(RequestBody); + Content.GetHeaders(ContentHeaders); + ContentHeaders.Clear(); + ContentHeaders.Add('Content-Type', 'application/json'); + Client.Post('https://example.local/exports', Content, Response); + if not Response.IsSuccessStatusCode() then + Error('Export failed with HTTP status %1.', Response.HttpStatusCode()); + + // The external service must atomically create a record only when idempotencyKey + // does not exist. When the key already exists, it must return the existing record + // without repeating the side effect. + UpdateLocalStatus(); + QueuedExport.Delete(); + end; + + local procedure UpdateLocalStatus() + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.md b/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.md new file mode 100644 index 0000000..a5932df --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.md @@ -0,0 +1,30 @@ +--- +bc-version: [all] +domain: performance +keywords: [job-queue, idempotency, retry, outbox, httpclient, external-effect] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Job queue external effects must be idempotent + +> Contributions welcome — open a PR to refine or extend this article. + +## Description + +A job queue handler can successfully create something in an external system and then fail while updating Business Central. Business Central rolls back its database changes, but it cannot roll back the external request. The same work can later run again through configured retries, recurrence, rescheduling, or manual restart. Without a way for the external system to recognize the repeated request, a later run can create a duplicate shipment, payment, notification, or other side effect. + +## Best Practice + +Use a stable request ID that exists before the job queue processes the outbox row. For example, include the outbox record's `SystemId` as an `idempotencyKey` value in the JSON body of every POST attempt. The external service must enforce uniqueness on that value: when it receives the key again, it returns the existing record instead of creating another one. Delete the outbox row only after the external call and all required local updates succeed. + +A `Processed` flag set after the external call does not solve this failure window. If a later AL error rolls back that flag, the outbox row again looks unprocessed even though the external operation already happened. + +See sample: `job-queue-external-effects-must-be-idempotent.good.al`. + +## Anti Pattern + +Sending a state-changing request from a job queue handler with no stable request ID understood by the external API. Specifically, look for this sequence: read an outbox row, call `HttpClient.Post` or another side-effecting API, update or delete local data, and propagate an error after which the same outbox row can be processed again. The key may be part of the request body, URI, headers, or an existing business key; a naturally idempotent remote operation is already safe and should not be flagged. + +See sample: `job-queue-external-effects-must-be-idempotent.bad.al`. \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.bad.al b/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.bad.al new file mode 100644 index 0000000..eecfeaf --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.bad.al @@ -0,0 +1,17 @@ +codeunit 50110 "Job Queue UI Bad" +{ + TableNo = "Job Queue Entry"; + + trigger OnRun() + begin + if not Confirm('Process the queued export now?') then + exit; + + ProcessExport(Rec."Parameter String"); + Message('The queued export completed.'); + end; + + local procedure ProcessExport(ParameterString: Text) + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.good.al b/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.good.al new file mode 100644 index 0000000..490720e --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.good.al @@ -0,0 +1,14 @@ +codeunit 50110 "Job Queue UI Good" +{ + TableNo = "Job Queue Entry"; + + trigger OnRun() + begin + Rec.TestField("Parameter String"); + ProcessExport(Rec."Parameter String"); + end; + + local procedure ProcessExport(ParameterString: Text) + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.md b/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.md new file mode 100644 index 0000000..f780ab5 --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.md @@ -0,0 +1,28 @@ +--- +bc-version: [all] +domain: performance +keywords: [job-queue, background-session, guiallowed, confirm, runmodal, client-callback] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Job queue handlers must not require user interaction + +> Contributions welcome — open a PR to refine or extend this article. + +## Description + +A job queue handler runs in a background session with no client UI. Calls that require a client callback, such as `Confirm`, `Page.RunModal`, `Report.RunModal`, upload, or download, can stop the job with a non-retriable callback error. `Message` is suppressed and logged by the server, so it cannot communicate a result to the user who scheduled the job. + +## Best Practice + +Make a dedicated job queue entry point non-interactive. Validate parameters and data in AL, persist business-visible status when needed, and let failures propagate to the job queue log. If one procedure genuinely serves both foreground and background callers, isolate optional UI-only behavior behind `GuiAllowed`; do not use the guard to silently skip a decision that the operation requires. + +See sample: `job-queue-handlers-must-not-require-ui.good.al`. + +## Anti Pattern + +Calling `Confirm`, `Page.Run`, `Page.RunModal`, `Report.Run`, `Report.RunModal`, `Hyperlink`, `File.Upload`, or `File.Download` from a codeunit run by the job queue. Another signal is using `Message` as the only success or failure notification: no user is attached to receive it. + +See sample: `job-queue-handlers-must-not-require-ui.bad.al`. \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.bad.al b/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.bad.al new file mode 100644 index 0000000..550506a --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.bad.al @@ -0,0 +1,23 @@ +codeunit 50111 "Job Queue Failure Bad" +{ + TableNo = "Job Queue Entry"; + + trigger OnRun() + begin + if not TryProcessCustomer(Rec."Parameter String") then + exit; + end; + + [TryFunction] + local procedure TryProcessCustomer(CustomerNo: Code[20]) + var + Customer: Record Customer; + begin + Customer.Get(CustomerNo); + ProcessCustomer(Customer); + end; + + local procedure ProcessCustomer(Customer: Record Customer) + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.good.al b/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.good.al new file mode 100644 index 0000000..8a99875 --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.good.al @@ -0,0 +1,16 @@ +codeunit 50111 "Job Queue Failure Good" +{ + TableNo = "Job Queue Entry"; + + trigger OnRun() + var + Customer: Record Customer; + begin + Customer.Get(Rec."Parameter String"); + ProcessCustomer(Customer); + end; + + local procedure ProcessCustomer(Customer: Record Customer) + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.md b/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.md new file mode 100644 index 0000000..1c7b83f --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.md @@ -0,0 +1,28 @@ +--- +bc-version: [all] +domain: performance +keywords: [job-queue, error-propagation, tryfunction, retry, dispatcher, job-queue-log] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Job queue handlers must propagate execution failures + +> Contributions welcome — open a PR to refine or extend this article. + +## Description + +The job queue dispatcher can mark an entry as failed, record the error, and apply its configured retry behavior only when the handler terminates with an error. A handler that catches a failed `TryFunction` or Boolean-returning operation and then returns normally reports success to the dispatcher, even though its work did not complete. + +## Best Practice + +Let an error that invalidates the whole run propagate out of the job queue entry point. Add context only when it helps an operator diagnose the failure and does not expose sensitive data. Per-item failures may be collected deliberately, but the batch must persist or emit an observable aggregate outcome instead of silently treating incomplete work as success. + +See sample: `job-queue-handlers-must-propagate-failures.good.al`. + +## Anti Pattern + +Calling a `TryFunction`, `Codeunit.Run`, or another Boolean-returning operation from a job queue handler and using `exit` or normal fall-through on failure without recording an intentional partial-success outcome. The dispatcher sees a successful return, so the entry's status and log do not represent the failed work and configured retries are not applied. + +See sample: `job-queue-handlers-must-propagate-failures.bad.al`. \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.bad.al b/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.bad.al new file mode 100644 index 0000000..434f2b9 --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.bad.al @@ -0,0 +1,18 @@ +codeunit 50114 "Job Queue On Hold Bad" +{ + TableNo = "Job Queue Entry"; + + trigger OnRun() + begin + repeat + if not ProcessNextBatch() then + exit; + Rec.Get(Rec.ID); + until Rec.Status = Rec.Status::"On Hold"; + end; + + local procedure ProcessNextBatch(): Boolean + begin + exit(false); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.good.al b/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.good.al new file mode 100644 index 0000000..b924297 --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.good.al @@ -0,0 +1,49 @@ +table 50114 "Job Cancellation Control" +{ + DataClassification = SystemMetadata; + + fields + { + field(1; "Job Queue Entry ID"; Guid) + { + } + field(2; "Stop Requested"; Boolean) + { + } + } + + keys + { + key(PK; "Job Queue Entry ID") + { + Clustered = true; + } + } +} + +codeunit 50114 "Job Queue On Hold Good" +{ + TableNo = "Job Queue Entry"; + + trigger OnRun() + begin + while not IsStopRequested(Rec.ID) do + if not ProcessNextBatch() then + exit; + end; + + local procedure IsStopRequested(JobQueueEntryId: Guid): Boolean + var + JobCancellationControl: Record "Job Cancellation Control"; + begin + if not JobCancellationControl.Get(JobQueueEntryId) then + exit(false); + + exit(JobCancellationControl."Stop Requested"); + end; + + local procedure ProcessNextBatch(): Boolean + begin + exit(false); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.md b/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.md new file mode 100644 index 0000000..6eec05c --- /dev/null +++ b/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.md @@ -0,0 +1,28 @@ +--- +bc-version: [all] +domain: performance +keywords: [job-queue, on-hold, cancellation, in-process, long-running, stop-request] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Putting a job queue entry on hold does not stop its current run + +> Contributions welcome — open a PR to refine or extend this article. + +## Description + +The On Hold status prevents a job queue entry from starting again, but it does not cancel a run that is already in process. A long-running handler continues until it completes, fails, reaches a cancellation point implemented by the application, or its session is stopped externally. + +## Best Practice + +Use On Hold to pause future scheduling. When a long-running operation must support graceful cancellation, store a separate application-owned stop request and check it before every bounded unit of work, including the first. Exit only at a point where completed work and the checkpoint are consistent. The code that resumes scheduling must clear the stop request before restarting the job. Use administrative session termination only when graceful cancellation is impossible. + +See sample: `job-queue-on-hold-does-not-stop-running-work.good.al`. + +## Anti Pattern + +Polling the job queue entry's Status field from inside its handler and expecting a change to On Hold to cancel the active run. The status controls scheduling, not cooperative cancellation, so the handler can continue processing despite the operator's action. + +See sample: `job-queue-on-hold-does-not-stop-running-work.bad.al`. \ No newline at end of file diff --git a/microsoft/knowledge/performance/oncompanyopen-subscribers-must-not-do-io.md b/microsoft/knowledge/performance/oncompanyopen-subscribers-must-not-do-io.md index e8da1c5..95ac3d7 100644 --- a/microsoft/knowledge/performance/oncompanyopen-subscribers-must-not-do-io.md +++ b/microsoft/knowledge/performance/oncompanyopen-subscribers-must-not-do-io.md @@ -17,7 +17,7 @@ application-area: [all] ## Best Practice -Keep company-open subscribers to cheap in-memory work: set a flag, enqueue a job-queue entry, or `TaskScheduler.CreateTask`. Perform HTTP and large SQL after the session is running, in that background work. +Keep company-open subscribers to cheap in-memory work: set a flag, enqueue a job-queue entry, or `TaskScheduler.CreateTask`. Perform HTTP and large SQL after the session is running, in that background work. When the subscriber can run repeatedly, use `store-scheduled-task-id-to-avoid-duplicate-tasks.md` to avoid creating the same logical task more than once. See sample: [`oncompanyopen-subscribers-must-not-do-io.good.al`](oncompanyopen-subscribers-must-not-do-io.good.al). diff --git a/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.bad.al b/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.bad.al new file mode 100644 index 0000000..d692317 --- /dev/null +++ b/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.bad.al @@ -0,0 +1,15 @@ +codeunit 50115 "Scheduled Task Duplicate Bad" +{ + procedure EnsureCleanupTask() + begin + // Every call creates another task for the same cleanup work. + TaskScheduler.CreateTask(Codeunit::"Scheduled Cleanup Work Bad", 0, true, CompanyName()); + end; +} + +codeunit 50116 "Scheduled Cleanup Work Bad" +{ + trigger OnRun() + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.good.al b/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.good.al new file mode 100644 index 0000000..df52c62 --- /dev/null +++ b/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.good.al @@ -0,0 +1,23 @@ +codeunit 50115 "Scheduled Task Duplicate Good" +{ + internal procedure EnsureCleanupTask() + var + TaskId: Guid; + StoredTaskId: Text; + begin + if IsolatedStorage.Get('CleanupTaskId', DataScope::Company, StoredTaskId) then + if Evaluate(TaskId, StoredTaskId) then + if TaskScheduler.TaskExists(TaskId) then + exit; + + TaskId := TaskScheduler.CreateTask(Codeunit::"Scheduled Cleanup Work Good", 0, true, CompanyName()); + IsolatedStorage.Set('CleanupTaskId', Format(TaskId), DataScope::Company); + end; +} + +codeunit 50116 "Scheduled Cleanup Work Good" +{ + trigger OnRun() + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.md b/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.md new file mode 100644 index 0000000..600c671 --- /dev/null +++ b/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.md @@ -0,0 +1,28 @@ +--- +bc-version: [all] +domain: performance +keywords: [task-scheduler, scheduled-task, taskexists, duplicate-task, createtask, guid] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Store the scheduled task ID to avoid duplicate tasks + +> Contributions welcome — open a PR to refine or extend this article. + +## Description + +Every call to `TaskScheduler.CreateTask` creates a new scheduled task and returns its unique GUID. Repeating setup or lifecycle code without retaining that GUID can create multiple tasks for the same logical work, consuming scheduler capacity and running the work more than once. + +## Best Practice + +Persist the GUID returned by `CreateTask` at the same scope as the logical task. Before creating a replacement, parse the stored GUID and call `TaskScheduler.TaskExists`; create and store a new task only when the previous task no longer exists. `TaskExists` checks one GUID, not whether an equivalent codeunit is already scheduled, so callers that can schedule concurrently still need serialization around this check-and-create sequence. + +See sample: `store-scheduled-task-id-to-avoid-duplicate-tasks.good.al`. + +## Anti Pattern + +Calling `TaskScheduler.CreateTask` every time initialization, login, setup, or another repeatable path runs while ignoring its return value. Each invocation creates another independent task even when an equivalent task is already pending. + +See sample: `store-scheduled-task-id-to-avoid-duplicate-tasks.bad.al`. \ No newline at end of file diff --git a/microsoft/skills/review/al-performance-review.md b/microsoft/skills/review/al-performance-review.md index 664f20d..7829d70 100644 --- a/microsoft/skills/review/al-performance-review.md +++ b/microsoft/skills/review/al-performance-review.md @@ -39,7 +39,7 @@ Narrow the relevant files to the subset that applies to the changes under review - The changed AL object names and types — especially tables, pages with SourceTable bindings, reports, queries, and codeunits performing record iteration. - The changed procedures and triggers, weighted toward those that perform loops, Find/FindSet/FindFirst calls, CalcFields, SetAutoCalcFields, CalcSums, FlowField access, Commit calls, checkpoint helpers, record copying, RecordRef conversion, Modify/Delete calls, or cross-table navigation. -- Tokens extracted from the diff that relate to data access and hot-path costs (`SetRange`, `SetFilter`, `SetLoadFields`, `SetCurrentKey`, `FindSet`, `ReadIsolation`, `LockTable`, `ModifyAll`, `DeleteAll`, `Modify`, `Delete`, `Commit`, `checkpoint`, `Copy`, `RecordRef`, `GetTable`, `TextBuilder`, `Dictionary`, `temporary`, `repeat`, `until`, `CalcFields`, `SetAutoCalcFields`, `CalcSums`, `FlowField`, `Visible`). +- Tokens extracted from the diff that relate to data access, hot-path costs, and background scheduling (`SetRange`, `SetFilter`, `SetLoadFields`, `SetCurrentKey`, `FindSet`, `ReadIsolation`, `LockTable`, `ModifyAll`, `DeleteAll`, `Modify`, `Delete`, `Commit`, `checkpoint`, `Copy`, `RecordRef`, `GetTable`, `TextBuilder`, `Dictionary`, `temporary`, `repeat`, `until`, `CalcFields`, `SetAutoCalcFields`, `CalcSums`, `FlowField`, `Visible`, `Job Queue Entry`, `Job Queue Category Code`, `Confirm`, `RunModal`, `GuiAllowed`, `TryFunction`, `Codeunit.Run`, `HttpClient`, `Status`, `On Hold`, `stop request`, `TaskScheduler.CreateTask`, `TaskScheduler.TaskExists`). A file enters the candidate worklist when its `keywords` intersect the extracted tokens or its topic (derived from the index entry's `path`, `title`, and `description`) matches a changed object type. Read an article's full file — its `## Best Practice` / `## Anti Pattern` bodies — only after it makes the worklist; candidate selection uses the index alone. @@ -52,6 +52,12 @@ Apply these targeted cues even when simple token overlap would rank the article - Worklist `avoid-cloning-records-before-modify-delete-in-loops.md` when an iteration calls `Copy` or `RecordRef.GetTable` before `Modify`/`Delete`, or passes the iterated record without `var` to a helper that writes that record. Do not worklist it from `Modify`, `Delete`, or `RecordRef` alone; exclude a direct write on the iterator, a read-only copy, a temporary record, a different target table, and a `RecordRef` opened and iterated directly. - Worklist `use-tryfunction-for-error-catching-not-rollback.md` only when writes occur inside a try method and the code or surrounding flow expects an error to roll them back. A bare try-method call whose Boolean result is ignored belongs exclusively to `error-handling/ignored-tryfunction-return-disables-try-semantics.md`; do not worklist the performance article from that call shape alone. - For `LockTable` in a pure read helper, select exactly one owner. Use `do-not-locktable-in-read-only-procedure.md` when the helper needs no stronger isolation and should remove the lock. Use `prefer-readisolation-over-locktable-for-reads.md` instead when the code explicitly requires committed-read semantics and `ReadIsolation` is the replacement. Never emit both findings for the same call. +- Worklist `job-queue-handlers-must-not-require-ui.md` when a codeunit run by the job queue calls `Confirm`, `Page.Run`, `Page.RunModal`, `Report.Run`, `Report.RunModal`, `Hyperlink`, `File.Upload`, or `File.Download`, or uses `Message` as its only success or failure notification. Exclude optional UI-only behavior guarded by `GuiAllowed`; do not exclude a guard that silently skips a decision required by the operation. +- Worklist `job-queue-handlers-must-propagate-failures.md` when a codeunit run by the job queue handles a failed `TryFunction`, `Codeunit.Run`, or another Boolean-returning operation with `exit` or normal fall-through, causing the dispatcher to observe success. Exclude intentional partial-success handling that persists or emits an observable aggregate outcome. A bare try-method call whose Boolean result is ignored remains owned exclusively by `error-handling/ignored-tryfunction-return-disables-try-semantics.md`. +- Worklist `job-queue-external-effects-must-be-idempotent.md` when rerunnable job queue work reads an outbox row, performs a state-changing external request, then updates or deletes local data without sending a stable request ID understood by the external system. Exclude naturally idempotent operations and requests whose body, URI, headers, or business key lets the external service return the existing result instead of repeating the side effect. +- Worklist `job-queue-on-hold-does-not-stop-running-work.md` when a running job queue handler polls the entry's `Status` or `On Hold` value as a cancellation signal. Exclude application-owned stop requests that are checked before every bounded unit of work, including the first, when completed work and its checkpoint remain consistent and resume logic clears the request. +- Worklist `job-queue-category-code-serializes-conflicting-jobs.md` when two or more job queue entries in the same company are shown by the changed context to require mutual exclusion but have empty or different Job Queue Category Codes. Do not infer a conflict merely because jobs touch the same tables, and do not recommend a category to coordinate across companies, environments, or workers outside the job queue dispatcher. +- Worklist `store-scheduled-task-id-to-avoid-duplicate-tasks.md` when `TaskScheduler.CreateTask` runs from initialization, login, setup, or another repeatable path without persisting its returned GUID and checking it with `TaskScheduler.TaskExists` before creating a replacement. Exclude one-shot creation and correctly persisted check-before-create flows; concurrent callers still require serialization around that sequence. These targeted inclusions and exclusions override generic token overlap. Do not retain an excluded article solely because the diff contains one of its keywords. diff --git a/tools/Test-ReviewFixtures.ps1 b/tools/Test-ReviewFixtures.ps1 index c4b0bf1..8cf019f 100644 --- a/tools/Test-ReviewFixtures.ps1 +++ b/tools/Test-ReviewFixtures.ps1 @@ -195,12 +195,42 @@ foreach ($domain in $leafDomains) { } $override = if ($overrides.ContainsKey($domain)) { $overrides[$domain] } else { $null } - $selectedArticle = $null - if ($override -and ($override.PSObject.Properties.Name -contains 'article')) { - $articleName = [string]$override.article + $hasArticleOverride = $override -and ($override.PSObject.Properties.Name -contains 'article') + $hasArticlesOverride = $override -and ($override.PSObject.Properties.Name -contains 'articles') + if ($hasArticleOverride -and $hasArticlesOverride) { + $problems.Add("${domain}: override must specify either 'article' or 'articles', not both.") | Out-Null + continue + } + + $articleNames = @() + if ($hasArticlesOverride) { + $articleNames = @($override.articles) + if (-not $articleNames.Count) { + $problems.Add("${domain}: override 'articles' must contain at least one article.") | Out-Null + continue + } + } elseif ($hasArticleOverride) { + $articleNames = @($override.article) + } else { + $articleNames = @($articles | Select-Object -First 1 | ForEach-Object BaseName) + } + + $selectedArticles = [System.Collections.Generic.List[object]]::new() + $seenArticleNames = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::OrdinalIgnoreCase) + foreach ($articleNameValue in $articleNames) { + if ($articleNameValue -isnot [string] -or [string]::IsNullOrWhiteSpace([string]$articleNameValue)) { + $problems.Add("${domain}: override article names must be non-empty strings.") | Out-Null + continue + } + $articleName = [string]$articleNameValue if ($articleName.EndsWith('.md')) { $articleName = [System.IO.Path]::GetFileNameWithoutExtension($articleName) } + if (-not $seenArticleNames.Add($articleName)) { + $problems.Add("${domain}: override contains duplicate article: $articleName.md") | Out-Null + continue + } + $selectedArticle = $articles | Where-Object BaseName -eq $articleName | Select-Object -First 1 if (-not $selectedArticle) { $articleExists = @( @@ -218,32 +248,41 @@ foreach ($domain in $leafDomains) { } continue } - } else { - $selectedArticle = $articles | Select-Object -First 1 + $selectedArticles.Add($selectedArticle) | Out-Null } - if (-not $selectedArticle) { - $problems.Add("${domain}: no article has both .good.al and .bad.al companion samples.") | Out-Null + if (-not $selectedArticles.Count) { + if (-not $articleNames.Count) { + $problems.Add("${domain}: no article has both .good.al and .bad.al companion samples.") | Out-Null + } continue } - $articlePath = [string]$selectedArticle.ArticlePath - $sampleDirectory = (Split-Path -Parent $articlePath).Replace('\', '/') $context = if ($override -and ($override.PSObject.Properties.Name -contains 'context')) { [string]$override.context } else { $null } - foreach ($kind in 'bad', 'good') { - $case = [pscustomobject]@{ - id = "$domain-$kind" - domain = $domain - input = "$sampleDirectory/$($selectedArticle.BaseName).$kind.al" - expected = if ($kind -eq 'bad') { @($articlePath) } else { @() } + for ($articleIndex = 0; $articleIndex -lt $selectedArticles.Count; $articleIndex++) { + $selectedArticle = $selectedArticles[$articleIndex] + $articlePath = [string]$selectedArticle.ArticlePath + $sampleDirectory = (Split-Path -Parent $articlePath).Replace('\', '/') + foreach ($kind in 'bad', 'good') { + $caseId = if ($articleIndex -eq 0) { + "$domain-$kind" + } else { + "$domain-$($selectedArticle.BaseName)-$kind" + } + $case = [pscustomobject]@{ + id = $caseId + domain = $domain + input = "$sampleDirectory/$($selectedArticle.BaseName).$kind.al" + expected = if ($kind -eq 'bad') { @($articlePath) } else { @() } + } + if ($context) { + $case | Add-Member -NotePropertyName context -NotePropertyValue $context + } + $caseList.Add($case) | Out-Null } - if ($context) { - $case | Add-Member -NotePropertyName context -NotePropertyValue $context - } - $caseList.Add($case) | Out-Null } } $cases = @($caseList) From 852a6762857cdd2a13d675ea0245e24471914e95 Mon Sep 17 00:00:00 2001 From: dayland <48474707+dayland@users.noreply.github.com> Date: Tue, 15 Sep 2026 09:32:40 +0200 Subject: [PATCH 05/19] Fix Job Queue sample links (#184) Use the required READ-convention Markdown links so knowledge retrieval can associate all new samples with their articles. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: dayland Copilot-Session: 76eb42c4-2acd-4f9c-a898-f4a44f9d46f7 --- .../job-queue-category-code-serializes-conflicting-jobs.md | 4 ++-- .../job-queue-external-effects-must-be-idempotent.md | 4 ++-- .../performance/job-queue-handlers-must-not-require-ui.md | 4 ++-- .../performance/job-queue-handlers-must-propagate-failures.md | 4 ++-- .../job-queue-on-hold-does-not-stop-running-work.md | 4 ++-- .../store-scheduled-task-id-to-avoid-duplicate-tasks.md | 4 ++-- 6 files changed, 12 insertions(+), 12 deletions(-) diff --git a/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.md b/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.md index d92647e..190fb5e 100644 --- a/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.md +++ b/microsoft/knowledge/performance/job-queue-category-code-serializes-conflicting-jobs.md @@ -19,10 +19,10 @@ Different job queue entries can run at the same time. When two jobs update the s Assign the same non-empty Job Queue Category Code to job queue entries in the same company that must not overlap, regardless of which codeunit they run. Define categories around the shared resource or exclusivity requirement, not merely around object names. Leave independent jobs in different categories so they can still run concurrently. A category does not serialize work across companies or environments, or coordinate workers outside the job queue dispatcher. Protect shared external or cross-company resources with a separate application-level locking mechanism. -See sample: `job-queue-category-code-serializes-conflicting-jobs.good.al`. +See sample: [`job-queue-category-code-serializes-conflicting-jobs.good.al`](job-queue-category-code-serializes-conflicting-jobs.good.al). ## Anti Pattern Creating or configuring multiple job queue entries that update the same exclusive resource while leaving their Job Queue Category Code empty or different. Do not flag jobs merely because they touch the same tables; the rule applies when their operation requires mutual exclusion. -See sample: `job-queue-category-code-serializes-conflicting-jobs.bad.al`. \ No newline at end of file +See sample: [`job-queue-category-code-serializes-conflicting-jobs.bad.al`](job-queue-category-code-serializes-conflicting-jobs.bad.al). \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.md b/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.md index a5932df..66bb11b 100644 --- a/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.md +++ b/microsoft/knowledge/performance/job-queue-external-effects-must-be-idempotent.md @@ -21,10 +21,10 @@ Use a stable request ID that exists before the job queue processes the outbox ro A `Processed` flag set after the external call does not solve this failure window. If a later AL error rolls back that flag, the outbox row again looks unprocessed even though the external operation already happened. -See sample: `job-queue-external-effects-must-be-idempotent.good.al`. +See sample: [`job-queue-external-effects-must-be-idempotent.good.al`](job-queue-external-effects-must-be-idempotent.good.al). ## Anti Pattern Sending a state-changing request from a job queue handler with no stable request ID understood by the external API. Specifically, look for this sequence: read an outbox row, call `HttpClient.Post` or another side-effecting API, update or delete local data, and propagate an error after which the same outbox row can be processed again. The key may be part of the request body, URI, headers, or an existing business key; a naturally idempotent remote operation is already safe and should not be flagged. -See sample: `job-queue-external-effects-must-be-idempotent.bad.al`. \ No newline at end of file +See sample: [`job-queue-external-effects-must-be-idempotent.bad.al`](job-queue-external-effects-must-be-idempotent.bad.al). \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.md b/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.md index f780ab5..5987270 100644 --- a/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.md +++ b/microsoft/knowledge/performance/job-queue-handlers-must-not-require-ui.md @@ -19,10 +19,10 @@ A job queue handler runs in a background session with no client UI. Calls that r Make a dedicated job queue entry point non-interactive. Validate parameters and data in AL, persist business-visible status when needed, and let failures propagate to the job queue log. If one procedure genuinely serves both foreground and background callers, isolate optional UI-only behavior behind `GuiAllowed`; do not use the guard to silently skip a decision that the operation requires. -See sample: `job-queue-handlers-must-not-require-ui.good.al`. +See sample: [`job-queue-handlers-must-not-require-ui.good.al`](job-queue-handlers-must-not-require-ui.good.al). ## Anti Pattern Calling `Confirm`, `Page.Run`, `Page.RunModal`, `Report.Run`, `Report.RunModal`, `Hyperlink`, `File.Upload`, or `File.Download` from a codeunit run by the job queue. Another signal is using `Message` as the only success or failure notification: no user is attached to receive it. -See sample: `job-queue-handlers-must-not-require-ui.bad.al`. \ No newline at end of file +See sample: [`job-queue-handlers-must-not-require-ui.bad.al`](job-queue-handlers-must-not-require-ui.bad.al). \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.md b/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.md index 1c7b83f..361fc7d 100644 --- a/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.md +++ b/microsoft/knowledge/performance/job-queue-handlers-must-propagate-failures.md @@ -19,10 +19,10 @@ The job queue dispatcher can mark an entry as failed, record the error, and appl Let an error that invalidates the whole run propagate out of the job queue entry point. Add context only when it helps an operator diagnose the failure and does not expose sensitive data. Per-item failures may be collected deliberately, but the batch must persist or emit an observable aggregate outcome instead of silently treating incomplete work as success. -See sample: `job-queue-handlers-must-propagate-failures.good.al`. +See sample: [`job-queue-handlers-must-propagate-failures.good.al`](job-queue-handlers-must-propagate-failures.good.al). ## Anti Pattern Calling a `TryFunction`, `Codeunit.Run`, or another Boolean-returning operation from a job queue handler and using `exit` or normal fall-through on failure without recording an intentional partial-success outcome. The dispatcher sees a successful return, so the entry's status and log do not represent the failed work and configured retries are not applied. -See sample: `job-queue-handlers-must-propagate-failures.bad.al`. \ No newline at end of file +See sample: [`job-queue-handlers-must-propagate-failures.bad.al`](job-queue-handlers-must-propagate-failures.bad.al). \ No newline at end of file diff --git a/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.md b/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.md index 6eec05c..b0411a8 100644 --- a/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.md +++ b/microsoft/knowledge/performance/job-queue-on-hold-does-not-stop-running-work.md @@ -19,10 +19,10 @@ The On Hold status prevents a job queue entry from starting again, but it does n Use On Hold to pause future scheduling. When a long-running operation must support graceful cancellation, store a separate application-owned stop request and check it before every bounded unit of work, including the first. Exit only at a point where completed work and the checkpoint are consistent. The code that resumes scheduling must clear the stop request before restarting the job. Use administrative session termination only when graceful cancellation is impossible. -See sample: `job-queue-on-hold-does-not-stop-running-work.good.al`. +See sample: [`job-queue-on-hold-does-not-stop-running-work.good.al`](job-queue-on-hold-does-not-stop-running-work.good.al). ## Anti Pattern Polling the job queue entry's Status field from inside its handler and expecting a change to On Hold to cancel the active run. The status controls scheduling, not cooperative cancellation, so the handler can continue processing despite the operator's action. -See sample: `job-queue-on-hold-does-not-stop-running-work.bad.al`. \ No newline at end of file +See sample: [`job-queue-on-hold-does-not-stop-running-work.bad.al`](job-queue-on-hold-does-not-stop-running-work.bad.al). \ No newline at end of file diff --git a/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.md b/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.md index 600c671..1e64c32 100644 --- a/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.md +++ b/microsoft/knowledge/performance/store-scheduled-task-id-to-avoid-duplicate-tasks.md @@ -19,10 +19,10 @@ Every call to `TaskScheduler.CreateTask` creates a new scheduled task and return Persist the GUID returned by `CreateTask` at the same scope as the logical task. Before creating a replacement, parse the stored GUID and call `TaskScheduler.TaskExists`; create and store a new task only when the previous task no longer exists. `TaskExists` checks one GUID, not whether an equivalent codeunit is already scheduled, so callers that can schedule concurrently still need serialization around this check-and-create sequence. -See sample: `store-scheduled-task-id-to-avoid-duplicate-tasks.good.al`. +See sample: [`store-scheduled-task-id-to-avoid-duplicate-tasks.good.al`](store-scheduled-task-id-to-avoid-duplicate-tasks.good.al). ## Anti Pattern Calling `TaskScheduler.CreateTask` every time initialization, login, setup, or another repeatable path runs while ignoring its return value. Each invocation creates another independent task even when an equivalent task is already pending. -See sample: `store-scheduled-task-id-to-avoid-duplicate-tasks.bad.al`. \ No newline at end of file +See sample: [`store-scheduled-task-id-to-avoid-duplicate-tasks.bad.al`](store-scheduled-task-id-to-avoid-duplicate-tasks.bad.al). \ No newline at end of file From b74967bc5b7a454eae19d6a1250199afd869f064 Mon Sep 17 00:00:00 2001 From: dayland <48474707+dayland@users.noreply.github.com> Date: Tue, 15 Sep 2026 10:27:40 +0200 Subject: [PATCH 06/19] Add machine-readable review contracts (#182) Generate a deterministic action-skill index from frontmatter, publish structural schemas for orchestration and findings, and validate flat review composition in CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: dayland Copilot-Session: 76eb42c4-2acd-4f9c-a898-f4a44f9d46f7 --- .github/scripts/Test-SkillIndex.ps1 | 240 +++++++++++++++++++++ .github/scripts/validate_frontmatter.py | 41 +++- .github/workflows/skill-index.yml | 18 ++ microsoft/skills/review/al-code-review.md | 5 + schemas/findings-report.schema.json | 159 ++++++++++++++ schemas/skill-index.schema.json | 84 ++++++++ skills/do.md | 12 ++ tools/Build-SkillIndex.ps1 | 247 ++++++++++++++++++++++ 8 files changed, 803 insertions(+), 3 deletions(-) create mode 100644 .github/scripts/Test-SkillIndex.ps1 create mode 100644 .github/workflows/skill-index.yml create mode 100644 schemas/findings-report.schema.json create mode 100644 schemas/skill-index.schema.json create mode 100644 tools/Build-SkillIndex.ps1 diff --git a/.github/scripts/Test-SkillIndex.ps1 b/.github/scripts/Test-SkillIndex.ps1 new file mode 100644 index 0000000..839042f --- /dev/null +++ b/.github/scripts/Test-SkillIndex.ps1 @@ -0,0 +1,240 @@ +<# +.SYNOPSIS + Validates the BCQuality action-skill index generator and shared schemas. +#> +[CmdletBinding()] +param( + [string] $Root = (Resolve-Path (Join-Path -Path $PSScriptRoot -ChildPath '..' '..')) +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +$Root = (Resolve-Path -LiteralPath $Root).Path + +function Assert-ThrowsLike { + param( + [scriptblock] $Action, + [string] $Pattern + ) + + try { + & $Action + } + catch { + if ($_.Exception.Message -like $Pattern) { + return + } + throw "Expected error like '$Pattern', received: $($_.Exception.Message)" + } + throw "Expected error like '$Pattern', but no error was thrown." +} + +$generator = Join-Path $Root 'tools/Build-SkillIndex.ps1' +$indexSchema = Join-Path $Root 'schemas/skill-index.schema.json' +$reportSchema = Join-Path $Root 'schemas/findings-report.schema.json' +foreach ($path in $generator, $indexSchema, $reportSchema) { + if (-not (Test-Path -LiteralPath $path -PathType Leaf)) { + throw "Required contract file not found: $path" + } +} + +$tmp = Join-Path ([IO.Path]::GetTempPath()) ("skillindex_" + [guid]::NewGuid().ToString('N')) +New-Item -ItemType Directory -Path $tmp -Force | Out-Null +try { + $first = Join-Path $tmp 'first.json' + $second = Join-Path $tmp 'second.json' + & $generator -BCQualityRoot $Root -IndexPath $first | Out-Null + & $generator -BCQualityRoot $Root -IndexPath $second | Out-Null + + $normalize = { + param([string] $Path) + return ((Get-Content -LiteralPath $Path -Raw) -replace '"generatedAt":"[^"]*"', '"generatedAt":""') + } + if ((& $normalize $first) -ne (& $normalize $second)) { + throw 'Skill index is not deterministic beyond generatedAt.' + } + + $raw = Get-Content -LiteralPath $first -Raw + if (-not ($raw | Test-Json -SchemaFile $indexSchema -ErrorAction Stop)) { + throw 'Generated skill index does not satisfy schemas/skill-index.schema.json.' + } + + $index = $raw | ConvertFrom-Json + $skills = @($index.skills) + if ($index.skillCount -ne $skills.Count) { + throw "skillCount is $($index.skillCount), but the index contains $($skills.Count) records." + } + + $paths = @($skills.path) + $duplicates = @($paths | Group-Object | Where-Object Count -gt 1) + if ($duplicates.Count) { + throw "Duplicate skill paths: $($duplicates.Name -join ', ')" + } + foreach ($path in $paths) { + if (-not (Test-Path -LiteralPath (Join-Path $Root $path) -PathType Leaf)) { + throw "Indexed skill does not exist: $path" + } + } + + $expectedLeaves = @( + 'microsoft/skills/review/al-performance-review.md', + 'microsoft/skills/review/al-security-review.md', + 'microsoft/skills/review/al-privacy-review.md', + 'microsoft/skills/review/al-upgrade-review.md', + 'microsoft/skills/review/al-style-review.md', + 'microsoft/skills/review/al-ui-review.md', + 'microsoft/skills/review/al-error-handling-review.md', + 'microsoft/skills/review/al-events-review.md', + 'microsoft/skills/review/al-interfaces-review.md', + 'microsoft/skills/review/al-breaking-changes-review.md', + 'microsoft/skills/review/al-web-services-review.md', + 'microsoft/skills/review/al-testing-review.md', + 'microsoft/skills/review/al-data-modeling-review.md', + 'microsoft/skills/review/al-query-review.md', + 'microsoft/skills/review/al-appsource-review.md', + 'microsoft/skills/review/al-telemetry-review.md' + ) + $review = @($skills | Where-Object id -eq 'al-code-review') + if ($review.Count -ne 1) { + throw "Expected exactly one al-code-review record, found $($review.Count)." + } + if ((@($review[0].subSkills) -join "`n") -cne ($expectedLeaves -join "`n")) { + throw 'al-code-review subSkills did not preserve the declared 16-leaf order.' + } + foreach ($leafPath in $expectedLeaves) { + $leaf = @($skills | Where-Object path -ceq $leafPath) + if ($leaf.Count -ne 1 -or @($leaf[0].subSkills).Count -ne 0) { + throw "Expected '$leafPath' to resolve to exactly one leaf action skill." + } + } + + $minimalReport = @{ + skill = @{ id = 'al-style-review'; version = 1 } + outcome = 'completed' + summary = @{ + counts = @{ blocker = 0; major = 0; minor = 0; info = 0 } + coverage = @{ 'worklist-size' = 0; 'items-evaluated' = 0 } + } + findings = @() + suppressed = @() + } | ConvertTo-Json -Depth 8 + if (-not ($minimalReport | Test-Json -SchemaFile $reportSchema -ErrorAction Stop)) { + throw 'Minimal findings report does not satisfy schemas/findings-report.schema.json.' + } + + $reviewSkillText = Get-Content -LiteralPath ( + Join-Path -Path $Root -ChildPath 'microsoft/skills/review/al-code-review.md' + ) -Raw + $reportExamples = [regex]::Matches($reviewSkillText, '(?s)```json\s*(\{.*?\})\s*```') + if ($reportExamples.Count -ne 2) { + throw "Expected two al-code-review JSON examples, found $($reportExamples.Count)." + } + foreach ($example in $reportExamples) { + if (-not ($example.Groups[1].Value | Test-Json -SchemaFile $reportSchema -ErrorAction Stop)) { + throw 'An al-code-review output example does not satisfy schemas/findings-report.schema.json.' + } + } + + $fixtureRoot = Join-Path -Path $tmp -ChildPath 'fixture' + $fixtureSkills = Join-Path -Path $fixtureRoot -ChildPath 'microsoft/skills/review' + New-Item -ItemType Directory -Path $fixtureSkills -Force | Out-Null + $leaf = @' +--- +kind: action-skill +id: al-leaf-review +version: 1 +title: Leaf +description: Test leaf. +inputs: [file-path] +outputs: [findings-report] +--- + +# Leaf + +## Source +Source. +## Relevance +Relevance. +## Worklist +Worklist. +## Action +Action. +## Output +Output. +'@ + Set-Content -LiteralPath (Join-Path $fixtureSkills 'al-leaf-review.md') -Value $leaf -Encoding utf8NoBOM + + $duplicateSuper = @' +--- +kind: action-skill +id: al-code-review +version: 1 +title: Review +description: Test super-skill. +inputs: [file-path] +outputs: [findings-report] +sub-skills: + - microsoft/skills/review/al-leaf-review.md + - microsoft/skills/review/al-leaf-review.md +--- + +# Review + +## Source +Source. +## Relevance +Relevance. +## Worklist +Worklist. +## Action +Action. +## Output +Output. +'@ + $superPath = Join-Path $fixtureSkills 'al-code-review.md' + Set-Content -LiteralPath $superPath -Value $duplicateSuper -Encoding utf8NoBOM + Assert-ThrowsLike -Pattern '*duplicate sub-skill*' -Action { + & $generator -BCQualityRoot $fixtureRoot -IndexPath (Join-Path $tmp 'invalid.json') + } + + $nestedLeaf = $leaf.Replace('id: al-leaf-review', 'id: al-nested-review').Replace( + 'outputs: [findings-report]', + "outputs: [findings-report]`nsub-skills:`n - microsoft/skills/review/al-leaf-review.md" + ) + Set-Content -LiteralPath (Join-Path $fixtureSkills 'al-nested-review.md') -Value $nestedLeaf -Encoding utf8NoBOM + $nestedSuper = @' +--- +kind: action-skill +id: al-code-review +version: 1 +title: Review +description: Test super-skill. +inputs: [file-path] +outputs: [findings-report] +sub-skills: + - microsoft/skills/review/al-nested-review.md +--- + +# Review + +## Source +Source. +## Relevance +Relevance. +## Worklist +Worklist. +## Action +Action. +## Output +Output. +'@ + Set-Content -LiteralPath $superPath -Value $nestedSuper -Encoding utf8NoBOM + Assert-ThrowsLike -Pattern '*Nested super-skills are not supported*' -Action { + & $generator -BCQualityRoot $fixtureRoot -IndexPath (Join-Path $tmp 'nested.json') + } +} +finally { + Remove-Item -LiteralPath $tmp -Recurse -Force -ErrorAction SilentlyContinue +} + +Write-Output 'Skill-index check PASSED: deterministic, schema-valid, and all 16 review leaves preserved in order.' diff --git a/.github/scripts/validate_frontmatter.py b/.github/scripts/validate_frontmatter.py index f422d53..20f33d6 100644 --- a/.github/scripts/validate_frontmatter.py +++ b/.github/scripts/validate_frontmatter.py @@ -381,6 +381,20 @@ def validate_action_skill(path: Path, parsed: Parsed, report: Report) -> None: bad = [x for x in ss if not x.endswith(".md")] if bad: report.error(path, "R20", f"sub-skills entries must end in '.md': {bad}", 1) + non_canonical = [ + x for x in ss + if "\\" in x or x.startswith("/") or ".." in Path(x).parts or x.startswith("./") + ] + if non_canonical: + report.error( + path, + "R20", + f"sub-skills entries must be canonical repo-relative paths: {non_canonical}", + 1, + ) + duplicates = sorted({x for x in ss if ss.count(x) > 1}) + if duplicates: + report.error(path, "R20", f"sub-skills contains duplicate paths: {duplicates}", 1) # R21 five required sections, in order, each exactly once heads = [h for h, _ in headings_in_order(parsed.body)] @@ -565,7 +579,13 @@ class SkillRecord: skill_id: str | None -def validate_sub_skills_registry(path: Path, fm: dict[str, Any], root: Path, report: Report) -> None: +def validate_sub_skills_registry( + path: Path, + fm: dict[str, Any], + root: Path, + action_skills_by_path: dict[str, dict[str, Any]], + report: Report, +) -> None: """R26: a super-skill's declared `sub-skills` must exactly match the `al-*-review.md` leaf files present in the same directory (set equality, ordering-agnostic). This keeps the registered leaf list the single source @@ -598,6 +618,17 @@ def validate_sub_skills_registry(path: Path, fm: dict[str, Any], root: Path, rep f"sub-skills entry is not a sibling 'al-*-review.md' leaf: {entry}", 1, ) + for entry in ss: + leaf = action_skills_by_path.get(entry) + if leaf is None: + if (root / entry).exists(): + report.error(path, "R26", f"sub-skills entry is not an action skill: {entry}", 1) + continue + if is_non_empty_list_of_str(leaf.get("sub-skills")): + report.error(path, "R26", f"nested super-skill is not permitted in v1 composition: {entry}", 1) + if leaf.get("outputs") != ["findings-report"]: + report.error(path, "R26", f"sub-skill must produce findings-report: {entry}", 1) + # Sibling leaves on disk that were never registered ('forgot to wire it up'). for leaf in sorted(leaves - declared): report.error(path, "R26", f"leaf not registered in sub-skills: {leaf}", 1) @@ -670,9 +701,13 @@ def run(root: Path) -> Report: others = [q.relative_to(root).as_posix() for q in paths if q != p] report.error(p, "R24", f"skill id '{sid}' ({kind}) is not unique; also defined in: {others}") - # Fourth pass: R26 sub-skills registry matches leaf files on disk + # Fourth pass: R26 sub-skills registry matches compatible leaf files on disk + action_skills_by_path = { + path.relative_to(root).as_posix(): fm + for path, fm in action_skill_fms + } for path, fm in action_skill_fms: - validate_sub_skills_registry(path, fm, root, report) + validate_sub_skills_registry(path, fm, root, action_skills_by_path, report) return report diff --git a/.github/workflows/skill-index.yml b/.github/workflows/skill-index.yml new file mode 100644 index 0000000..b1b8e3a --- /dev/null +++ b/.github/workflows/skill-index.yml @@ -0,0 +1,18 @@ +name: Validate skill index and report schemas + +on: + pull_request: + branches: [main] + push: + branches: [main] + +jobs: + validate-contract: + runs-on: ubuntu-latest + steps: + - name: Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - name: Validate skill-index generator and schemas + shell: pwsh + run: ./.github/scripts/Test-SkillIndex.ps1 -Root . diff --git a/microsoft/skills/review/al-code-review.md b/microsoft/skills/review/al-code-review.md index 9239220..9b5f739 100644 --- a/microsoft/skills/review/al-code-review.md +++ b/microsoft/skills/review/al-code-review.md @@ -41,6 +41,11 @@ An orchestrator invokes this skill with a `pr-diff`, `file-path`, or `folder-pat The sub-skills invoked by this skill are those listed in frontmatter `sub-skills`. Additional leaf skills are added by updating the `sub-skills` list. The skill does not discover sub-skills implicitly. +Hosts that orchestrate leaves mechanically SHOULD run +`tools/Build-SkillIndex.ps1` and resolve this skill by `id: al-code-review`. +The generated `subSkills` array preserves the frontmatter order and avoids +host-specific Markdown parsing. + ## Relevance A sub-skill is relevant when both of the following hold: diff --git a/schemas/findings-report.schema.json b/schemas/findings-report.schema.json new file mode 100644 index 0000000..76712f4 --- /dev/null +++ b/schemas/findings-report.schema.json @@ -0,0 +1,159 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "https://github.com/microsoft/BCQuality/schemas/findings-report.schema.json", + "title": "BCQuality findings report", + "type": "object", + "additionalProperties": false, + "required": ["skill", "outcome", "summary", "findings", "suppressed"], + "properties": { + "skill": { "$ref": "#/definitions/skillReference" }, + "outcome": { + "enum": ["completed", "not-applicable", "no-knowledge", "partial", "failed"] + }, + "outcome-reason": { "type": "string", "minLength": 1 }, + "summary": { "$ref": "#/definitions/summary" }, + "findings": { + "type": "array", + "items": { "$ref": "#/definitions/finding" } + }, + "suppressed": { + "type": "array", + "items": { "$ref": "#/definitions/suppressed" } + }, + "sub-results": { + "type": "array", + "items": { "$ref": "#" } + }, + "skipped-sub-skills": { + "type": "array", + "items": { "$ref": "#/definitions/skippedSubSkill" } + } + }, + "allOf": [ + { + "if": { + "properties": { + "outcome": { "enum": ["partial", "failed"] } + } + }, + "then": { "required": ["outcome-reason"] } + }, + { + "if": { + "properties": { + "outcome": { "enum": ["not-applicable", "no-knowledge", "failed"] } + } + }, + "then": { + "properties": { + "findings": { "maxItems": 0 } + } + } + } + ], + "definitions": { + "skillReference": { + "type": "object", + "additionalProperties": false, + "required": ["id", "version"], + "properties": { + "id": { "type": "string", "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$" }, + "version": { "type": "integer", "minimum": 1 } + } + }, + "counts": { + "type": "object", + "additionalProperties": false, + "required": ["blocker", "major", "minor", "info"], + "properties": { + "blocker": { "type": "integer", "minimum": 0 }, + "major": { "type": "integer", "minimum": 0 }, + "minor": { "type": "integer", "minimum": 0 }, + "info": { "type": "integer", "minimum": 0 } + } + }, + "coverage": { + "type": "object", + "additionalProperties": false, + "required": ["worklist-size", "items-evaluated"], + "properties": { + "worklist-size": { "type": "integer", "minimum": 0 }, + "items-evaluated": { "type": "integer", "minimum": 0 } + } + }, + "summary": { + "type": "object", + "additionalProperties": false, + "required": ["counts", "coverage"], + "properties": { + "counts": { "$ref": "#/definitions/counts" }, + "coverage": { "$ref": "#/definitions/coverage" } + } + }, + "reference": { + "type": "object", + "additionalProperties": false, + "required": ["path"], + "properties": { + "path": { "type": "string", "minLength": 1, "pattern": "^[^\\\\]+$" }, + "sha": { "type": "string", "pattern": "^[a-fA-F0-9]{40}$" } + } + }, + "location": { + "type": "object", + "additionalProperties": false, + "required": ["file", "line"], + "properties": { + "file": { "type": "string", "minLength": 1, "pattern": "^[^\\\\]+$" }, + "line": { "type": "integer", "minimum": 1 }, + "range": { + "type": "object", + "additionalProperties": false, + "required": ["start-line", "end-line"], + "properties": { + "start-line": { "type": "integer", "minimum": 1 }, + "end-line": { "type": "integer", "minimum": 1 } + } + } + } + }, + "finding": { + "type": "object", + "additionalProperties": false, + "required": ["id", "severity", "message", "references", "confidence"], + "properties": { + "id": { "type": "string", "minLength": 1 }, + "severity": { "enum": ["blocker", "major", "minor", "info"] }, + "message": { "type": "string", "minLength": 1 }, + "location": { "$ref": "#/definitions/location" }, + "references": { + "type": "array", + "items": { "$ref": "#/definitions/reference" } + }, + "confidence": { "enum": ["high", "medium", "low"] }, + "from-sub-skill": { "type": "string", "minLength": 1 }, + "domain": { "type": "string", "minLength": 1, "pattern": "^[^\\r\\n]+$" }, + "suggested-code": { "type": "string", "minLength": 1 }, + "suggested-code-omission-reason": { "type": "string", "minLength": 1 } + } + }, + "suppressed": { + "type": "object", + "additionalProperties": false, + "required": ["reference", "reason"], + "properties": { + "reference": { "$ref": "#/definitions/reference" }, + "reason": { "enum": ["layer-precedence", "configuration"] } + } + }, + "skippedSubSkill": { + "type": "object", + "additionalProperties": false, + "required": ["skill", "reason"], + "properties": { + "skill": { "$ref": "#/definitions/skillReference" }, + "reason": { "enum": ["configuration", "not-applicable"] } + } + } + } +} diff --git a/schemas/skill-index.schema.json b/schemas/skill-index.schema.json new file mode 100644 index 0000000..01944bb --- /dev/null +++ b/schemas/skill-index.schema.json @@ -0,0 +1,84 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "https://github.com/microsoft/BCQuality/schemas/skill-index.schema.json", + "title": "BCQuality action-skill index", + "type": "object", + "additionalProperties": false, + "required": ["version", "generatedAt", "skillCount", "sourceSnapshot", "skills"], + "properties": { + "version": { "const": 1 }, + "generatedAt": { "type": "string", "format": "date-time" }, + "skillCount": { "type": "integer", "minimum": 0 }, + "sourceSnapshot": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "skills": { + "type": "array", + "items": { "$ref": "#/definitions/skill" } + } + }, + "definitions": { + "stringArray": { + "type": "array", + "items": { "type": "string", "minLength": 1 } + }, + "skill": { + "type": "object", + "additionalProperties": false, + "required": [ + "path", + "layer", + "id", + "version", + "title", + "description", + "inputs", + "outputs", + "filters", + "subSkills", + "sourceSha256" + ], + "properties": { + "path": { "type": "string", "pattern": "^(microsoft|community|custom)/skills/.+\\.md$" }, + "layer": { "enum": ["microsoft", "community", "custom"] }, + "id": { "type": "string", "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$" }, + "version": { "type": "integer", "minimum": 1 }, + "title": { "type": "string", "minLength": 1 }, + "description": { "type": "string", "minLength": 1 }, + "inputs": { "$ref": "#/definitions/stringArray" }, + "outputs": { + "type": "array", + "minItems": 1, + "maxItems": 1, + "items": { "const": "findings-report" } + }, + "filters": { + "type": "object", + "additionalProperties": false, + "required": ["bc-version", "technologies", "countries", "application-area"], + "properties": { + "bc-version": { + "type": "array", + "items": { + "oneOf": [ + { "type": "integer", "minimum": 1 }, + { "type": "string", "pattern": "^(all|[1-9][0-9]*\\.\\.[1-9][0-9]*|[1-9][0-9]*\\.\\.)$" } + ] + } + }, + "technologies": { "$ref": "#/definitions/stringArray" }, + "countries": { "$ref": "#/definitions/stringArray" }, + "application-area": { "$ref": "#/definitions/stringArray" } + } + }, + "subSkills": { + "type": "array", + "uniqueItems": true, + "items": { + "type": "string", + "pattern": "^(microsoft|community|custom)/skills/.+\\.md$" + } + }, + "sourceSha256": { "type": "string", "pattern": "^[a-f0-9]{64}$" } + } + } + } +} diff --git a/skills/do.md b/skills/do.md index 9abdf58..e67faf6 100644 --- a/skills/do.md +++ b/skills/do.md @@ -106,6 +106,12 @@ Every action skill MUST contain these five sections, in order: Every action skill emits a single JSON document that conforms to this schema: +The machine-readable structural schema is +[`schemas/findings-report.schema.json`](../schemas/findings-report.schema.json). +The rules below remain authoritative for semantic checks that JSON Schema +cannot perform by itself, including summary arithmetic, reference existence, +source-scope locations, and article-body retrieval. + ```json { "skill": { "id": "string", "version": 1 }, @@ -349,6 +355,12 @@ the declared worklist order, and wait for every invocation to finish before performing any super-skill self-review or final rollup. Scheduling MUST NOT change relevance, coverage, failure, reference-integrity, or output semantics. +Orchestrators SHOULD generate `skill-index.json` with +`tools/Build-SkillIndex.ps1` and consume the super-skill's ordered `subSkills` +from that index instead of parsing Markdown. Action-skill frontmatter remains +the source of truth; the generated index conforms to +`schemas/skill-index.schema.json`. + ### Section interpretation for super-skills The five required sections still apply. Their meaning shifts from knowledge files to sub-skills: diff --git a/tools/Build-SkillIndex.ps1 b/tools/Build-SkillIndex.ps1 new file mode 100644 index 0000000..022898e --- /dev/null +++ b/tools/Build-SkillIndex.ps1 @@ -0,0 +1,247 @@ +<# +.SYNOPSIS + Builds the machine-readable BCQuality action-skill index. + +.DESCRIPTION + Action-skill frontmatter remains the source of truth. This script emits the + versioned JSON contract orchestrators consume so they do not need to parse + Markdown or duplicate composition rules. + +.PARAMETER BCQualityRoot + BCQuality repository or filtered content root. + +.PARAMETER IndexPath + Output path. Defaults to /skill-index.json. + +.OUTPUTS + Returns the number of indexed action skills. +#> +[CmdletBinding()] +param( + [string] $BCQualityRoot, + [string] $IndexPath +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' + +if (-not $BCQualityRoot) { + $BCQualityRoot = (Resolve-Path (Join-Path $PSScriptRoot '..')).Path +} +if (-not (Test-Path -LiteralPath $BCQualityRoot -PathType Container)) { + throw "BCQuality root not found: $BCQualityRoot" +} +$BCQualityRoot = (Resolve-Path -LiteralPath $BCQualityRoot).Path +if (-not $IndexPath) { + $IndexPath = Join-Path $BCQualityRoot 'skill-index.json' +} + +function Get-RelativePath { + param([string] $Root, [string] $Full) + + return ($Full.Substring($Root.Length).TrimStart([char]'/', [char]'\') -replace '\\', '/') +} + +function Get-Sha256 { + param([byte[]] $Bytes) + + $sha = [Security.Cryptography.SHA256]::Create() + try { + return ([BitConverter]::ToString($sha.ComputeHash($Bytes)) -replace '-', '').ToLowerInvariant() + } + finally { + $sha.Dispose() + } +} + +function Get-ValueSha256 { + param([Parameter(Mandatory)] $Value) + + return Get-Sha256 -Bytes ([Text.Encoding]::UTF8.GetBytes( + (ConvertTo-Json -InputObject $Value -Depth 12 -Compress) + )) +} + +function ConvertFrom-SkillFrontmatter { + param( + [string] $Path, + [string] $Text + ) + + $lines = [regex]::Split($Text.TrimStart([char]0xfeff), '\r\n|\n|\r') + if ($lines.Count -lt 3 -or $lines[0].Trim() -ne '---') { + throw [IO.InvalidDataException]::new("Missing frontmatter in '$Path'.") + } + + $end = -1 + for ($i = 1; $i -lt $lines.Count; $i++) { + if ($lines[$i].Trim() -eq '---') { + $end = $i + break + } + } + if ($end -lt 0) { + throw [IO.InvalidDataException]::new("Unterminated frontmatter in '$Path'.") + } + + $frontmatter = [ordered]@{} + for ($i = 1; $i -lt $end; $i++) { + $line = $lines[$i] + if ($line -notmatch '^([a-zA-Z][\w-]*)\s*:\s*(.*)$') { + continue + } + + $key = $Matches[1] + $value = $Matches[2].Trim() + if ($value -eq '') { + $items = [System.Collections.Generic.List[string]]::new() + while ($i + 1 -lt $end -and $lines[$i + 1] -match '^\s+-\s+(.+?)\s*$') { + $i++ + $items.Add($Matches[1].Trim().Trim('"', "'")) | Out-Null + } + $frontmatter[$key] = @($items) + continue + } + + if ($value -match '^\[(.*)\]$') { + $inner = $Matches[1].Trim() + $values = [System.Collections.Generic.List[object]]::new() + if ($inner) { + foreach ($item in $inner -split '\s*,\s*') { + $normalized = $item.Trim().Trim('"', "'") + $number = 0 + if ($key -eq 'bc-version' -and [int]::TryParse($normalized, [ref]$number)) { + $values.Add($number) | Out-Null + } + else { + $values.Add($normalized) | Out-Null + } + } + } + $frontmatter[$key] = [object[]]@($values) + continue + } + + $frontmatter[$key] = $value.Trim('"', "'") + } + + return $frontmatter +} + +$records = [System.Collections.Generic.List[object]]::new() +$recordsByPath = [Collections.Generic.Dictionary[string, object]]::new([StringComparer]::Ordinal) +$sourceManifest = [System.Collections.Generic.List[object]]::new() + +foreach ($layer in 'microsoft', 'community', 'custom') { + $skillsRoot = Join-Path $BCQualityRoot (Join-Path $layer 'skills') + if (-not (Test-Path -LiteralPath $skillsRoot -PathType Container)) { + continue + } + + foreach ($file in Get-ChildItem -LiteralPath $skillsRoot -Recurse -File -Filter '*.md' | Sort-Object FullName) { + $bytes = [IO.File]::ReadAllBytes($file.FullName) + try { + $text = [Text.UTF8Encoding]::new($false, $true).GetString($bytes) + } + catch [Text.DecoderFallbackException] { + throw [IO.InvalidDataException]::new("Invalid UTF-8 in '$($file.FullName)'.", $_.Exception) + } + + $frontmatter = ConvertFrom-SkillFrontmatter -Path $file.FullName -Text $text + if ($frontmatter['kind'] -ne 'action-skill') { + continue + } + + foreach ($required in 'id', 'version', 'title', 'description', 'inputs', 'outputs') { + if (-not $frontmatter.Contains($required) -or $null -eq $frontmatter[$required] -or + ([string]$frontmatter[$required]).Trim() -eq '') { + throw [IO.InvalidDataException]::new( + "Action skill '$($file.FullName)' is missing required frontmatter '$required'." + ) + } + } + + $path = Get-RelativePath -Root $BCQualityRoot -Full $file.FullName + $sourceSha256 = Get-Sha256 -Bytes $bytes + $version = 0 + if (-not [int]::TryParse([string]$frontmatter['version'], [ref]$version) -or $version -le 0) { + throw [IO.InvalidDataException]::new("Action skill '$path' has an invalid version.") + } + + $subSkills = @() + if ($frontmatter.Contains('sub-skills')) { + $subSkills = @($frontmatter['sub-skills']) + if (-not $subSkills.Count) { + throw [IO.InvalidDataException]::new("Super-skill '$path' has an empty sub-skills list.") + } + } + + $record = [pscustomobject][ordered]@{ + path = $path + layer = $layer + id = [string]$frontmatter['id'] + version = $version + title = [string]$frontmatter['title'] + description = [string]$frontmatter['description'] + inputs = [string[]]@($frontmatter['inputs']) + outputs = [string[]]@($frontmatter['outputs']) + filters = [ordered]@{ + 'bc-version' = [object[]]$(if ($frontmatter.Contains('bc-version')) { $frontmatter['bc-version'] }) + technologies = [string[]]$(if ($frontmatter.Contains('technologies')) { $frontmatter['technologies'] }) + countries = [string[]]$(if ($frontmatter.Contains('countries')) { $frontmatter['countries'] }) + 'application-area' = [string[]]$(if ($frontmatter.Contains('application-area')) { $frontmatter['application-area'] }) + } + subSkills = [string[]]$subSkills + sourceSha256 = $sourceSha256 + } + + if (-not $recordsByPath.TryAdd($path, $record)) { + throw "Duplicate action-skill path: $path" + } + $records.Add($record) | Out-Null + $sourceManifest.Add([ordered]@{ path = $path; sha256 = $sourceSha256 }) | Out-Null + } +} + +$ids = @($records | Group-Object id | Where-Object Count -gt 1) +if ($ids.Count) { + throw "Duplicate action-skill IDs: $($ids.Name -join ', ')" +} + +foreach ($record in $records) { + $seen = [Collections.Generic.HashSet[string]]::new([StringComparer]::Ordinal) + foreach ($subSkillPath in @($record.subSkills)) { + if (-not $seen.Add($subSkillPath)) { + throw "Super-skill '$($record.path)' declares duplicate sub-skill '$subSkillPath'." + } + if (-not $recordsByPath.ContainsKey($subSkillPath)) { + throw "Super-skill '$($record.path)' references missing action skill '$subSkillPath'." + } + + $leaf = $recordsByPath[$subSkillPath] + if (@($leaf.subSkills).Count) { + throw "Nested super-skills are not supported: '$($record.path)' references '$subSkillPath'." + } + if (@($leaf.outputs).Count -ne 1 -or $leaf.outputs[0] -ne 'findings-report') { + throw "Sub-skill '$subSkillPath' must produce findings-report." + } + } +} + +$index = [ordered]@{ + version = 1 + generatedAt = (Get-Date).ToUniversalTime().ToString('o') + skillCount = $records.Count + sourceSnapshot = Get-ValueSha256 -Value @($sourceManifest) + skills = @($records) +} + +$parent = Split-Path -Parent $IndexPath +if ($parent -and -not (Test-Path -LiteralPath $parent)) { + New-Item -ItemType Directory -Path $parent -Force | Out-Null +} +Set-Content -LiteralPath $IndexPath -Value ( + ConvertTo-Json -InputObject $index -Depth 12 -Compress +) -Encoding utf8NoBOM + +return $records.Count From b545b22fb9173e0a1f17c3b2e02dc77ea6c92dab Mon Sep 17 00:00:00 2001 From: Stefano Demiliani <33155438+demiliani@users.noreply.github.com> Date: Tue, 15 Sep 2026 10:38:32 +0200 Subject: [PATCH 07/19] Add reporting review guidance and evaluation fixtures (#183) * knowledge(performance): add job queue reliability guidance * Address Job Queue review feedback * Address Job Queue routing review feedback * Encode job queue conflict in fixtures * Add reporting review guidance and fixtures * Fix reporting article reference --- docs/using-bcquality.md | 3 +- evaluation/review-fixtures.json | 13 ++++ ...ariable-before-independent-runmodal.bad.al | 16 +++++ ...riable-before-independent-runmodal.good.al | 17 +++++ ...rt-variable-before-independent-runmodal.md | 32 ++++++++++ ...port-break-ends-the-current-trigger.bad.al | 31 +++++++++ ...ort-break-ends-the-current-trigger.good.al | 31 +++++++++ ...rrreport-break-ends-the-current-trigger.md | 30 +++++++++ ...t-rolls-back-and-skips-onpostreport.bad.al | 27 ++++++++ ...-rolls-back-and-skips-onpostreport.good.al | 28 +++++++++ ...-quit-rolls-back-and-skips-onpostreport.md | 30 +++++++++ ...ort-skip-does-not-stop-trigger-code.bad.al | 26 ++++++++ ...rt-skip-does-not-stop-trigger-code.good.al | 28 +++++++++ ...rreport-skip-does-not-stop-trigger-code.md | 30 +++++++++ ...in-a-loop-needs-one-client-download.bad.al | 14 +++++ ...n-a-loop-needs-one-client-download.good.al | 37 +++++++++++ ...put-in-a-loop-needs-one-client-download.md | 32 ++++++++++ ...-dataitem-trigger-order-is-explicit.bad.al | 30 +++++++++ ...dataitem-trigger-order-is-explicit.good.al | 30 +++++++++ ...sion-dataitem-trigger-order-is-explicit.md | 32 ++++++++++ ...rt-triggers-run-after-base-triggers.bad.al | 31 +++++++++ ...t-triggers-run-after-base-triggers.good.al | 37 +++++++++++ ...report-triggers-run-after-base-triggers.md | 30 +++++++++ ...ew-cannot-broaden-dataitemtableview.bad.al | 26 ++++++++ ...w-cannot-broaden-dataitemtableview.good.al | 26 ++++++++ ...leview-cannot-broaden-dataitemtableview.md | 30 +++++++++ ...equestpage-returns-empty-parameters.bad.al | 17 +++++ ...questpage-returns-empty-parameters.good.al | 20 ++++++ ...runrequestpage-returns-empty-parameters.md | 30 +++++++++ microsoft/skills/review/al-code-review.md | 1 + .../skills/review/al-reporting-review.md | 63 +++++++++++++++++++ 31 files changed, 827 insertions(+), 1 deletion(-) create mode 100644 microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.bad.al create mode 100644 microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.good.al create mode 100644 microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.md create mode 100644 microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.bad.al create mode 100644 microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.good.al create mode 100644 microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.md create mode 100644 microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.bad.al create mode 100644 microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.good.al create mode 100644 microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.md create mode 100644 microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.bad.al create mode 100644 microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.good.al create mode 100644 microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.md create mode 100644 microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.bad.al create mode 100644 microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.good.al create mode 100644 microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.md create mode 100644 microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.bad.al create mode 100644 microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.good.al create mode 100644 microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.md create mode 100644 microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.bad.al create mode 100644 microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.good.al create mode 100644 microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.md create mode 100644 microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.bad.al create mode 100644 microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.good.al create mode 100644 microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.md create mode 100644 microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.bad.al create mode 100644 microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.good.al create mode 100644 microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.md create mode 100644 microsoft/skills/review/al-reporting-review.md diff --git a/docs/using-bcquality.md b/docs/using-bcquality.md index dcaddde..e55751f 100644 --- a/docs/using-bcquality.md +++ b/docs/using-bcquality.md @@ -183,7 +183,7 @@ using your normal compilation, analyzer, test, and human-review workflow. ## Coverage and limits -The Microsoft broad review composes the 16 Microsoft domains listed below. +The Microsoft broad review composes the 17 Microsoft domains listed below. The Community Agents review is a separate skill selected by the request, not a nested part of that coordinator. All current review leaves accept app folders, files, and diffs; request an Agent SDK review explicitly when that @@ -219,6 +219,7 @@ Each article describes one concern. Where samples exist, use its linked | Performance | [Performance](../microsoft/knowledge/performance/) | | Privacy | [Privacy](../microsoft/knowledge/privacy/) | | Query objects | [Query](../microsoft/knowledge/query/) | +| Reporting | [Reporting](../microsoft/knowledge/reporting/) | | Security | [Security](../microsoft/knowledge/security/) | | Style | [Style](../microsoft/knowledge/style/) | | Telemetry | [Telemetry](../microsoft/knowledge/telemetry/) | diff --git a/evaluation/review-fixtures.json b/evaluation/review-fixtures.json index 352fccf..3eeed68 100644 --- a/evaluation/review-fixtures.json +++ b/evaluation/review-fixtures.json @@ -30,6 +30,19 @@ "privacy": { "article": "no-pii-in-telemetry-message-string" }, + "reporting": { + "articles": [ + "clear-report-variable-before-independent-runmodal", + "currreport-break-ends-the-current-trigger", + "currreport-quit-rolls-back-and-skips-onpostreport", + "currreport-skip-does-not-stop-trigger-code", + "report-output-in-a-loop-needs-one-client-download", + "reportextension-dataitem-trigger-order-is-explicit", + "reportextension-report-triggers-run-after-base-triggers", + "settableview-cannot-broaden-dataitemtableview", + "stop-when-runrequestpage-returns-empty-parameters" + ] + }, "style": { "article": "label-comment-explains-placeholders" }, diff --git a/microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.bad.al b/microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.bad.al new file mode 100644 index 0000000..736ddbf --- /dev/null +++ b/microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.bad.al @@ -0,0 +1,16 @@ +codeunit 50102 "Run Customer Reports" +{ + procedure RunBlockedAndUnblockedCustomers() + var + Customer: Record Customer; + CustomerList: Report "Customer - List"; + begin + Customer.SetRange(Blocked, Customer.Blocked::All); + CustomerList.SetTableView(Customer); + CustomerList.RunModal(); + + Customer.SetRange(Blocked, Customer.Blocked::" "); + CustomerList.SetTableView(Customer); + CustomerList.RunModal(); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.good.al b/microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.good.al new file mode 100644 index 0000000..51e5a0e --- /dev/null +++ b/microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.good.al @@ -0,0 +1,17 @@ +codeunit 50102 "Run Customer Reports" +{ + procedure RunBlockedAndUnblockedCustomers() + var + Customer: Record Customer; + CustomerList: Report "Customer - List"; + begin + Customer.SetRange(Blocked, Customer.Blocked::All); + CustomerList.SetTableView(Customer); + CustomerList.RunModal(); + + Clear(CustomerList); + Customer.SetRange(Blocked, Customer.Blocked::" "); + CustomerList.SetTableView(Customer); + CustomerList.RunModal(); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.md b/microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.md new file mode 100644 index 0000000..b82c1fb --- /dev/null +++ b/microsoft/knowledge/reporting/clear-report-variable-before-independent-runmodal.md @@ -0,0 +1,32 @@ +--- +bc-version: [all] +domain: reporting +keywords: [report, runmodal, clear, settableview, instance, state, filters] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Clear a Report variable before an independent RunModal execution + +## Description + +`Report.Run()` automatically clears the report variable after execution, but `Report.RunModal()` does not. Reconfiguring and running the same variable for an independent operation can therefore retain filters and other instance state from the previous run. + +## Best Practice + +Call `Clear(ReportVariable)` before configuring a new, logically independent `RunModal()` execution on a reused report variable. No clear is required after a single execution, and retaining state is valid when the subsequent run intentionally continues with the same configuration. + +See sample: [`clear-report-variable-before-independent-runmodal.good.al`](clear-report-variable-before-independent-runmodal.good.al). + +## Anti Pattern + +Run the same report variable modally for two independent views without clearing it between runs. The second `SetTableView` can only narrow the existing report view, so filters retained by the instance can make the second result incomplete or empty. + +See sample: [`clear-report-variable-before-independent-runmodal.bad.al`](clear-report-variable-before-independent-runmodal.bad.al). + +## References + +`Report.RunModal()` method — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/methods-auto/report/reportinstance-runmodal-method + +`Report.Run()` method — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/methods-auto/report/reportinstance-run-method \ No newline at end of file diff --git a/microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.bad.al b/microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.bad.al new file mode 100644 index 0000000..dd88abc --- /dev/null +++ b/microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.bad.al @@ -0,0 +1,31 @@ +report 50105 "Customer Entry Review" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + trigger OnAfterGetRecord() + var + EntryNo: Integer; + begin + repeat + EntryNo += 1; + if EntryNo = 5 then + CurrReport.Break(); + until EntryNo = 10; + + MarkCustomerReviewed(); + end; + } + } + + local procedure MarkCustomerReviewed() + begin + ReviewedCustomerCount += 1; + end; + + var + ReviewedCustomerCount: Integer; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.good.al b/microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.good.al new file mode 100644 index 0000000..c220732 --- /dev/null +++ b/microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.good.al @@ -0,0 +1,31 @@ +report 50105 "Customer Entry Review" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + trigger OnAfterGetRecord() + var + EntryNo: Integer; + StopReview: Boolean; + begin + repeat + EntryNo += 1; + StopReview := EntryNo = 5; + until StopReview or (EntryNo = 10); + + MarkCustomerReviewed(); + end; + } + } + + local procedure MarkCustomerReviewed() + begin + ReviewedCustomerCount += 1; + end; + + var + ReviewedCustomerCount: Integer; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.md b/microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.md new file mode 100644 index 0000000..ec22ea4 --- /dev/null +++ b/microsoft/knowledge/reporting/currreport-break-ends-the-current-trigger.md @@ -0,0 +1,30 @@ +--- +bc-version: [all] +domain: reporting +keywords: [report, currreport, break, loop, trigger, control-flow] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# CurrReport.Break ends the current trigger + +## Description + +`CurrReport.Break()` inside a report dataitem trigger does more than leave an AL loop. It terminates the current trigger and omits the current record from the dataset. The report runtime still invokes the remaining triggers for that record. Consequently, statements after the loop in the current trigger do not run, while later report triggers can still produce side effects. + +## Best Practice + +Use an explicit loop condition or the AL `break` statement when only the loop must end and the current trigger must continue. Use `CurrReport.Break()` only when ending the trigger and omitting the current record are both intended, and keep subsequent report triggers safe for that omitted record. + +See sample: [`currreport-break-ends-the-current-trigger.good.al`](currreport-break-ends-the-current-trigger.good.al). + +## Anti Pattern + +Call `CurrReport.Break()` inside a loop and rely on statements after the loop to finish processing the current record. Those statements are unreachable when the call executes, the record is omitted, and remaining report triggers still run. + +See sample: [`currreport-break-ends-the-current-trigger.bad.al`](currreport-break-ends-the-current-trigger.bad.al). + +## References + +`Report.Break()` method — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/methods-auto/report/reportinstance-break-method \ No newline at end of file diff --git a/microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.bad.al b/microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.bad.al new file mode 100644 index 0000000..262ca78 --- /dev/null +++ b/microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.bad.al @@ -0,0 +1,27 @@ +report 50101 "Update Customer Review" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + trigger OnAfterGetRecord() + begin + "Last Date Modified" := Today(); + Modify(); + + if Blocked <> Blocked::" " then + CurrReport.Quit(); + end; + } + } + + trigger OnPostReport() + begin + Message(CompletedMsg); + end; + + var + CompletedMsg: Label 'Customer review completed.'; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.good.al b/microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.good.al new file mode 100644 index 0000000..f7a50c5 --- /dev/null +++ b/microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.good.al @@ -0,0 +1,28 @@ +report 50101 "Update Customer Review" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + trigger OnAfterGetRecord() + begin + if Blocked <> Blocked::" " then + Error(BlockedCustomerErr, "No."); + + "Last Date Modified" := Today(); + Modify(); + end; + } + } + + trigger OnPostReport() + begin + Message(CompletedMsg); + end; + + var + BlockedCustomerErr: Label 'Customer %1 is blocked.', Comment = '%1 = customer number'; + CompletedMsg: Label 'Customer review completed.'; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.md b/microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.md new file mode 100644 index 0000000..287f625 --- /dev/null +++ b/microsoft/knowledge/reporting/currreport-quit-rolls-back-and-skips-onpostreport.md @@ -0,0 +1,30 @@ +--- +bc-version: [all] +domain: reporting +keywords: [report, currreport, quit, rollback, onpostreport, transaction, control-flow] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# CurrReport.Quit rolls back report changes and skips OnPostReport + +## Description + +`CurrReport.Quit()` aborts the report without committing database changes made during its execution. It also prevents `OnPostReport` from running. It is therefore not a normal early-return mechanism for a processing report that expects earlier writes or finalization in `OnPostReport` to survive. + +## Best Practice + +Use `CurrReport.Quit()` only when silently aborting the report, rolling back its database changes, and skipping `OnPostReport` are all intentional. When processing must stop with a failure, raise an error. When completed work and `OnPostReport` must be preserved, structure the dataitem control flow without `Quit()`. + +See sample: [`currreport-quit-rolls-back-and-skips-onpostreport.good.al`](currreport-quit-rolls-back-and-skips-onpostreport.good.al). + +## Anti Pattern + +Modify data and then call `CurrReport.Quit()` while relying on those writes or on `OnPostReport` finalization. The report exits without committing its changes and never invokes `OnPostReport`. + +See sample: [`currreport-quit-rolls-back-and-skips-onpostreport.bad.al`](currreport-quit-rolls-back-and-skips-onpostreport.bad.al). + +## References + +`Report.Quit()` method — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/methods-auto/report/reportinstance-quit-method \ No newline at end of file diff --git a/microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.bad.al b/microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.bad.al new file mode 100644 index 0000000..1debd99 --- /dev/null +++ b/microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.bad.al @@ -0,0 +1,26 @@ +report 50100 "Released Customer List" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + trigger OnAfterGetRecord() + begin + if Blocked <> Blocked::" " then + CurrReport.Skip(); + + CountIncludedCustomer(); + end; + } + } + + local procedure CountIncludedCustomer() + begin + IncludedCustomerCount += 1; + end; + + var + IncludedCustomerCount: Integer; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.good.al b/microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.good.al new file mode 100644 index 0000000..f239a2b --- /dev/null +++ b/microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.good.al @@ -0,0 +1,28 @@ +report 50100 "Released Customer List" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + trigger OnAfterGetRecord() + begin + if Blocked <> Blocked::" " then begin + CurrReport.Skip(); + exit; + end; + + CountIncludedCustomer(); + end; + } + } + + local procedure CountIncludedCustomer() + begin + IncludedCustomerCount += 1; + end; + + var + IncludedCustomerCount: Integer; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.md b/microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.md new file mode 100644 index 0000000..d2b4e34 --- /dev/null +++ b/microsoft/knowledge/reporting/currreport-skip-does-not-stop-trigger-code.md @@ -0,0 +1,30 @@ +--- +bc-version: [all] +domain: reporting +keywords: [report, currreport, skip, trigger, onaftergetrecord, control-flow] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# CurrReport.Skip omits the record but does not stop trigger code + +## Description + +`CurrReport.Skip()` omits the current record from the report dataset and continues processing with the next record. It does not terminate the current trigger, and the remaining triggers for the current record still run. Code placed after `Skip()` can therefore produce side effects for a record that never appears in the output. + +## Best Practice + +When no further code in the current trigger should run for a skipped record, call `CurrReport.Skip()` and then exit the trigger explicitly. Keep later record triggers safe for skipped records because the report runtime still invokes them. + +See sample: [`currreport-skip-does-not-stop-trigger-code.good.al`](currreport-skip-does-not-stop-trigger-code.good.al). + +## Anti Pattern + +Call `CurrReport.Skip()` and rely on it to bypass subsequent statements or later record triggers. The record is removed from the dataset, but those statements and triggers can still update state, write data, or perform expensive work. + +See sample: [`currreport-skip-does-not-stop-trigger-code.bad.al`](currreport-skip-does-not-stop-trigger-code.bad.al). + +## References + +`Report.Skip()` method — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/methods-auto/report/reportinstance-skip-method \ No newline at end of file diff --git a/microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.bad.al b/microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.bad.al new file mode 100644 index 0000000..945c0ae --- /dev/null +++ b/microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.bad.al @@ -0,0 +1,14 @@ +codeunit 50106 "Download Customer Reports" +{ + procedure DownloadReports(var Customer: Record Customer) + var + CustomerView: Record Customer; + begin + if Customer.FindSet() then + repeat + CustomerView := Customer; + CustomerView.SetRecFilter(); + Report.Run(Report::"Customer - List", false, false, CustomerView); + until Customer.Next() = 0; + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.good.al b/microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.good.al new file mode 100644 index 0000000..dea2e7f --- /dev/null +++ b/microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.good.al @@ -0,0 +1,37 @@ +codeunit 50106 "Download Customer Reports" +{ + procedure DownloadReports(var Customer: Record Customer) + var + CustomerView: Record Customer; + CustomerList: Report "Customer - List"; + DataCompression: Codeunit "Data Compression"; + ReportTempBlob: Codeunit "Temp Blob"; + ZipTempBlob: Codeunit "Temp Blob"; + ReportInStream: InStream; + ZipInStream: InStream; + ReportOutStream: OutStream; + ZipOutStream: OutStream; + ZipFileName: Text; + begin + DataCompression.CreateZipArchive(); + if Customer.FindSet() then + repeat + Clear(CustomerList); + Clear(ReportTempBlob); + CustomerView := Customer; + CustomerView.SetRecFilter(); + CustomerList.SetTableView(CustomerView); + ReportTempBlob.CreateOutStream(ReportOutStream); + CustomerList.SaveAs('', ReportFormat::Pdf, ReportOutStream); + ReportTempBlob.CreateInStream(ReportInStream); + DataCompression.AddEntry(ReportInStream, Customer."No." + '.pdf'); + until Customer.Next() = 0; + + ZipTempBlob.CreateOutStream(ZipOutStream); + DataCompression.SaveZipArchive(ZipOutStream); + DataCompression.CloseZipArchive(); + ZipTempBlob.CreateInStream(ZipInStream); + ZipFileName := 'CustomerReports.zip'; + DownloadFromStream(ZipInStream, '', '', '*.zip', ZipFileName); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.md b/microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.md new file mode 100644 index 0000000..8aede8b --- /dev/null +++ b/microsoft/knowledge/reporting/report-output-in-a-loop-needs-one-client-download.md @@ -0,0 +1,32 @@ +--- +bc-version: [all] +domain: reporting +keywords: [report, run, saveas, downloadfromstream, web-client, loop, zip, data-compression] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Report output in a loop needs one client download + +## Description + +The Business Central Web client can deliver only one file per request. When AL generates or downloads a report file repeatedly in the same request, only the last file is delivered to the browser. Earlier report output is silently unavailable to the user even though every iteration ran. + +## Best Practice + +Generate each report into a stream, add the streams to one archive, and call `DownloadFromStream` once after the loop. A direct report run or download inside a loop is valid only when the execution context does not use the Web client or the loop is guaranteed to execute at most once. + +See sample: [`report-output-in-a-loop-needs-one-client-download.good.al`](report-output-in-a-loop-needs-one-client-download.good.al). + +## Anti Pattern + +Call `Report.Run`, `Report.RunModal`, or `DownloadFromStream` repeatedly in a loop initiated by one Web client action and expect every generated file to reach the browser. The client receives only the last download. + +See sample: [`report-output-in-a-loop-needs-one-client-download.bad.al`](report-output-in-a-loop-needs-one-client-download.bad.al). + +## References + +`File.DownloadFromStream` method — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/methods-auto/file/file-downloadfromstream-method + +`Data Compression` codeunit — https://learn.microsoft.com/dynamics365/business-central/application/system-application/codeunit/system.io.data-compression \ No newline at end of file diff --git a/microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.bad.al b/microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.bad.al new file mode 100644 index 0000000..7a25a12 --- /dev/null +++ b/microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.bad.al @@ -0,0 +1,30 @@ +report 50103 "Base Customer Export" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + trigger OnPreDataItem() + begin + SetRange(Blocked, Blocked::" "); + SetRange("Country/Region Code"); + end; + } + } +} + +reportextension 50104 "Local Customer Export" extends "Base Customer Export" +{ + dataset + { + modify(Customer) + { + trigger OnBeforePreDataItem() + begin + SetFilter("Country/Region Code", '<>%1', ''); + end; + } + } +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.good.al b/microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.good.al new file mode 100644 index 0000000..52c3ebe --- /dev/null +++ b/microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.good.al @@ -0,0 +1,30 @@ +report 50103 "Base Customer Export" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + trigger OnPreDataItem() + begin + SetRange(Blocked, Blocked::" "); + SetRange("Country/Region Code"); + end; + } + } +} + +reportextension 50104 "Local Customer Export" extends "Base Customer Export" +{ + dataset + { + modify(Customer) + { + trigger OnAfterPreDataItem() + begin + SetFilter("Country/Region Code", '<>%1', ''); + end; + } + } +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.md b/microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.md new file mode 100644 index 0000000..c149748 --- /dev/null +++ b/microsoft/knowledge/reporting/reportextension-dataitem-trigger-order-is-explicit.md @@ -0,0 +1,32 @@ +--- +bc-version: [19..] +domain: reporting +keywords: [reportextension, report, dataitem, trigger-order, onbeforepredataitem, onafterpredataitem, onbeforeaftergetrecord, onafteraftergetrecord] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Choose ReportExtension dataitem triggers by their order around the base trigger + +## Description + +ReportExtension dataitem triggers run at defined points around the corresponding base-report trigger. `OnBeforePreDataItem` and `OnBeforeAfterGetRecord` run before the base trigger; `OnAfterPreDataItem` and `OnAfterAfterGetRecord` run after it. A filter or calculated value can be overwritten when an extension uses a before-trigger even though its result must be final after base processing. + +## Best Practice + +Choose the before or after trigger from the required ordering relative to base behavior. Use an after-trigger when the extension must observe or refine the final view or value produced by the base trigger. A before-trigger is valid when the base report must consume the extension's state. + +See sample: [`reportextension-dataitem-trigger-order-is-explicit.good.al`](reportextension-dataitem-trigger-order-is-explicit.good.al). + +## Anti Pattern + +Place extension logic in a before-trigger while relying on its filter or value to survive a base trigger that can replace it. Do not report a before-trigger merely because an after-trigger exists; the defect requires visible base behavior or another reliable source showing that ordering changes the result. + +See sample: [`reportextension-dataitem-trigger-order-is-explicit.bad.al`](reportextension-dataitem-trigger-order-is-explicit.bad.al). + +## References + +`OnBeforePreDataItem` report-extension trigger — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/triggers-auto/reportextensiondatasetmodify/devenv-onbeforepredataitem-reportextensiondatasetmodify-trigger + +`OnAfterPreDataItem` report-extension trigger — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/triggers-auto/reportextensiondatasetmodify/devenv-onafterpredataitem-reportextensiondatasetmodify-trigger \ No newline at end of file diff --git a/microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.bad.al b/microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.bad.al new file mode 100644 index 0000000..d932f91 --- /dev/null +++ b/microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.bad.al @@ -0,0 +1,31 @@ +report 50110 "Customer Export" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + } + } + + trigger OnPreReport() + var + ExportSetup: Record "Customer Export Setup"; + begin + ExportSetup.Get(); + ExportSetup.TestField("Export Date"); + end; +} + +reportextension 50111 "Customer Export Extension" extends "Customer Export" +{ + trigger OnPreReport() + var + ExportSetup: Record "Customer Export Setup"; + begin + ExportSetup.Get(); + ExportSetup.Validate("Export Date", Today()); + ExportSetup.Modify(true); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.good.al b/microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.good.al new file mode 100644 index 0000000..293784b --- /dev/null +++ b/microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.good.al @@ -0,0 +1,37 @@ +report 50110 "Customer Export" +{ + ProcessingOnly = true; + + dataset + { + dataitem(Customer; Customer) + { + } + } + + trigger OnPreReport() + var + ExportDate: Date; + begin + OnBeforeResolveExportDate(ExportDate); + if ExportDate = 0D then + Error(ExportDateRequiredErr); + end; + + [IntegrationEvent(false, false)] + local procedure OnBeforeResolveExportDate(var ExportDate: Date) + begin + end; + + var + ExportDateRequiredErr: Label 'An export date is required.'; +} + +codeunit 50111 "Customer Export Extension" +{ + [EventSubscriber(ObjectType::Report, Report::"Customer Export", 'OnBeforeResolveExportDate', '', false, false)] + local procedure SetExportDate(var ExportDate: Date) + begin + ExportDate := Today(); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.md b/microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.md new file mode 100644 index 0000000..414752a --- /dev/null +++ b/microsoft/knowledge/reporting/reportextension-report-triggers-run-after-base-triggers.md @@ -0,0 +1,30 @@ +--- +bc-version: [18..] +domain: reporting +keywords: [reportextension, report, trigger-order, onprereport, onpostreport, base-report, integration-event] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# ReportExtension report triggers run after base report triggers + +## Description + +`OnPreReport` and `OnPostReport` on a ReportExtension run after the corresponding triggers on the base report. An extension `OnPreReport` cannot prepare state that the base `OnPreReport` must consume, and an extension `OnPostReport` cannot affect finalization that the base `OnPostReport` has already completed. + +## Best Practice + +Use a base-report event at the required execution point when extension logic must run before or within a base trigger. Use ReportExtension `OnPreReport` and `OnPostReport` only for work that is correct after the corresponding base trigger. Report a violation only when the base trigger and extension dependency are both visible or otherwise established. + +See sample: [`reportextension-report-triggers-run-after-base-triggers.good.al`](reportextension-report-triggers-run-after-base-triggers.good.al). + +## Anti Pattern + +Initialize data in a ReportExtension `OnPreReport` and rely on the base report's `OnPreReport` to consume it, or perform extension `OnPostReport` work that the base `OnPostReport` needed beforehand. The base trigger has already run. + +See sample: [`reportextension-report-triggers-run-after-base-triggers.bad.al`](reportextension-report-triggers-run-after-base-triggers.bad.al). + +## References + +Report extension object — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/devenv-report-ext-object \ No newline at end of file diff --git a/microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.bad.al b/microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.bad.al new file mode 100644 index 0000000..844c542 --- /dev/null +++ b/microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.bad.al @@ -0,0 +1,26 @@ +report 50107 "Selected Sales Orders" +{ + ProcessingOnly = true; + + dataset + { + dataitem(SalesHeader; "Sales Header") + { + DataItemTableView = where("Document Type" = const(Order), Status = const(Open)); + } + } +} + +codeunit 50108 "Run Selected Sales Orders" +{ + procedure RunReleasedOrders() + var + SalesHeader: Record "Sales Header"; + SelectedSalesOrders: Report "Selected Sales Orders"; + begin + SalesHeader.SetRange("Document Type", SalesHeader."Document Type"::Order); + SalesHeader.SetRange(Status, SalesHeader.Status::Released); + SelectedSalesOrders.SetTableView(SalesHeader); + SelectedSalesOrders.RunModal(); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.good.al b/microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.good.al new file mode 100644 index 0000000..5dc4f2e --- /dev/null +++ b/microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.good.al @@ -0,0 +1,26 @@ +report 50107 "Selected Sales Orders" +{ + ProcessingOnly = true; + + dataset + { + dataitem(SalesHeader; "Sales Header") + { + DataItemTableView = where("Document Type" = const(Order)); + } + } +} + +codeunit 50108 "Run Selected Sales Orders" +{ + procedure RunReleasedOrders() + var + SalesHeader: Record "Sales Header"; + SelectedSalesOrders: Report "Selected Sales Orders"; + begin + SalesHeader.SetRange("Document Type", SalesHeader."Document Type"::Order); + SalesHeader.SetRange(Status, SalesHeader.Status::Released); + SelectedSalesOrders.SetTableView(SalesHeader); + SelectedSalesOrders.RunModal(); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.md b/microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.md new file mode 100644 index 0000000..2dfc691 --- /dev/null +++ b/microsoft/knowledge/reporting/settableview-cannot-broaden-dataitemtableview.md @@ -0,0 +1,30 @@ +--- +bc-version: [all] +domain: reporting +keywords: [report, settableview, dataitemtableview, filter, view, narrowing] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# SetTableView cannot broaden DataItemTableView + +## Description + +`Report.SetTableView()` applies the supplied record view by narrowing the view already defined by the report dataitem's `DataItemTableView`. It cannot remove or broaden a static dataitem filter. A caller that requests records excluded by `DataItemTableView` therefore produces an empty dataset rather than overriding the report filter. + +## Best Practice + +Keep only invariant restrictions in `DataItemTableView`. When callers must select among values, leave that dimension open in the static view and pass the required filter through `SetTableView`. Review this as a defect only when the report definition and caller together show a contradictory filter. + +See sample: [`settableview-cannot-broaden-dataitemtableview.good.al`](settableview-cannot-broaden-dataitemtableview.good.al). + +## Anti Pattern + +Define a static filter in `DataItemTableView` and call `SetTableView` with a mutually exclusive filter while expecting the runtime view to replace the static one. The filters are intersected and no records are selected. + +See sample: [`settableview-cannot-broaden-dataitemtableview.bad.al`](settableview-cannot-broaden-dataitemtableview.bad.al). + +## References + +`Report.SetTableView()` method — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/methods-auto/report/reportinstance-settableview-method \ No newline at end of file diff --git a/microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.bad.al b/microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.bad.al new file mode 100644 index 0000000..feccfb6 --- /dev/null +++ b/microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.bad.al @@ -0,0 +1,17 @@ +codeunit 50109 "Export Customer Report" +{ + procedure ExportReport() + var + TempBlob: Codeunit "Temp Blob"; + ReportOutStream: OutStream; + RequestPageParameters: Text; + begin + RequestPageParameters := Report.RunRequestPage(Report::"Customer - List"); + TempBlob.CreateOutStream(ReportOutStream); + Report.SaveAs( + Report::"Customer - List", + RequestPageParameters, + ReportFormat::Pdf, + ReportOutStream); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.good.al b/microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.good.al new file mode 100644 index 0000000..d706a2f --- /dev/null +++ b/microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.good.al @@ -0,0 +1,20 @@ +codeunit 50109 "Export Customer Report" +{ + procedure ExportReport() + var + TempBlob: Codeunit "Temp Blob"; + ReportOutStream: OutStream; + RequestPageParameters: Text; + begin + RequestPageParameters := Report.RunRequestPage(Report::"Customer - List"); + if RequestPageParameters = '' then + exit; + + TempBlob.CreateOutStream(ReportOutStream); + Report.SaveAs( + Report::"Customer - List", + RequestPageParameters, + ReportFormat::Pdf, + ReportOutStream); + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.md b/microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.md new file mode 100644 index 0000000..0cc0e71 --- /dev/null +++ b/microsoft/knowledge/reporting/stop-when-runrequestpage-returns-empty-parameters.md @@ -0,0 +1,30 @@ +--- +bc-version: [all] +domain: reporting +keywords: [report, runrequestpage, cancel, parameters, saveas, execute, print] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Stop when RunRequestPage returns empty parameters + +## Description + +`Report.RunRequestPage()` returns an empty string when the user chooses **Cancel**. Passing that value to `Report.Execute`, `Report.Print`, or `Report.SaveAs` ignores the cancellation and can run the report with default parameters instead. + +## Best Practice + +Test the returned parameter string immediately after `RunRequestPage()` and exit when it is empty. Pass the value to `Execute`, `Print`, or `SaveAs` only after the user has confirmed the request page. + +See sample: [`stop-when-runrequestpage-returns-empty-parameters.good.al`](stop-when-runrequestpage-returns-empty-parameters.good.al). + +## Anti Pattern + +Call `RunRequestPage()` and unconditionally pass its return value to a report execution method. Choosing **Cancel** can still execute, print, or save the report. + +See sample: [`stop-when-runrequestpage-returns-empty-parameters.bad.al`](stop-when-runrequestpage-returns-empty-parameters.bad.al). + +## References + +`Report.RunRequestPage()` method — https://learn.microsoft.com/dynamics365/business-central/dev-itpro/developer/methods-auto/report/report-runrequestpage-method \ No newline at end of file diff --git a/microsoft/skills/review/al-code-review.md b/microsoft/skills/review/al-code-review.md index 9b5f739..dc1f1db 100644 --- a/microsoft/skills/review/al-code-review.md +++ b/microsoft/skills/review/al-code-review.md @@ -25,6 +25,7 @@ sub-skills: - microsoft/skills/review/al-testing-review.md - microsoft/skills/review/al-data-modeling-review.md - microsoft/skills/review/al-query-review.md + - microsoft/skills/review/al-reporting-review.md - microsoft/skills/review/al-appsource-review.md - microsoft/skills/review/al-telemetry-review.md --- diff --git a/microsoft/skills/review/al-reporting-review.md b/microsoft/skills/review/al-reporting-review.md new file mode 100644 index 0000000..52f8f53 --- /dev/null +++ b/microsoft/skills/review/al-reporting-review.md @@ -0,0 +1,63 @@ +--- +kind: action-skill +id: al-reporting-review +version: 1 +title: AL reporting review +description: Reviews AL Report and ReportExtension code against BCQuality reporting guidance. +inputs: [pr-diff, file-path, folder-path] +outputs: [findings-report] +bc-version: [all] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# AL reporting review + +Reviews AL source changes against the `reporting` knowledge domain in BCQuality. This is a leaf action skill composed by `al-code-review`. + +## Source + +Use READ's **Bounded retrieval for review skills** workflow with `-Domain reporting`. Consume every catalog page across enabled layers before applying this leaf's Relevance and Worklist; preserve each exact catalog path and open complete bodies only for exact paths selected by the Worklist. If the helper or prepared index is unavailable or invalid, use READ's explicit path-discovery and bounded native-read fallback. + +## Relevance + +Apply READ's frontmatter matching rules against the task context. Use the target version from `app.json` when available and `[al]` for technologies. Retain conditionally applicable files only when configured; cap resulting confidence at `medium` and name every unknown dimension in the finding message. + +Return `not-applicable` when the input contains no Report or ReportExtension declaration and no Report variable or method call. + +## Worklist + +Match relevant entries against changed `report` and `reportextension` objects, variables typed as `Report`, and the tokens `CurrReport`, `Skip`, `Break`, `Quit`, `Run`, `RunModal`, `RunRequestPage`, `Execute`, `Print`, `SaveAs`, `DownloadFromStream`, `Data Compression`, `SetTableView`, `DataItemTableView`, `OnPreReport`, `OnPostReport`, `OnPreDataItem`, `OnAfterGetRecord`, and report-extension dataset triggers. + +Apply this targeted check even when token overlap would rank the article below the worklist cutoff: + +- The same Report variable has two logically independent `RunModal()` executions without `Clear` before the second configuration — `clear-report-variable-before-independent-runmodal`. +- `CurrReport.Break()` is used inside an explicit loop while reachable statements after the loop are expected to finish the current trigger — `currreport-break-ends-the-current-trigger`. +- `CurrReport.Quit()` follows database writes or the report relies on `OnPostReport` finalization — `currreport-quit-rolls-back-and-skips-onpostreport`. +- `CurrReport.Skip()` is followed by reachable code in the same trigger, or later record triggers contain work that is unsafe for skipped records — `currreport-skip-does-not-stop-trigger-code`. +- A loop reachable from one Web client action calls `Report.Run`, `Report.RunModal`, or `DownloadFromStream` more than once instead of producing one archive download — `report-output-in-a-loop-needs-one-client-download`. Do not select this article when the context is non-Web or the loop is provably single-iteration. +- A ReportExtension before-trigger establishes a filter or value that visible base-trigger code subsequently replaces — `reportextension-dataitem-trigger-order-is-explicit`. Do not select this article from a before-trigger alone. +- A ReportExtension `OnPreReport` prepares state consumed by the base `OnPreReport`, or its `OnPostReport` prepares state already consumed by the base `OnPostReport` — `reportextension-report-triggers-run-after-base-triggers`. Require visible base behavior or equivalent established evidence. +- A report's `DataItemTableView` and a caller's `SetTableView` apply mutually exclusive filters to the same field — `settableview-cannot-broaden-dataitemtableview`. Require both views or equivalent direct evidence; `SetTableView` alone is not a finding. +- The value returned by `Report.RunRequestPage()` reaches `Report.Execute`, `Report.Print`, or `Report.SaveAs` without an empty-string cancellation check — `stop-when-runrequestpage-returns-empty-parameters`. + +Resolve layer conflicts per READ. When no reporting knowledge exists, emit `no-knowledge`; when knowledge exists but no article matches the changed report code, emit `completed` with no findings. + +## Action + +Evaluate every worklist article against the diff's report control flow and surrounding triggers. + +- Emit `major` for an unambiguous Anti Pattern that causes incorrect output, persisted side effects, or lost work. +- Emit `minor` when code contradicts a Best Practice but the effect depends on unseen report or caller context. +- Do not emit applicability-only information. A reporting article produces a finding only when changed code violates its normative guidance. + +Set confidence to `high` for locally visible control flow and `medium` when base-report behavior, callers, or missing context affect the conclusion. Domain-scoped agent findings follow DO's precision bar and remain capped at `minor`/`medium`. + +Provide `suggested-code` only when the replacement is complete, local, and unambiguous. Otherwise set `suggested-code-omission-reason`. + +Outcome selection follows DO: `completed`, `no-knowledge`, `not-applicable`, `partial`, or `failed`. + +## Output + +Output conforms to the DO findings-report contract. Every finding this skill emits MUST set `findings[].domain` to `"Reporting"`. \ No newline at end of file From d24dc7b14b39624e8b2160dbca73d66140106e4c Mon Sep 17 00:00:00 2001 From: Jesper Schulz-Wedde Date: Tue, 15 Sep 2026 10:45:35 +0200 Subject: [PATCH 08/19] Fix reporting skill index expectation (#185) Co-authored-by: Jesper Schulz-Wedde Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/scripts/Test-SkillIndex.ps1 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/scripts/Test-SkillIndex.ps1 b/.github/scripts/Test-SkillIndex.ps1 index 839042f..ed07204 100644 --- a/.github/scripts/Test-SkillIndex.ps1 +++ b/.github/scripts/Test-SkillIndex.ps1 @@ -91,6 +91,7 @@ try { 'microsoft/skills/review/al-testing-review.md', 'microsoft/skills/review/al-data-modeling-review.md', 'microsoft/skills/review/al-query-review.md', + 'microsoft/skills/review/al-reporting-review.md', 'microsoft/skills/review/al-appsource-review.md', 'microsoft/skills/review/al-telemetry-review.md' ) @@ -99,7 +100,7 @@ try { throw "Expected exactly one al-code-review record, found $($review.Count)." } if ((@($review[0].subSkills) -join "`n") -cne ($expectedLeaves -join "`n")) { - throw 'al-code-review subSkills did not preserve the declared 16-leaf order.' + throw 'al-code-review subSkills did not preserve the declared 17-leaf order.' } foreach ($leafPath in $expectedLeaves) { $leaf = @($skills | Where-Object path -ceq $leafPath) @@ -237,4 +238,4 @@ finally { Remove-Item -LiteralPath $tmp -Recurse -Force -ErrorAction SilentlyContinue } -Write-Output 'Skill-index check PASSED: deterministic, schema-valid, and all 16 review leaves preserved in order.' +Write-Output 'Skill-index check PASSED: deterministic, schema-valid, and all 17 review leaves preserved in order.' From b7617fb48a01c9f41116e37edaede3daed424236 Mon Sep 17 00:00:00 2001 From: waldo Date: Tue, 15 Sep 2026 12:34:28 +0200 Subject: [PATCH 09/19] knowledge(events): ChangeCompany leaves triggers and trigger-event subscribers running in the calling company (#152) * knowledge(events): ChangeCompany leaves triggers and trigger-event subscribers running in the calling company ChangeCompany redirects only the data access of a record variable; Learn states that triggers still run in the current company. The database trigger events are raised on every database operation and only pass RunTrigger to the subscriber, so Insert(false) after ChangeCompany still runs every subscriber in the calling company. Generated code either assumes the record 'becomes' a target-company record, or switches RunTrigger off and hand-copies the trigger logic, leaving the subscribers writing to the wrong company; none of the tested runs reached StartSession with the company parameter. Co-Authored-By: Claude Fable 5.1 * knowledge(events): qualify StartSession async semantics and fix concurrency-unsafe sample key Addresses PR #152 review: StartSession is a fire-and-forget background session (Ok reports only whether it started, not whether the codeunit succeeded, and errors inside it do not propagate), so the Best Practice now scopes the recommendation and calls out the durable status/error channel a synchronous-success write needs. The good sample's FindLast()+1 entry-number pattern raced under concurrent background sessions; switched to AutoIncrement, which the platform guarantees is unique across concurrent transactions. Co-Authored-By: Claude Sonnet 5 * knowledge(events): serialize the setup-counter increment; promote article and wire the review skill PR #152 round 3 (JesperSchulz): - The good sample's OnAfterInsertEvent subscriber still raced on the shared "Transfer Setup Good" singleton (Get/increment/Modify); AutoIncrement only protected the request key. Added TransferSetup.LockTable() before Get() to serialize concurrent background sessions. - Promoted changecompany-runs-triggers-in-the-calling-company from community/knowledge/events/ to microsoft/knowledge/events/, and wired ChangeCompany/StartSession/RunTrigger tokens plus a targeted detection cue into microsoft/skills/review/al-events-review.md so a diff containing the anti-pattern reliably worklists this article, preserving the documented RunTrigger=false hand-off exception. Co-Authored-By: Claude Sonnet 5 --------- Co-authored-by: waldo1001 <12088142+waldo1001@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 --- ...uns-triggers-in-the-calling-company.bad.al | 91 +++++++++++++++++++ ...ns-triggers-in-the-calling-company.good.al | 84 +++++++++++++++++ ...ny-runs-triggers-in-the-calling-company.md | 42 +++++++++ microsoft/skills/review/al-events-review.md | 3 +- 4 files changed, 219 insertions(+), 1 deletion(-) create mode 100644 microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.bad.al create mode 100644 microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.good.al create mode 100644 microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.md diff --git a/microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.bad.al b/microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.bad.al new file mode 100644 index 0000000..7b8e7d5 --- /dev/null +++ b/microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.bad.al @@ -0,0 +1,91 @@ +codeunit 50100 "Transfer Request Bad" +{ + // Self-contained demonstration of the anti pattern. Not derived from base-app source. + procedure RequestFromCompany(TargetCompany: Text[30]; ItemNo: Code[20]; Quantity: Decimal) + var + TransferRequest: Record "Transfer Request Bad"; + TransferSetup: Record "Transfer Setup Bad"; + begin + TransferRequest.ChangeCompany(TargetCompany); + TransferSetup.ChangeCompany(TargetCompany); + TransferSetup.Get(); + + TransferRequest.Init(); + TransferRequest."Entry No." := NextEntryNo(TargetCompany); + TransferRequest."Item No." := ItemNo; + TransferRequest.Quantity := Quantity; + // OnInsert is skipped below, so the default is copied by hand from the target company's setup. + TransferRequest."Location Code" := TransferSetup."Default Location Code"; + // The OnAfterInsertEvent subscriber still fires, in the calling company, and grows the caller's counter. + TransferRequest.Insert(false); + + TransferSetup."Open Requests" += 1; + TransferSetup.Modify(); + end; + + local procedure NextEntryNo(TargetCompany: Text[30]): Integer + var + LastRequest: Record "Transfer Request Bad"; + begin + LastRequest.ChangeCompany(TargetCompany); + if LastRequest.FindLast() then + exit(LastRequest."Entry No." + 1); + exit(1); + end; +} + +table 50100 "Transfer Request Bad" +{ + DataClassification = CustomerContent; + + fields + { + field(1; "Entry No."; Integer) { } + field(2; "Item No."; Code[20]) { } + field(3; Quantity; Decimal) { } + field(4; "Location Code"; Code[10]) { } + } + + keys + { + key(PK; "Entry No.") { Clustered = true; } + } + + trigger OnInsert() + var + TransferSetup: Record "Transfer Setup Bad"; + begin + TransferSetup.Get(); + "Location Code" := TransferSetup."Default Location Code"; + end; +} + +table 50101 "Transfer Setup Bad" +{ + DataClassification = CustomerContent; + + fields + { + field(1; "Primary Key"; Code[10]) { } + field(2; "Default Location Code"; Code[10]) { } + field(3; "Open Requests"; Integer) { } + } + + keys + { + key(PK; "Primary Key") { Clustered = true; } + } +} + +codeunit 50101 "Transfer Request Count Bad" +{ + [EventSubscriber(ObjectType::Table, Database::"Transfer Request Bad", OnAfterInsertEvent, '', false, false)] + local procedure CountOpenRequest(var Rec: Record "Transfer Request Bad"; RunTrigger: Boolean) + var + TransferSetup: Record "Transfer Setup Bad"; + begin + TransferSetup.Get(); + TransferSetup."Open Requests" += 1; + TransferSetup.Modify(); + end; +} diff --git a/microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.good.al b/microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.good.al new file mode 100644 index 0000000..dd92d9d --- /dev/null +++ b/microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.good.al @@ -0,0 +1,84 @@ +codeunit 50100 "Transfer Request Good" +{ + // Self-contained demonstration of the best practice. Not derived from base-app source. + procedure RequestFromCompany(TargetCompany: Text[30]; ItemNo: Code[20]; Quantity: Decimal) + var + TransferRequest: Record "Transfer Request Good"; + SessionId: Integer; + begin + TransferRequest.Init(); + TransferRequest."Item No." := ItemNo; + TransferRequest.Quantity := Quantity; + // The insert runs inside TargetCompany, so OnInsert and the subscriber read that company's setup. + StartSession(SessionId, Codeunit::"Transfer Request Create Good", TargetCompany, TransferRequest); + end; +} + +codeunit 50102 "Transfer Request Create Good" +{ + TableNo = "Transfer Request Good"; + + trigger OnRun() + begin + // "Entry No." is AutoIncrement, so concurrent background sessions in TargetCompany never race on the same value. + Rec.Insert(true); + end; +} + +table 50100 "Transfer Request Good" +{ + DataClassification = CustomerContent; + + fields + { + field(1; "Entry No."; Integer) { AutoIncrement = true; } + field(2; "Item No."; Code[20]) { } + field(3; Quantity; Decimal) { } + field(4; "Location Code"; Code[10]) { } + } + + keys + { + key(PK; "Entry No.") { Clustered = true; } + } + + trigger OnInsert() + var + TransferSetup: Record "Transfer Setup Good"; + begin + TransferSetup.Get(); + "Location Code" := TransferSetup."Default Location Code"; + end; +} + +table 50101 "Transfer Setup Good" +{ + DataClassification = CustomerContent; + + fields + { + field(1; "Primary Key"; Code[10]) { } + field(2; "Default Location Code"; Code[10]) { } + field(3; "Open Requests"; Integer) { } + } + + keys + { + key(PK; "Primary Key") { Clustered = true; } + } +} + +codeunit 50101 "Transfer Request Count Good" +{ + [EventSubscriber(ObjectType::Table, Database::"Transfer Request Good", OnAfterInsertEvent, '', false, false)] + local procedure CountOpenRequest(var Rec: Record "Transfer Request Good"; RunTrigger: Boolean) + var + TransferSetup: Record "Transfer Setup Good"; + begin + // Serializes the read-modify-write so concurrent background sessions don't lose an increment. + TransferSetup.LockTable(); + TransferSetup.Get(); + TransferSetup."Open Requests" += 1; + TransferSetup.Modify(); + end; +} diff --git a/microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.md b/microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.md new file mode 100644 index 0000000..4a524f4 --- /dev/null +++ b/microsoft/knowledge/events/changecompany-runs-triggers-in-the-calling-company.md @@ -0,0 +1,42 @@ +--- +bc-version: [all] +domain: events +keywords: [changecompany, cross-company, runtrigger, trigger-event, subscriber, onafterinsertevent, insert, startsession, multi-company] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# ChangeCompany leaves triggers and trigger-event subscribers running in the calling company + +> Contributions welcome — open a PR to refine or extend this article. + +## Description + +`ChangeCompany` redirects the data access of one record variable to another company's table. Execution context does not move with it: Microsoft Learn states that triggers still run in the current company, not in the company passed to `ChangeCompany`. Code that knows this usually reaches for `Insert(false)` and copies the trigger's work by hand from the target company's setup. That closes only half of the gap. The runtime raises the database trigger events (`OnBeforeInsertEvent`, `OnAfterInsertEvent`, and their modify, delete, and rename counterparts) on every database operation and only passes the `RunTrigger` flag to the subscriber, so every subscriber that does not exit on `RunTrigger = false` still runs, in the calling company, against the calling company's setup, number series, and companion tables. The row lands in the target company, the side effects land in the caller, and nothing reports an error. The per-row cost of the call is a separate concern, see `changecompany-in-loop-drops-caches`. + +## Best Practice + +Use `ChangeCompany` to read. Access rights in the target company are still enforced, so reads are safe. When the goal is business data in another company, run the code in that company: `StartSession` takes a company name and runs a codeunit there, so triggers, validation, and subscribers all execute with the target company as their context. `StartSession` is a background session, not a synchronous call: the `Ok` return value reports only whether the session started, not whether the codeunit's work inside it succeeded, the caller's transaction does not extend into it, and an error raised there does not come back to the caller — it has to be logged or telemetered from inside that session. Reach for `StartSession` only for work the caller does not need to confirm before it continues; a write whose success the caller must know synchronously needs a durable status or error channel (a field the caller polls, a job queue with retry) rather than a bare `StartSession` call. Learn notes that a background session costs as much as a user session to start, so batch the work rather than starting one session per row, or let the target company process a hand-off row on its own schedule. A direct cross-company write is acceptable only as such a hand-off into a table the writing extension owns, whose triggers do not read company data and whose trigger-event subscribers exit when `RunTrigger` is false, using `Insert(false)`, `Modify(false)`, or `Delete(false)`, and never `Validate`. + +See sample: [`changecompany-runs-triggers-in-the-calling-company.good.al`](changecompany-runs-triggers-in-the-calling-company.good.al). + +## Anti Pattern + +An `Insert`, `Modify`, `Delete`, or `Validate` on a record variable after `ChangeCompany()`, on a table whose triggers or trigger-event subscribers read setup, consume a number series, or write companion rows. With `RunTrigger = true` the trigger code fills the row from the caller's setup. With `RunTrigger = false` the trigger code is skipped, but the subscribers still fire in the caller, so a counter, log, or companion row maintained by a subscriber is written in the wrong company, and a caller that also updates the target by hand counts twice. + +Detection signal: a record variable that has had `ChangeCompany` called on it with a company name and is later used with `Insert`, `Modify`, `Delete`, or `Validate`, where the table is not owned by the extension, or has triggers that read company data, or has trigger-event subscribers that do not exit on `RunTrigger = false`. Do not flag reads after `ChangeCompany`; writes with `RunTrigger = false` into an owned table whose triggers do not read company data and whose subscribers exit on `RunTrigger = false`; or `ChangeCompany()` without an argument, which points the variable back at the current company. + +See sample: [`changecompany-runs-triggers-in-the-calling-company.bad.al`](changecompany-runs-triggers-in-the-calling-company.bad.al). + +## See also + +- Record.ChangeCompany method, Remarks — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/methods-auto/record/record-changecompany-method +- Record.Insert(Boolean) method, RunTrigger — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/methods-auto/record/record-insert-boolean-method +- Record.Delete method, RunTrigger defaults to false — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/methods-auto/record/record-delete-method +- OnInsert (Table) trigger, Remarks — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/triggers-auto/table/devenv-oninsert-table-trigger +- Event types, Database trigger events and order of event execution — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/devenv-event-types +- OnAfterInsertEvent trigger event, RunTrigger parameter — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/triggers-auto/events/table/devenv-onafterinsertevent-table-trigger +- Session.StartSession method, Company parameter, Remarks (background session, no UI), and Return Value (`Ok` reports whether the session started, not whether the codeunit's work succeeded) — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/methods-auto/session/session-startsession-integer-integer-string-table-method +- AL error handling, error handling strategies: an error inside a rolled-back transaction is logged from a background session or telemetry, it does not return to the caller — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/devenv-al-error-handling +- AutoIncrement property, Remarks: "if several transactions are performed at the same time, they will each be assigned a different number" — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/properties/devenv-autoincrement-property diff --git a/microsoft/skills/review/al-events-review.md b/microsoft/skills/review/al-events-review.md index 68bcdb0..b257d3d 100644 --- a/microsoft/skills/review/al-events-review.md +++ b/microsoft/skills/review/al-events-review.md @@ -39,7 +39,7 @@ Narrow the relevant files to the subset that applies to the changes under review - The changed AL object names and types — especially codeunits that publish events or host event subscribers, posting/release/validation routines that should expose extension points, and test codeunits that bind subscribers. - The changed procedures and triggers, weighted toward event publisher methods, methods carrying the `[EventSubscriber(...)]` attribute, routines that raise `OnBefore`/`OnAfter` events, and any procedure that calls `BindSubscription`/`UnbindSubscription`. -- Tokens extracted from the diff that relate to events and the publish/subscribe model (`IntegrationEvent`, `BusinessEvent`, `InternalEvent`, `EventSubscriber`, `IsHandled`, `BindSubscription`, `UnbindSubscription`, `EventSubscriberInstance`, `OnBefore`, `OnAfter`, `Manual`, `IncludeSender`, `GlobalVarAccess`, `Isolated`, `local`, `internal`, `Sender`, `this`, `RecordRef`, `xRec`, `temporary`, `Temp`, `repeat`). +- Tokens extracted from the diff that relate to events and the publish/subscribe model (`IntegrationEvent`, `BusinessEvent`, `InternalEvent`, `EventSubscriber`, `IsHandled`, `BindSubscription`, `UnbindSubscription`, `EventSubscriberInstance`, `OnBefore`, `OnAfter`, `Manual`, `IncludeSender`, `GlobalVarAccess`, `Isolated`, `local`, `internal`, `Sender`, `this`, `RecordRef`, `xRec`, `temporary`, `Temp`, `repeat`, `ChangeCompany`, `StartSession`, `RunTrigger`). A file enters the candidate worklist when its `keywords` intersect the extracted tokens or its topic (derived from the index entry's `path`, `title`, and `description`) matches a changed object type. Read an article's full file — its `## Best Practice` / `## Anti Pattern` bodies — only after it makes the worklist; candidate selection uses the index alone. @@ -65,6 +65,7 @@ The following targeted checks map diff signals to specific `events` articles. Tr - A `RecordRef` event parameter, or a passed-through `xRec`, where a concrete typed record fits — `avoid-loosely-typed-event-parameters`. - A `var IsHandled` added to a pre-existing event rather than introduced through a new `OnBefore` publisher — `do-not-add-ishandled-to-an-existing-event`. - An `if IsHandled then exit;` whose skipped body performs posting, ledger-entry creation, number-series consumption, or integrity/permission validation — `do-not-bypass-critical-operations-with-ishandled`. +- A record variable that had `ChangeCompany()` called on it and is later used with `Insert`, `Modify`, `Delete`, or `Validate`, where the table is not owned by the extension, has triggers that read company data, or has trigger-event subscribers that do not exit on `RunTrigger = false` — `changecompany-runs-triggers-in-the-calling-company`. Do not match a read-only use after `ChangeCompany`, a write with `RunTrigger = false` into an extension-owned table whose triggers do not read company data and whose trigger-event subscribers exit on `RunTrigger = false`, or the parameterless `ChangeCompany()` reset. ## Action From 861f53dd97fcc25cc797e79902884c7bbb612178 Mon Sep 17 00:00:00 2001 From: Stefano Demiliani <33155438+demiliani@users.noreply.github.com> Date: Tue, 15 Sep 2026 12:51:15 +0200 Subject: [PATCH 10/19] Add query filter semantics guidance (#186) --- evaluation/review-fixtures.json | 8 ++++ ...er-cannot-be-overwritten-at-runtime.bad.al | 39 +++++++++++++++++++ ...r-cannot-be-overwritten-at-runtime.good.al | 38 ++++++++++++++++++ ...filter-cannot-be-overwritten-at-runtime.md | 30 ++++++++++++++ ...ilter-overwrites-query-columnfilter.bad.al | 37 ++++++++++++++++++ ...lter-overwrites-query-columnfilter.good.al | 38 ++++++++++++++++++ ...setfilter-overwrites-query-columnfilter.md | 30 ++++++++++++++ microsoft/skills/review/al-query-review.md | 8 ++-- 8 files changed, 225 insertions(+), 3 deletions(-) create mode 100644 microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.bad.al create mode 100644 microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.good.al create mode 100644 microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.md create mode 100644 microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.bad.al create mode 100644 microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.good.al create mode 100644 microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.md diff --git a/evaluation/review-fixtures.json b/evaluation/review-fixtures.json index 3eeed68..90c42c6 100644 --- a/evaluation/review-fixtures.json +++ b/evaluation/review-fixtures.json @@ -30,6 +30,14 @@ "privacy": { "article": "no-pii-in-telemetry-message-string" }, + "query": { + "articles": [ + "dataitemtablefilter-cannot-be-overwritten-at-runtime", + "reopening-query-resets-cursor-but-keeps-filters", + "set-query-filters-before-open", + "setfilter-overwrites-query-columnfilter" + ] + }, "reporting": { "articles": [ "clear-report-variable-before-independent-runmodal", diff --git a/microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.bad.al b/microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.bad.al new file mode 100644 index 0000000..67012eb --- /dev/null +++ b/microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.bad.al @@ -0,0 +1,39 @@ +query 50428 "Static Query Filter Bad" +{ + QueryType = Normal; + + elements + { + dataitem(SalesHeader; "Sales Header") + { + DataItemTableFilter = Status = const(Open); + + column(DocumentNo; "No.") + { + } + filter(StatusFilter; Status) + { + } + } + } +} + +codeunit 50429 "Static Query Filter Bad" +{ + procedure ReadReleasedOrders() + var + SalesHeader: Record "Sales Header"; + SalesHeaderQuery: Query "Static Query Filter Bad"; + begin + // This is combined with Status = Open and returns no rows. + SalesHeaderQuery.SetRange(StatusFilter, SalesHeader.Status::Released); + SalesHeaderQuery.Open(); + while SalesHeaderQuery.Read() do + ProcessOrder(SalesHeaderQuery.DocumentNo); + SalesHeaderQuery.Close(); + end; + + local procedure ProcessOrder(DocumentNo: Code[20]) + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.good.al b/microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.good.al new file mode 100644 index 0000000..d095330 --- /dev/null +++ b/microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.good.al @@ -0,0 +1,38 @@ +query 50430 "Static Query Filter Good" +{ + QueryType = Normal; + + elements + { + dataitem(SalesHeader; "Sales Header") + { + DataItemTableFilter = "Document Type" = const(Order); + + column(DocumentNo; "No.") + { + } + filter(StatusFilter; Status) + { + } + } + } +} + +codeunit 50431 "Static Query Filter Good" +{ + procedure ReadReleasedOrders() + var + SalesHeader: Record "Sales Header"; + SalesHeaderQuery: Query "Static Query Filter Good"; + begin + SalesHeaderQuery.SetRange(StatusFilter, SalesHeader.Status::Released); + SalesHeaderQuery.Open(); + while SalesHeaderQuery.Read() do + ProcessOrder(SalesHeaderQuery.DocumentNo); + SalesHeaderQuery.Close(); + end; + + local procedure ProcessOrder(DocumentNo: Code[20]) + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.md b/microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.md new file mode 100644 index 0000000..6a96db1 --- /dev/null +++ b/microsoft/knowledge/query/dataitemtablefilter-cannot-be-overwritten-at-runtime.md @@ -0,0 +1,30 @@ +--- +bc-version: [all] +domain: query +keywords: [query, dataitemtablefilter, setfilter, setrange, static-filter, filter-precedence] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# DataItemTableFilter cannot be overwritten at runtime + +## Description + +`DataItemTableFilter` defines a static filter on a Query dataitem. A runtime `SetFilter` or `SetRange` on the same source field does not replace that filter. The static and runtime filters are combined with AND, so contradictory values produce an empty dataset instead of broadening or replacing the query definition. + +## Best Practice + +Keep only invariant restrictions in `DataItemTableFilter`. Expose caller-selectable fields through a column or filter row and apply their values with `SetFilter` or `SetRange` before `Open()`. When both filter types intentionally target the same field, ensure their intersection represents the required dataset. + +See sample: [`dataitemtablefilter-cannot-be-overwritten-at-runtime.good.al`](dataitemtablefilter-cannot-be-overwritten-at-runtime.good.al). + +## Anti Pattern + +Define a static filter in `DataItemTableFilter`, then apply a contradictory runtime filter to the same source field while expecting the runtime filter to replace the static one. Both filters remain effective and the query returns no rows. + +See sample: [`dataitemtablefilter-cannot-be-overwritten-at-runtime.bad.al`](dataitemtablefilter-cannot-be-overwritten-at-runtime.bad.al). + +## References + +Filtering in Query objects — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/devenv-query-filters \ No newline at end of file diff --git a/microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.bad.al b/microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.bad.al new file mode 100644 index 0000000..f286334 --- /dev/null +++ b/microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.bad.al @@ -0,0 +1,37 @@ +query 50432 "Column Query Filter Bad" +{ + QueryType = Normal; + + elements + { + dataitem(SalesLine; "Sales Line") + { + column(DocumentNo; "Document No.") + { + } + column(LineQuantity; Quantity) + { + ColumnFilter = LineQuantity = filter(> 0); + } + } + } +} + +codeunit 50433 "Column Query Filter Bad" +{ + procedure ReadSmallPositiveLines() + var + SalesLineQuery: Query "Column Query Filter Bad"; + begin + // This replaces > 0, so negative quantities are also returned. + SalesLineQuery.SetFilter(LineQuantity, '<100'); + SalesLineQuery.Open(); + while SalesLineQuery.Read() do + ProcessLine(SalesLineQuery.DocumentNo, SalesLineQuery.LineQuantity); + SalesLineQuery.Close(); + end; + + local procedure ProcessLine(DocumentNo: Code[20]; Quantity: Decimal) + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.good.al b/microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.good.al new file mode 100644 index 0000000..49353e6 --- /dev/null +++ b/microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.good.al @@ -0,0 +1,38 @@ +query 50434 "Column Query Filter Good" +{ + QueryType = Normal; + + elements + { + dataitem(SalesLine; "Sales Line") + { + DataItemTableFilter = Quantity = filter(> 0); + + column(DocumentNo; "Document No.") + { + } + column(LineQuantity; Quantity) + { + } + } + } +} + +codeunit 50435 "Column Query Filter Good" +{ + procedure ReadSmallPositiveLines() + var + SalesLineQuery: Query "Column Query Filter Good"; + begin + // This combines with the invariant Quantity > 0 dataitem filter. + SalesLineQuery.SetFilter(LineQuantity, '<100'); + SalesLineQuery.Open(); + while SalesLineQuery.Read() do + ProcessLine(SalesLineQuery.DocumentNo, SalesLineQuery.LineQuantity); + SalesLineQuery.Close(); + end; + + local procedure ProcessLine(DocumentNo: Code[20]; Quantity: Decimal) + begin + end; +} \ No newline at end of file diff --git a/microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.md b/microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.md new file mode 100644 index 0000000..9f9dd7a --- /dev/null +++ b/microsoft/knowledge/query/setfilter-overwrites-query-columnfilter.md @@ -0,0 +1,30 @@ +--- +bc-version: [all] +domain: query +keywords: [query, columnfilter, setfilter, setrange, filter-precedence, runtime-filter] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# SetFilter and SetRange overwrite Query ColumnFilter + +## Description + +`ColumnFilter` on a Query column or filter row defines a dynamic filter. A runtime `SetFilter` or `SetRange` on that same column or filter row replaces the `ColumnFilter`; it does not combine the two conditions. Rows excluded by the declarative filter can therefore reappear when the runtime filter omits that restriction. + +## Best Practice + +Place invariant restrictions in `DataItemTableFilter`, which runtime filters cannot overwrite. When a `ColumnFilter` is intentionally replaceable, make each runtime `SetFilter` or `SetRange` express the complete required condition before `Open()`. + +See sample: [`setfilter-overwrites-query-columnfilter.good.al`](setfilter-overwrites-query-columnfilter.good.al). + +## Anti Pattern + +Apply `SetFilter` or `SetRange` to a column or filter row and rely on its existing `ColumnFilter` to remain effective. The runtime call replaces that filter and can admit rows that the query definition appeared to exclude. + +See sample: [`setfilter-overwrites-query-columnfilter.bad.al`](setfilter-overwrites-query-columnfilter.bad.al). + +## References + +Filtering in Query objects — https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/devenv-query-filters \ No newline at end of file diff --git a/microsoft/skills/review/al-query-review.md b/microsoft/skills/review/al-query-review.md index c52ddd8..e97a20a 100644 --- a/microsoft/skills/review/al-query-review.md +++ b/microsoft/skills/review/al-query-review.md @@ -32,22 +32,24 @@ Match relevant entries against changed `query` objects, variables typed as `Quer The following targeted checks cover every current `query` article: +- A `DataItemTableFilter` and a runtime `SetFilter` or `SetRange` constrain the same source field incompatibly, while the runtime call is intended to replace or broaden the static filter — `dataitemtablefilter-cannot-be-overwritten-at-runtime`. - `SetFilter` or `SetRange` occurs after `Open()` without a new `Open()` before the next `Read()` — `set-query-filters-before-open`. +- A runtime `SetFilter` or `SetRange` replaces a `ColumnFilter` on the same column or filter row, while later code relies on the declarative restriction remaining effective — `setfilter-overwrites-query-columnfilter`. - An already-open query is opened again as if that advanced the cursor, or a query variable is reused for an independent operation without `Clear` even though old filters must not carry over — `reopening-query-resets-cursor-but-keeps-filters`. Resolve layer conflicts per READ. When no query knowledge exists, emit `no-knowledge`; when knowledge exists but no article matches the changed Query usage, emit `completed` with no findings. ## Action -Evaluate every worklist article against the diff's Query call order and surrounding control flow. +Evaluate every worklist article against the Query definition, the diff's call order, and surrounding control flow. For filter-precedence findings, require both the declarative filter and the runtime call to be visible, and require local evidence that replacement, broadening, or retention of the original filter is intended. -- Emit `major` for an unambiguous Anti Pattern that can close the dataset, restart processing, or retain an unintended filter. +- Emit `major` for an unambiguous Anti Pattern that can close the dataset, restart processing, retain an unintended filter, produce an empty intersection, or admit rows excluded by an overwritten filter. - Emit `minor` when code contradicts a Best Practice but the resulting behavior depends on unseen control flow. - Do not emit applicability-only information. A Query article produces a finding only when the changed code violates its normative guidance. Set confidence to `high` for a locally visible call sequence and `medium` when aliases, helper calls, or missing context obscure the sequence. Domain-scoped agent findings follow DO's precision bar and remain capped at `minor`/`medium`. -Provide `suggested-code` only when moving a filter before `Open()` or adding `Clear` is a complete, local, unambiguous replacement. Otherwise set `suggested-code-omission-reason`. +Provide `suggested-code` only when moving a filter before `Open()`, adding `Clear`, moving an invariant restriction to `DataItemTableFilter`, or composing the complete runtime filter is a complete, local, unambiguous replacement. Otherwise set `suggested-code-omission-reason`. Outcome selection follows DO: `completed`, `no-knowledge`, `not-applicable`, `partial`, or `failed`. From 2c45021cb37d6b60f33ebb945fa88e4ac71462b2 Mon Sep 17 00:00:00 2001 From: Djordje Cenic Date: Tue, 15 Sep 2026 13:16:53 +0200 Subject: [PATCH 11/19] Add security knowledge: validate unauthenticated endpoint responses New remedial article for spotting when AL calls an endpoint that does not authenticate itself to the client (bare HttpClient.Get, blank SOAP SecretText, post-DisableHttpsCheck HTTP) and requires the response to be size-, schema-, and request/response-integrity-validated before it is trusted. Includes the BC-specific false-positive clarifications (platform buffers the full body, so an in-AL size check after buffering is correct; no DNS-rebinding/bounded-read demand; HTTPS not always enforceable) plus good/bad AL samples. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- ...unauthenticated-response-before-use.bad.al | 23 ++++++++++ ...nauthenticated-response-before-use.good.al | 46 +++++++++++++++++++ ...ate-unauthenticated-response-before-use.md | 22 +++++++++ 3 files changed, 91 insertions(+) create mode 100644 microsoft/knowledge/security/validate-unauthenticated-response-before-use.bad.al create mode 100644 microsoft/knowledge/security/validate-unauthenticated-response-before-use.good.al create mode 100644 microsoft/knowledge/security/validate-unauthenticated-response-before-use.md diff --git a/microsoft/knowledge/security/validate-unauthenticated-response-before-use.bad.al b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.bad.al new file mode 100644 index 0000000..ee0d25e --- /dev/null +++ b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.bad.al @@ -0,0 +1,23 @@ +codeunit 50541 "Sec Sample UnauthResp Bad" +{ + procedure IsVatNumberValid(RequestedCountryCode: Text; RequestedVatNumber: Text): Boolean + var + HttpClient: HttpClient; + Response: HttpResponseMessage; + JsonResponse: JsonObject; + JsonToken: JsonToken; + Content: Text; + begin + // Anti-pattern: the endpoint is unauthenticated, yet the response is trusted with no + // size cap, no schema check, and no request-to-response integrity check. + HttpClient.Get('http://vat-service.example/check?cc=' + RequestedCountryCode + '&vat=' + RequestedVatNumber, Response); + Response.Content().ReadAs(Content); + JsonResponse.ReadFrom(Content); + + // Trusts valid=true for ANY input: a spoofed or MITM response that omits the echoed + // countryCode/vatNumber is accepted as valid for whatever number was requested. + if JsonResponse.Get('valid', JsonToken) then + exit(JsonToken.AsValue().AsBoolean()); + exit(false); + end; +} diff --git a/microsoft/knowledge/security/validate-unauthenticated-response-before-use.good.al b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.good.al new file mode 100644 index 0000000..2c3e437 --- /dev/null +++ b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.good.al @@ -0,0 +1,46 @@ +codeunit 50540 "Sec Sample UnauthResp Good" +{ + // The public VAT validation service does not authenticate itself to us (no OAuth, no + // certificate, plain HTTP), so its response must be validated before it is trusted. + procedure IsVatNumberValid(RequestedCountryCode: Text; RequestedVatNumber: Text): Boolean + var + HttpClient: HttpClient; + Response: HttpResponseMessage; + JsonResponse: JsonObject; + JsonToken: JsonToken; + Content: Text; + ResponseCountryCode: Text; + ResponseVatNumber: Text; + begin + HttpClient.Get('http://vat-service.example/check?cc=' + RequestedCountryCode + '&vat=' + RequestedVatNumber, Response); + if not Response.IsSuccessStatusCode() then + exit(false); + + Response.Content().ReadAs(Content); + + // 1) Size cap - the platform already buffered the whole body; reject abnormally large payloads. + if StrLen(Content) > 4096 then + Error('The VAT validation response exceeded the maximum allowed size and was rejected.'); + + // 2) Schema - require the expected scalar fields, not just a truthy flag. + if not JsonResponse.ReadFrom(Content) then + Error('The VAT validation response was not in the expected format and was rejected.'); + if not JsonResponse.Get('countryCode', JsonToken) then + Error('The VAT validation response did not include the requested identifiers and was rejected.'); + ResponseCountryCode := JsonToken.AsValue().AsText(); + if not JsonResponse.Get('vatNumber', JsonToken) then + Error('The VAT validation response did not include the requested identifiers and was rejected.'); + ResponseVatNumber := JsonToken.AsValue().AsText(); + + // 3) Integrity - the echoed identifiers must match the request, so a valid=true payload + // with the identifiers stripped cannot be accepted for an arbitrary VAT number. + if (UpperCase(ResponseCountryCode) <> UpperCase(RequestedCountryCode)) or + (UpperCase(ResponseVatNumber) <> UpperCase(RequestedVatNumber)) + then + Error('The VAT validation response did not match the requested identifiers and was rejected.'); + + if not JsonResponse.Get('valid', JsonToken) then + exit(false); + exit(JsonToken.AsValue().AsBoolean()); + end; +} diff --git a/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md new file mode 100644 index 0000000..2e99e2c --- /dev/null +++ b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md @@ -0,0 +1,22 @@ +--- +bc-version: [all] +domain: security +keywords: [unauthenticated, ssrf, httpclient, soap, response-validation, integrity, size-limit, disablehttpscheck, temp-blob, vies] +technologies: [al] +countries: [w1] +application-area: [all] +--- + +# Validate responses from unauthenticated endpoints before trusting them + +## Description + +When AL calls an external endpoint that does not authenticate *itself* to the client, the response is fully attacker-influenceable — cleartext MITM, a spoofed or compromised host, DNS/redirect games, or simply a misbehaving public service. Recognizing that a call is unauthenticated is the first review step, and the signals are BC-specific: a bare `HttpClient.Get`/`Post` with no `Authorization` header, no acquired OAuth token, and no client certificate; a SOAP request whose credentials are blank, such as `SOAP Web Service Request Mgt.SetGlobals(..., '', BlankSecretText)`; or any request issued after `DisableHttpsCheck()` over plain HTTP (for example the EU VIES VAT service, whose default endpoint is `http://`). Because the BC platform HTTP stack buffers the entire response body before AL is handed the stream or `Temp Blob`, the whole payload is already in memory by the time AL parses it — so the response must be range- and shape-checked in AL *before* any of it is written to tax, VAT, customer, or vendor tables. + +## Best Practice + +Before parsing or trusting a response from an unauthenticated endpoint: (1) enforce a maximum size — reject when the buffered `Temp Blob` length or `Content-Length` exceeds a small cap sized to the expected payload; (2) validate the schema/shape — require the specific scalar nodes you expect, not merely "the body contains a truthy flag"; (3) enforce request-to-response integrity — when the protocol echoes the identifiers you queried (VIES echoes `countryCode`/`vatNumber`; a public-IP service echoes an IP string), require them to be present and to match the request, so a response carrying only `valid=true` cannot be accepted for an arbitrary input; (4) on rejection raise an `Error` and record a security audit via `Audit Log.LogAuditMessage(...)` plus telemetry. See sample: `validate-unauthenticated-response-before-use.good.al`. For validating the outbound target/host, see `validate-user-configurable-urls.md`; for authenticating outbound calls, see `prefer-oauth2-over-api-keys-for-external-http-calls.md`. + +## Anti Pattern + +Feeding the parsed response straight into business logic — load the XML/JSON, read a `valid` flag or an IP-shaped substring, then `Customer.Modify()` — trusting it purely because the HTTP call returned 2xx, with no size, shape, or echoed-identifier check. Reviewers should flag an unauthenticated outbound call (no `Authorization`/OAuth/cert, blank SOAP `SecretText`, or a request after `DisableHttpsCheck`) whose response is parsed and persisted without a preceding size cap, schema check, and request-to-response integrity check. Do NOT, however, demand a streaming or bounded read that aborts the transfer mid-download, nor a resolved-IP/DNS-rebinding check: the platform buffers the full body before AL sees it and AL has no connection-time or DNS hook, so an in-AL size check necessarily runs after buffering and host-rebinding defense belongs to the platform egress layer — raising those is a false positive. HTTPS is likewise not always enforceable (VIES is HTTP by design); the mitigation there is response validation, not scheme enforcement. See sample: `validate-unauthenticated-response-before-use.bad.al`. From 58b3be23abee90269adbe434d8ae395ca8e78a30 Mon Sep 17 00:00:00 2001 From: Djordje Cenic Date: Tue, 15 Sep 2026 13:19:37 +0200 Subject: [PATCH 12/19] Name the three required checks explicitly: response size, schema compliance, content integrity Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../security/validate-unauthenticated-response-before-use.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md index 2e99e2c..27db51f 100644 --- a/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md +++ b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md @@ -11,11 +11,11 @@ application-area: [all] ## Description -When AL calls an external endpoint that does not authenticate *itself* to the client, the response is fully attacker-influenceable — cleartext MITM, a spoofed or compromised host, DNS/redirect games, or simply a misbehaving public service. Recognizing that a call is unauthenticated is the first review step, and the signals are BC-specific: a bare `HttpClient.Get`/`Post` with no `Authorization` header, no acquired OAuth token, and no client certificate; a SOAP request whose credentials are blank, such as `SOAP Web Service Request Mgt.SetGlobals(..., '', BlankSecretText)`; or any request issued after `DisableHttpsCheck()` over plain HTTP (for example the EU VIES VAT service, whose default endpoint is `http://`). Because the BC platform HTTP stack buffers the entire response body before AL is handed the stream or `Temp Blob`, the whole payload is already in memory by the time AL parses it — so the response must be range- and shape-checked in AL *before* any of it is written to tax, VAT, customer, or vendor tables. +When AL calls an external endpoint that does not authenticate *itself* to the client, the response is fully attacker-influenceable — cleartext MITM, a spoofed or compromised host, DNS/redirect games, or simply a misbehaving public service. Recognizing that a call is unauthenticated is the first review step, and the signals are BC-specific: a bare `HttpClient.Get`/`Post` with no `Authorization` header, no acquired OAuth token, and no client certificate; a SOAP request whose credentials are blank, such as `SOAP Web Service Request Mgt.SetGlobals(..., '', BlankSecretText)`; or any request issued after `DisableHttpsCheck()` over plain HTTP (for example the EU VIES VAT service, whose default endpoint is `http://`). Because the BC platform HTTP stack buffers the entire response body before AL is handed the stream or `Temp Blob`, the whole payload is already in memory by the time AL parses it — so the response must pass three checks in AL — **response size**, **schema compliance**, and **content integrity** — *before* any of it is written to tax, VAT, customer, or vendor tables. ## Best Practice -Before parsing or trusting a response from an unauthenticated endpoint: (1) enforce a maximum size — reject when the buffered `Temp Blob` length or `Content-Length` exceeds a small cap sized to the expected payload; (2) validate the schema/shape — require the specific scalar nodes you expect, not merely "the body contains a truthy flag"; (3) enforce request-to-response integrity — when the protocol echoes the identifiers you queried (VIES echoes `countryCode`/`vatNumber`; a public-IP service echoes an IP string), require them to be present and to match the request, so a response carrying only `valid=true` cannot be accepted for an arbitrary input; (4) on rejection raise an `Error` and record a security audit via `Audit Log.LogAuditMessage(...)` plus telemetry. See sample: `validate-unauthenticated-response-before-use.good.al`. For validating the outbound target/host, see `validate-user-configurable-urls.md`; for authenticating outbound calls, see `prefer-oauth2-over-api-keys-for-external-http-calls.md`. +Before parsing or trusting a response from an unauthenticated endpoint, apply all three of these checks before the payload reaches business logic: (1) **Response size** — reject when the buffered `Temp Blob` length or `Content-Length` exceeds a small cap sized to the expected payload; (2) **Schema compliance** — require the specific scalar nodes/fields you expect in the expected shape, not merely "the body contains a truthy flag"; (3) **Content integrity** — when the protocol echoes the identifiers you queried (VIES echoes `countryCode`/`vatNumber`; a public-IP service echoes an IP string), require them to be present and to match the request, so a response carrying only `valid=true` cannot be accepted for an arbitrary input. On any failing check, raise an `Error` and record a security audit via `Audit Log.LogAuditMessage(...)` plus telemetry. See sample: `validate-unauthenticated-response-before-use.good.al`. For validating the outbound target/host, see `validate-user-configurable-urls.md`; for authenticating outbound calls, see `prefer-oauth2-over-api-keys-for-external-http-calls.md`. ## Anti Pattern From 5bda05492752a5954b282dbce5fef8782b8ab474 Mon Sep 17 00:00:00 2001 From: Djordje Cenic Date: Tue, 15 Sep 2026 13:48:59 +0200 Subject: [PATCH 13/19] Link samples using the READ markdown-link convention Use [\slug.good.al\](slug.good.al) form so tools/Knowledge-Retrieval.ps1 Assert-SampleLink validation passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../security/validate-unauthenticated-response-before-use.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md index 27db51f..ca09d56 100644 --- a/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md +++ b/microsoft/knowledge/security/validate-unauthenticated-response-before-use.md @@ -15,8 +15,8 @@ When AL calls an external endpoint that does not authenticate *itself* to the cl ## Best Practice -Before parsing or trusting a response from an unauthenticated endpoint, apply all three of these checks before the payload reaches business logic: (1) **Response size** — reject when the buffered `Temp Blob` length or `Content-Length` exceeds a small cap sized to the expected payload; (2) **Schema compliance** — require the specific scalar nodes/fields you expect in the expected shape, not merely "the body contains a truthy flag"; (3) **Content integrity** — when the protocol echoes the identifiers you queried (VIES echoes `countryCode`/`vatNumber`; a public-IP service echoes an IP string), require them to be present and to match the request, so a response carrying only `valid=true` cannot be accepted for an arbitrary input. On any failing check, raise an `Error` and record a security audit via `Audit Log.LogAuditMessage(...)` plus telemetry. See sample: `validate-unauthenticated-response-before-use.good.al`. For validating the outbound target/host, see `validate-user-configurable-urls.md`; for authenticating outbound calls, see `prefer-oauth2-over-api-keys-for-external-http-calls.md`. +Before parsing or trusting a response from an unauthenticated endpoint, apply all three of these checks before the payload reaches business logic: (1) **Response size** — reject when the buffered `Temp Blob` length or `Content-Length` exceeds a small cap sized to the expected payload; (2) **Schema compliance** — require the specific scalar nodes/fields you expect in the expected shape, not merely "the body contains a truthy flag"; (3) **Content integrity** — when the protocol echoes the identifiers you queried (VIES echoes `countryCode`/`vatNumber`; a public-IP service echoes an IP string), require them to be present and to match the request, so a response carrying only `valid=true` cannot be accepted for an arbitrary input. On any failing check, raise an `Error` and record a security audit via `Audit Log.LogAuditMessage(...)` plus telemetry. See sample: [`validate-unauthenticated-response-before-use.good.al`](validate-unauthenticated-response-before-use.good.al). For validating the outbound target/host, see `validate-user-configurable-urls.md`; for authenticating outbound calls, see `prefer-oauth2-over-api-keys-for-external-http-calls.md`. ## Anti Pattern -Feeding the parsed response straight into business logic — load the XML/JSON, read a `valid` flag or an IP-shaped substring, then `Customer.Modify()` — trusting it purely because the HTTP call returned 2xx, with no size, shape, or echoed-identifier check. Reviewers should flag an unauthenticated outbound call (no `Authorization`/OAuth/cert, blank SOAP `SecretText`, or a request after `DisableHttpsCheck`) whose response is parsed and persisted without a preceding size cap, schema check, and request-to-response integrity check. Do NOT, however, demand a streaming or bounded read that aborts the transfer mid-download, nor a resolved-IP/DNS-rebinding check: the platform buffers the full body before AL sees it and AL has no connection-time or DNS hook, so an in-AL size check necessarily runs after buffering and host-rebinding defense belongs to the platform egress layer — raising those is a false positive. HTTPS is likewise not always enforceable (VIES is HTTP by design); the mitigation there is response validation, not scheme enforcement. See sample: `validate-unauthenticated-response-before-use.bad.al`. +Feeding the parsed response straight into business logic — load the XML/JSON, read a `valid` flag or an IP-shaped substring, then `Customer.Modify()` — trusting it purely because the HTTP call returned 2xx, with no size, shape, or echoed-identifier check. Reviewers should flag an unauthenticated outbound call (no `Authorization`/OAuth/cert, blank SOAP `SecretText`, or a request after `DisableHttpsCheck`) whose response is parsed and persisted without a preceding size cap, schema check, and request-to-response integrity check. Do NOT, however, demand a streaming or bounded read that aborts the transfer mid-download, nor a resolved-IP/DNS-rebinding check: the platform buffers the full body before AL sees it and AL has no connection-time or DNS hook, so an in-AL size check necessarily runs after buffering and host-rebinding defense belongs to the platform egress layer — raising those is a false positive. HTTPS is likewise not always enforceable (VIES is HTTP by design); the mitigation there is response validation, not scheme enforcement. See sample: [`validate-unauthenticated-response-before-use.bad.al`](validate-unauthenticated-response-before-use.bad.al). From 450e5965e180cb5c0fa9c69e4375b081ff43de01 Mon Sep 17 00:00:00 2001 From: Jesper Schulz-Wedde Date: Thu, 17 Sep 2026 13:34:22 +0200 Subject: [PATCH 14/19] Add BCQuality logo and brand assets (#190) * Add BCQuality logo and brand assets Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Remove brand assets link from README Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Use horizontal logo banner in README Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Enlarge banner wordmark Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Jesper Schulz-Wedde Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 4 ++++ docs/README.md | 5 +++++ docs/assets/bcq-logo.svg | 26 ++++++++++++++++++++++++ docs/assets/bcq-mark.svg | 22 ++++++++++++++++++++ docs/brand-assets.md | 43 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 100 insertions(+) create mode 100644 docs/assets/bcq-logo.svg create mode 100644 docs/assets/bcq-mark.svg create mode 100644 docs/brand-assets.md diff --git a/README.md b/README.md index e9e18d5..147a84c 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,7 @@ +

+ BCQuality logo +

+ # BCQuality Quality skills and knowledge that help AI tools make better Business Central diff --git a/docs/README.md b/docs/README.md index d9ca27b..7210cb7 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,3 +1,7 @@ +

+ BCQuality mark +

+ # BCQuality documentation **New to BCQuality? Start with the [quick start](../README.md#quick-start).** @@ -32,3 +36,4 @@ prerequisites for using the plugin. | [DO](../skills/do.md) | Action-skill format and structured output contract. | | [WRITE](../skills/write.md) | Knowledge-authoring rules. | | [Review evaluation](../evaluation/README.md) | Sample conventions, fixture preparation, and scoring. | +| [Brand assets](brand-assets.md) | BCQ logo files and usage guidance. | diff --git a/docs/assets/bcq-logo.svg b/docs/assets/bcq-logo.svg new file mode 100644 index 0000000..0c7aae5 --- /dev/null +++ b/docs/assets/bcq-logo.svg @@ -0,0 +1,26 @@ + + BCQuality banner logo + A blue letter Q crossed by a teal quality check beside the BCQuality wordmark. + + + + + + + + + + + + + + + + + + + + + + BCQuality + diff --git a/docs/assets/bcq-mark.svg b/docs/assets/bcq-mark.svg new file mode 100644 index 0000000..0b6c00e --- /dev/null +++ b/docs/assets/bcq-mark.svg @@ -0,0 +1,22 @@ + + BCQuality mark + A blue letter Q crossed by a teal quality check. + + + + + + + + + + + + + + + + + + + diff --git a/docs/brand-assets.md b/docs/brand-assets.md new file mode 100644 index 0000000..e578fcd --- /dev/null +++ b/docs/brand-assets.md @@ -0,0 +1,43 @@ +# BCQ brand assets + +

+ BCQuality logo +

+ +The BCQuality mark combines a **Q** with a check to represent review, +confidence, and quality. The vector artwork follows the supplied blue-to-teal +concept and connects the mark to the split-color BCQuality wordmark. A true +circular ring and square-ended 45-degree bars keep the check and Q tail aligned +at exact right angles; the check's vertical cut and the tail's horizontal cut +match the source concept. + +## Available artwork + +| Asset | Best use | +| --- | --- | +| [`bcq-logo.svg`](assets/bcq-logo.svg) | Horizontal banner with the mark and wordmark. | +| [`bcq-mark.svg`](assets/bcq-mark.svg) | Symbol without the wordmark. | + +The SVG files can be scaled without losing quality. + +## Color palette + +The artwork uses the dominant colors sampled from the supplied Business Central +logo reference. + +| Color | Hex | +| --- | --- | +| Deep blue | `#086194` | +| Blue | `#138EB7` | +| Cyan | `#17ABCB` | +| Aqua | `#2CD2CD` | +| Mint | `#57E6CB` | +| Light mint | `#9BF2C8` | + +## Usage + +- Prefer the full logo when at least 320 pixels of horizontal space is + available; use the mark at smaller sizes. +- Preserve the artwork's proportions, colors, and orientation. +- Use `BCQuality logo` as alternative text unless nearby text already names the + project, in which case the image can be decorative. From 2b96f5226d469542646d73e010c4099da46995a5 Mon Sep 17 00:00:00 2001 From: Jesper Schulz-Wedde Date: Thu, 17 Sep 2026 13:35:46 +0200 Subject: [PATCH 15/19] Update logo width and remove project title Reduced logo width in README and removed title. --- README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/README.md b/README.md index 147a84c..bdf10f3 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,7 @@

- BCQuality logo + BCQuality logo

-# BCQuality - Quality skills and knowledge that help AI tools make better Business Central development decisions: catch BC-specific defects, avoid misleading advice, and explain findings with references you can read. From d209cd0f73daadb1ca3b7fc9e476e6b1453c29a9 Mon Sep 17 00:00:00 2001 From: Jesper Schulz-Wedde Date: Thu, 17 Sep 2026 13:38:04 +0200 Subject: [PATCH 16/19] Left-align README logo (#191) * Add BCQuality logo and brand assets Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Remove brand assets link from README Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Use horizontal logo banner in README Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Enlarge banner wordmark Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Jesper Schulz-Wedde Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index bdf10f3..3f66a4e 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -

+

BCQuality logo

From b91443beec785606e08274dea3f402a99aaec7bd Mon Sep 17 00:00:00 2001 From: Jesper Schulz-Wedde Date: Thu, 17 Sep 2026 13:38:34 +0200 Subject: [PATCH 17/19] Update logo width in README Reduced logo width in README from 500 to 300 pixels. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3f66a4e..daea405 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@

- BCQuality logo + BCQuality logo

Quality skills and knowledge that help AI tools make better Business Central From 523fc88f877a5151d6a3a15ef52106fa8d886f90 Mon Sep 17 00:00:00 2001 From: Jesper Schulz-Wedde Date: Fri, 18 Sep 2026 09:05:29 +0200 Subject: [PATCH 18/19] Remove logo branding (#194) Co-authored-by: Jesper Schulz-Wedde Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 4 ---- docs/README.md | 5 ----- docs/assets/bcq-logo.svg | 26 ------------------------ docs/assets/bcq-mark.svg | 22 -------------------- docs/brand-assets.md | 43 ---------------------------------------- 5 files changed, 100 deletions(-) delete mode 100644 docs/assets/bcq-logo.svg delete mode 100644 docs/assets/bcq-mark.svg delete mode 100644 docs/brand-assets.md diff --git a/README.md b/README.md index daea405..d45a5d1 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,3 @@ -

- BCQuality logo -

- Quality skills and knowledge that help AI tools make better Business Central development decisions: catch BC-specific defects, avoid misleading advice, and explain findings with references you can read. diff --git a/docs/README.md b/docs/README.md index 7210cb7..d9ca27b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,7 +1,3 @@ -

- BCQuality mark -

- # BCQuality documentation **New to BCQuality? Start with the [quick start](../README.md#quick-start).** @@ -36,4 +32,3 @@ prerequisites for using the plugin. | [DO](../skills/do.md) | Action-skill format and structured output contract. | | [WRITE](../skills/write.md) | Knowledge-authoring rules. | | [Review evaluation](../evaluation/README.md) | Sample conventions, fixture preparation, and scoring. | -| [Brand assets](brand-assets.md) | BCQ logo files and usage guidance. | diff --git a/docs/assets/bcq-logo.svg b/docs/assets/bcq-logo.svg deleted file mode 100644 index 0c7aae5..0000000 --- a/docs/assets/bcq-logo.svg +++ /dev/null @@ -1,26 +0,0 @@ - - BCQuality banner logo - A blue letter Q crossed by a teal quality check beside the BCQuality wordmark. - - - - - - - - - - - - - - - - - - - - - - BCQuality - diff --git a/docs/assets/bcq-mark.svg b/docs/assets/bcq-mark.svg deleted file mode 100644 index 0b6c00e..0000000 --- a/docs/assets/bcq-mark.svg +++ /dev/null @@ -1,22 +0,0 @@ - - BCQuality mark - A blue letter Q crossed by a teal quality check. - - - - - - - - - - - - - - - - - - - diff --git a/docs/brand-assets.md b/docs/brand-assets.md deleted file mode 100644 index e578fcd..0000000 --- a/docs/brand-assets.md +++ /dev/null @@ -1,43 +0,0 @@ -# BCQ brand assets - -

- BCQuality logo -

- -The BCQuality mark combines a **Q** with a check to represent review, -confidence, and quality. The vector artwork follows the supplied blue-to-teal -concept and connects the mark to the split-color BCQuality wordmark. A true -circular ring and square-ended 45-degree bars keep the check and Q tail aligned -at exact right angles; the check's vertical cut and the tail's horizontal cut -match the source concept. - -## Available artwork - -| Asset | Best use | -| --- | --- | -| [`bcq-logo.svg`](assets/bcq-logo.svg) | Horizontal banner with the mark and wordmark. | -| [`bcq-mark.svg`](assets/bcq-mark.svg) | Symbol without the wordmark. | - -The SVG files can be scaled without losing quality. - -## Color palette - -The artwork uses the dominant colors sampled from the supplied Business Central -logo reference. - -| Color | Hex | -| --- | --- | -| Deep blue | `#086194` | -| Blue | `#138EB7` | -| Cyan | `#17ABCB` | -| Aqua | `#2CD2CD` | -| Mint | `#57E6CB` | -| Light mint | `#9BF2C8` | - -## Usage - -- Prefer the full logo when at least 320 pixels of horizontal space is - available; use the mark at smaller sizes. -- Preserve the artwork's proportions, colors, and orientation. -- Use `BCQuality logo` as alternative text unless nearby text already names the - project, in which case the image can be decorative. From 38ad6b8810440e9e78733e4c75b06fb8614db729 Mon Sep 17 00:00:00 2001 From: Jesper Schulz-Wedde Date: Fri, 18 Sep 2026 09:07:57 +0200 Subject: [PATCH 19/19] Add title and description to README --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index d45a5d1..8bf1cc7 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,5 @@ +# BC Quality - Don’t teach one agent. Teach the ecosystem. 🤝 + Quality skills and knowledge that help AI tools make better Business Central development decisions: catch BC-specific defects, avoid misleading advice, and explain findings with references you can read.