Driven by a parity comparison between BCAppsBCQuality PR #27 and
BCAppsCampAIRHack PR #162 on byte-identical content:
| | BCQuality | AIRHack |
|--|--|--|
| Total findings | 6 | 10 |
| Performance | 0 | 4 |
| Security | 0 | 5 |
Standalone runs of al-security-review and al-performance-review against
the SAME diff produced the expected matches (rimd-on-read-only via
inherent-permissions-minimal-grant; redundant-Get via
avoid-redundant-get-when-record-already-loaded). The miss in the live
run is therefore not a knowledge-coverage gap and not a worklist
filtering issue. It is attention dilution inside the al-code-review
super-skill, which the model collapses into one rolled-up generation
pass on real-size PRs.
Changes:
microsoft/skills/review/al-code-review.md
- New 'Execution discipline (mandatory)' subsection in the Action step
that explicitly forbids collapsing leaves into one shared reasoning
pass and requires each sub-skill to walk its Source -> Relevance ->
Worklist -> Action steps as its own iteration before the next leaf
starts.
- Self-review pass is now described as the final, mandatory iteration
with a concrete candidate-category checklist (architecture-level
smells, error-handling gaps, magic constants, privacy/telemetry,
resource lifecycle). Returning zero agent findings on a real-size
diff is explicitly defined as a defect.
microsoft/skills/review/al-{security,performance,privacy,style,
upgrade,ui}-review.md
- Each leaf skill now states that when an unambiguous .good.al
companion exists, findings[].suggested-code should carry the
literal replacement for the source lines. Closes the
one-click-suggestion gap created when BCQ#19 only updated
al-code-review.
microsoft/knowledge/security/case-must-handle-unknown-enum-values.{md,
bad.al,
good.al}
- New article: case over a security-sensitive enum (Authentication
Type, Authorization Mode, Identity Provider, Permission Scope,
Encryption Algorithm) MUST have an else arm. Without it, an unknown
enum value silently falls through and the security context never
initialises. The bad sample is lifted from the SharePoint Graph
helper that triggered the parity finding.
microsoft/knowledge/performance/instream-length-unreliable-for-bc-
streams.{md,bad,good}
- New article: InStream.Length returns 0 / partial for HTTP-response
streams and some file-API streams, breaking size-threshold branching
in upload code. Bad sample is the simple-vs-chunked Graph upload
pattern; good sample materialises into a Temp Blob first.
Companion change: microsoft/BCAppsBCQuality#28 extends the
orchestrator's bootstrap prompt with the same per-iteration execution
discipline and adds a CI warning when a >5-file PR returns zero agent
findings (regression signal).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The previous LLM-generated knowledge files contained factual
hallucinations. The most visible was the claim that `FindFirst` /
`FindLast` "forces a full-table scan" on an unfiltered record - it does
not; those APIs return a single row via the current key.
Other inaccuracies the audit found and fixed:
* `FindSet(true)` was described as "taking a LockTable". The correct
upstream phrasing is that `FindSet(true)` sets
`ReadIsolation::UpdLock` on the read. UpdLock and LockTable are
related but distinct mechanisms.
* The list of production-scale tables had been invented beyond the
upstream source (e.g. "Detailed Cust. Ledg. Entry") without a
citation. The regenerated list matches the ten tables upstream lists
with their P95 row counts.
* `SetLoadFields` guidance had been augmented with an extra mechanism
claim ("the database resolves the filter using the index without
hydrating the value") not present in upstream.
Approach: full regeneration of `microsoft/knowledge/` from the six
upstream BCApps Code Review instruction files, with Microsoft Learn /
the AL language reference as a secondary source. Every claim in every
regenerated file is anchored to a verbatim upstream quote (or a Learn
URL); the audit trail lives in artifacts/trace-<domain>.json on the
session workspace.
The PR #11 transaction/error-handling cluster is preserved verbatim:
* performance/understand-implicit-transaction-boundary.md
* performance/codeunit-run-as-atomic-sub-operation.{md,good.al,bad.al}
* performance/codeunit-run-requires-prior-commit-inside-transaction.{md,good.al,bad.al}
* performance/use-tryfunction-for-error-catching-not-rollback.{md,good.al,bad.al}
* performance/avoid-commit-inside-loops.{md,good.al,bad.al}
* security/commitbehavior-attribute-scopes-explicit-commits.{md,good.al,bad.al}
* testing/transactionmodel-attribute-governs-test-transactions.{md,good.al,bad.al}
These articles already cite Microsoft Learn and were carefully
cross-referenced; the regeneration skips their topics rather than
duplicating them.
File counts after regeneration:
performance 35 .md (5 preserved + 30 new)
privacy 17 .md
security 18 .md (1 preserved + 17 new)
style 33 .md
testing 1 .md (preserved)
ui 19 .md
upgrade 18 .md
Total 141 atomic knowledge files, each strictly one rule. All pass
.github/scripts/validate_frontmatter.py with 0 errors and 0 warnings.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The 35 articles still in their seed form previously carried a banner
reading "Seed article. ... Domain stewards should expand, restructure,
and refine as needed." For a community preview, that phrasing reads as
"TODO left in production" to first-time visitors.
Replace all three banner variants (performance-seeded, security-seeded,
community-ported) with a single positive invitation:
> Contributions welcome — open a PR to refine or extend this article.
Content and structure of the articles are unchanged; only the leading
quote block differs. Articles that had their banner fully stripped in
the earlier triage pass (the showcase-grade ten) are unaffected.
Adds 55 articles (plus 76 code samples) spanning four new domains and
two existing domains, extracted from the internal Business Central
review-agent prompt. Content was filtered against BCQuality's
remedial-knowledge premise: each article encodes BC-specific behaviour,
a CodeCop rule, a platform API semantic, or an anti-false-positive
guideline that a capable LLM would otherwise get wrong.
New domains:
- privacy (11 articles): DataClassification inheritance semantics, the
StrSubstNo-defeats-Error-telemetry-classification pitfall, Privacy
Notice consent for outgoing requests, anti-false-positives for pages
and in-memory data.
- upgrade (11 articles): upgrade-codeunit structure, upgrade-tag
lifecycle and registration, protected DB reads, DataTransfer for
large datasets, InitValue semantics, enum-ordinal preservation,
obsolete-workflow, first-install detection.
- ui (9 articles): caption capitalization by phrase type, tooltip voice,
teaching-tip vs tooltip, tour-tip conventions, character limits,
banned terms, ampersand handling, title punctuation.
- style (11 articles): label-suffix convention, API page naming,
temporary-variable prefix, label properties (Comment/Locked), named
invocations, FieldCaption in user messages, OptionCaption pairing,
Error-parameter passing, `this` keyword, required parentheses, file
naming.
Gaps in existing domains:
- performance (11 articles): production-scale table catalog (no row
counts, per internal-data concern), anti-false-positive for bounded
tables, guard-before-Get ordering, redundant-Get-in-OnAfterGetRecord,
LockTable in read-only helpers, combined ModifyAll passes, writes in
OnAfterGetRecord, SetLoadFields heuristics, temporary-table
regressions, FlowField source-table widening, MaintainSQLIndex
disabling SIFT.
- security (2 articles): environment-specific hardcoded GUIDs,
ValidateTableRelation=false on user input.
Intentionally excluded: specific production P95 row-count numbers
(aggregated internal telemetry); rewritten as categorical guidance on
which tables to treat as production-scale without publishing sizes.
All articles use `bc-version: [all]` (applies to every BC version, per
the new schema sentinel). Validator passes with 0 errors / 0 warnings.
Most of the corpus — FindSet/SetLoadFields/CalcFields patterns, permission
sets, SingleInstance codeunits, DataClassification, IsolatedStorage,
transaction scope, SecretText — describes BC platform behaviour that is
identical across supported versions. The seed [26..28] range on every
file implied a version-specificity the content does not actually have,
and there was no way to express "applies to every version" in the
schema the way [w1] and [all] already do for countries and
application-area.
Extend the v1 schema with a universal sentinel for bc-version, parallel
to the sentinels already defined for the other dimensions:
bc-version: [all] # applies to every BC version
[all] is mutually exclusive with explicit versions. Range shorthand
([26..28]) and explicit lists ([26, 27, 28]) continue to work for files
genuinely tied to a version-gated API or deprecation.
Update read.md (field definition, matching semantics, partial-context
rule), write.md (default to [all], use ranges only with a concrete
reason), README.md (frontmatter example), and the CI validator. All
forty existing knowledge files and the three action skills convert to
[all]; none of the current content is version-gated. Validator passes.
Remove seven knowledge files whose content is generic software-engineering
guidance that a capable LLM already applies without BCQuality present
(HTTPS-only, secret-leakage-in-errors, no-credentials-in-URLs, silent
security-error swallowing, short transaction scope, HTTP timeouts,
StrSubstNo-vs-concatenation). These fail the remedial-knowledge premise
and dilute the signal of the preview corpus.
Strip the "Seed article — domain stewards should expand" banner from ten
files that are ready to showcase (AA0232/AA0233 rules, FindSet read-only
semantics, SetLoadFields ordering and usage, CalcFields-in-loops,
SecretText end-to-end, DataClassification). The banner remains on files
that still need domain-steward refinement.
Add a "What belongs here" section to the README stating the admission
test: a file exists only if a modern LLM would get something wrong or
miss something without it. Gives contributors a concrete yes/no filter
before they open a PR.
The /samples/ top-level tree is replaced with sibling files in each
knowledge-layer folder. An article and its demonstrations now live
side-by-side:
microsoft/knowledge/<domain>/<slug>.md
microsoft/knowledge/<domain>/<slug>.good.al
microsoft/knowledge/<domain>/<slug>.bad.al
Rationale:
- Proximity. An article and its paired samples are one unit; the
filesystem now reflects that.
- Layer ownership. Samples inherit layer precedence for free -- a
/custom/ fork can override an article and its samples atomically,
which the shared /samples/ tree previously made awkward.
- Trivial migration path. Action-skill source globs
(*/knowledge/<domain>/**/*.md) are unchanged; sample discovery is a
sibling-filename lookup.
Changes:
- git mv of all 65 sample files from samples/<domain>/<slug>/{bad,good}.al
to microsoft/knowledge/<domain>/<slug>.{bad,good}.al (history preserved).
- Update See-sample references in all 37 articles that ship samples.
- skills/read.md: replace the no-code-blocks bullet with a pointer to a
new Sample files section that fully specifies the sibling convention,
the kinds (good/bad + forward-compatible), multi-technology rules,
demonstration-only status, and layer-precedence behaviour.
- skills/write.md: update the samples pointer to match.
- README.md: annotate the knowledge tree with the sample sibling shape.
- samples/README.md deleted; content lifted into skills/read.md.
- Both generators (C:\temp\gen_performance_knowledge.py,
C:\temp\gen_security_knowledge.py) updated to emit at the new paths
and to stop writing samples/README.md. Re-running them is idempotent
against the committed layout.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Converts Jesper's existing AL security-review prompt into BCQuality seed
knowledge articles so the al-security-review leaf has a real corpus to
match against. Mirrors the performance seed phase.
Articles under microsoft/knowledge/security/ (16):
- Permission model: follow-least-privilege-in-permission-sets,
use-indirect-permissions-for-elevated-access,
use-inherent-permissions-to-grant-minimal-access
- Secrets: never-hardcode-secrets-in-al,
use-isolated-storage-for-module-and-company-secrets,
prefer-azure-key-vault-for-production-secrets,
use-secrettext-for-credentials, use-secrettext-with-httpclient,
compose-secrets-with-secretstrsubstno,
use-nondebuggable-when-parsing-secrets
- External calls: require-https-for-external-calls,
set-timeouts-for-external-calls, do-not-put-credentials-in-urls
- Error handling: avoid-sensitive-data-in-error-messages,
do-not-swallow-security-errors-silently
- Extensibility: do-not-expose-sensitive-data-in-event-publishers
Paired AL samples under samples/security/<slug>/{bad,good}.al, object
IDs 50200-50231 (no overlap with performance 50100-50140).
Rubber-duck findings addressed:
- HttpClient secret-URI: SetSecretRequestUri is on HttpRequestMessage
(not HttpClient). Rewrote use-secrettext-with-httpclient and its
good sample to use HttpRequestMessage + HttpClient.Send.
- InherentPermissions only grants access to same-extension objects;
the sample now defines its own table 50230 "Sec Sample Lookup" and
grants 'r' on that, not on Database::Customer.
- Reworked compose-secrets-with-secretstrsubstno bad.al away from
Format(SecretText) (unreliable) to a plain Text+StrSubstNo anti-
pattern.
- Moved normative guidance out of Description in three articles
(compose-secrets-..., prefer-azure-key-vault-..., use-inherent-...)
so it sits in Best Practice / Anti Pattern per READ contract.
- Added a companion helper codeunit (50231) to the indirect-permissions
good sample so it actually demonstrates the controlled write path.
- Rebuilt the event-publisher good/bad pair on the same ExportCustomer
scenario so the contrast is the shape of the event signature, not a
different event.
Also: broaden samples/README.md object-ID range note to 50100-50299.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
