mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-10-05 14:46:55 +01:00
Add knowledge-backed AL development
Add read-only planning and repository-changing development skills so BCQuality knowledge can guide features, bug fixes, refactors, upgrades, and maintenance before the existing AL review gate runs. Track Microsoft Learn ingestion and add development and BCApps-shaped guidance evaluation fixtures. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 638b66d2-9f06-4f60-8781-808709e1485c
This commit is contained in:
Jesper Schulz-Wedde
parent
1a5afdc0eb
commit
56b80e6dcf
50 changed files with 11589 additions and 79 deletions
|
|
@ -54,3 +54,50 @@ This credential-free check proves every selected leaf maps to a same-named knowl
|
|||
For a single combined stress-test result, use `-ResultsPath` instead.
|
||||
|
||||
The committed gate requires full expected recall, the exact convention-derived article ID, and no findings on clean controls.
|
||||
|
||||
## AL development evaluation
|
||||
|
||||
`development-fixtures.json` defines end-to-end development requests rather than
|
||||
prewritten good/bad snippets. Each case declares its execution mode, the
|
||||
Business Central capabilities it exercises, the knowledge that should shape
|
||||
the implementation, acceptance criteria, and the real checks an external
|
||||
runner must perform.
|
||||
|
||||
Validate the fixture and capability manifests and prepare opaque requests:
|
||||
|
||||
```powershell
|
||||
pwsh ./tools/Test-DevelopmentFixtures.ps1 -Root . -PrepareDirectory ./.development-evaluation
|
||||
```
|
||||
|
||||
Each request runs `al-development` in a fresh writable AL repository. Cases
|
||||
may exercise feature, bug, refactor, upgrade, or maintenance mode.
|
||||
The runner compiles the generated project, runs its tests, invokes the review
|
||||
quality gate, and stores the resulting implementation report using the opaque
|
||||
`caseId` from its request, for example `result-case-a1b2c3d4.json`. It keeps the
|
||||
generated repository available at the wrapper's `workspaceRoot` so scoring can
|
||||
verify reported changed paths. Score all results with:
|
||||
|
||||
```powershell
|
||||
pwsh ./tools/Test-DevelopmentFixtures.ps1 -Root . -ResultsDirectory ./.development-evaluation
|
||||
```
|
||||
|
||||
The initial fixtures cover setup-backed master data, document header/line
|
||||
workflows, versioned API integrations, and surgical diagnosis and repair of a
|
||||
batch-processing bug. The broader capability roadmap lives in
|
||||
`coverage/development-capabilities.json`.
|
||||
|
||||
### Read-only plan guidance
|
||||
|
||||
`development-guidance-fixtures.json` evaluates the planning interface used by
|
||||
specialized orchestrators. It supplies an existing development plan and expects
|
||||
a referenced set of implementation constraints without any target-repository
|
||||
changes.
|
||||
|
||||
```powershell
|
||||
pwsh ./tools/Test-DevelopmentGuidanceFixtures.ps1 -Root . -PrepareDirectory ./.development-guidance-evaluation
|
||||
```
|
||||
|
||||
An external runner stores `result-<case-id>.json` beside the generated request
|
||||
and retains the clean fixture repository at `workspaceRoot`. Score the result
|
||||
with `-ResultsDirectory`; the scorer verifies knowledge recall and precision
|
||||
and fails if the planning pass changed the repository.
|
||||
|
|
|
|||
188
evaluation/development-fixtures.json
Normal file
188
evaluation/development-fixtures.json
Normal file
|
|
@ -0,0 +1,188 @@
|
|||
{
|
||||
"version": 1,
|
||||
"skill": "microsoft/skills/development/al-development.md",
|
||||
"minimumKnowledgeRecall": 1.0,
|
||||
"minimumKnowledgePrecision": 0.5,
|
||||
"cases": [
|
||||
{
|
||||
"id": "setup-backed-master-data",
|
||||
"title": "Build setup-backed loyalty member master data",
|
||||
"capabilities": [
|
||||
"setup-and-master-data"
|
||||
],
|
||||
"expectedKind": "feature",
|
||||
"development-request": {
|
||||
"kind": "feature",
|
||||
"description": "Add a Loyalty Member feature to an existing Business Central AL app. Administrators configure the member number series on a singleton setup card. Users create members through list and card pages, the table assigns numbers, and blocked members cannot be selected by consuming records. Include least-privilege permission sets and automated tests.",
|
||||
"acceptance-criteria": [
|
||||
"The setup is a blank-key singleton surfaced by a Card page.",
|
||||
"Member numbers use the current No. Series codeunit and support manual numbers according to setup.",
|
||||
"Blocked validation occurs where a member is consumed, not only on the member table.",
|
||||
"The feature includes assignable least-privilege permissions and automated tests."
|
||||
]
|
||||
},
|
||||
"context": {
|
||||
"technologies": [
|
||||
"al"
|
||||
],
|
||||
"countries": [
|
||||
"w1"
|
||||
],
|
||||
"application-area": [
|
||||
"all"
|
||||
]
|
||||
},
|
||||
"requiredKnowledge": [
|
||||
"microsoft/knowledge/data-modeling/setup-table-is-a-singleton.md",
|
||||
"microsoft/knowledge/data-modeling/master-table-no-from-number-series-in-oninsert.md",
|
||||
"microsoft/knowledge/data-modeling/check-blocked-in-referencing-code-not-in-master.md",
|
||||
"microsoft/knowledge/security/permission-set-avoid-wildcard-grants.md",
|
||||
"microsoft/knowledge/testing/use-library-codeunits-for-test-fixtures.md"
|
||||
],
|
||||
"optionalKnowledge": [
|
||||
"microsoft/knowledge/data-modeling/use-no-series-codeunit-not-noseriesmanagement.md",
|
||||
"microsoft/knowledge/security/compose-permission-sets-with-included-sets.md",
|
||||
"microsoft/knowledge/style/applicationarea-required-on-page-controls.md",
|
||||
"microsoft/knowledge/style/tooltip-required-on-page-fields.md",
|
||||
"microsoft/knowledge/ui/showmandatory-on-code-required-page-fields.md"
|
||||
],
|
||||
"requiredChecks": [
|
||||
"compile",
|
||||
"tests",
|
||||
"review"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "document-header-and-lines",
|
||||
"title": "Build a document header and lines workflow",
|
||||
"capabilities": [
|
||||
"document-workflows"
|
||||
],
|
||||
"expectedKind": "feature",
|
||||
"development-request": {
|
||||
"kind": "feature",
|
||||
"description": "Implement a Service Quote feature with a header, lines, document page, number series, posting and document dates, calculated totals, and tests. Line edits must refresh the total shown on the header. Structure initialization so API, UI, and test creation paths behave consistently.",
|
||||
"acceptance-criteria": [
|
||||
"The header assigns its number before InitRecord establishes document defaults.",
|
||||
"The document page links lines correctly and refreshes parent totals after edits.",
|
||||
"Tests exercise creation outside the UI as well as the document-page behavior.",
|
||||
"The implementation contains no obsolete NoSeriesManagement dependency."
|
||||
]
|
||||
},
|
||||
"context": {
|
||||
"technologies": [
|
||||
"al"
|
||||
],
|
||||
"countries": [
|
||||
"w1"
|
||||
],
|
||||
"application-area": [
|
||||
"service"
|
||||
]
|
||||
},
|
||||
"requiredKnowledge": [
|
||||
"microsoft/knowledge/data-modeling/initialize-document-defaults-in-initrecord.md",
|
||||
"microsoft/knowledge/data-modeling/use-no-series-codeunit-not-noseriesmanagement.md",
|
||||
"microsoft/knowledge/ui/updatepropagation-both-refreshes-main-page.md",
|
||||
"microsoft/knowledge/testing/use-library-codeunits-for-test-fixtures.md"
|
||||
],
|
||||
"optionalKnowledge": [
|
||||
"microsoft/knowledge/events/publish-thin-onbefore-onafter-integration-events.md",
|
||||
"microsoft/knowledge/style/applicationarea-required-on-page-controls.md",
|
||||
"microsoft/knowledge/style/tooltip-required-on-page-fields.md"
|
||||
],
|
||||
"requiredChecks": [
|
||||
"compile",
|
||||
"tests",
|
||||
"review"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "versioned-master-data-api",
|
||||
"title": "Expose master data through a versioned API",
|
||||
"capabilities": [
|
||||
"api-integrations"
|
||||
],
|
||||
"expectedKind": "feature",
|
||||
"development-request": {
|
||||
"kind": "auto",
|
||||
"plan": "Expose Loyalty Member master data through a Business Central API page. Use a stable v1.0 contract, address records by SystemId, support insert and update, use conventional entity naming, and include permissions and automated API-oriented tests.",
|
||||
"acceptance-criteria": [
|
||||
"The API declares all routing properties and a stable APIVersion.",
|
||||
"ODataKeyFields uses SystemId and the exposed SystemId field is not editable.",
|
||||
"EntityName is singular, EntitySetName is plural, and both are lower camel case.",
|
||||
"The API is covered by least-privilege permissions and automated tests."
|
||||
]
|
||||
},
|
||||
"context": {
|
||||
"technologies": [
|
||||
"al"
|
||||
],
|
||||
"countries": [
|
||||
"w1"
|
||||
],
|
||||
"application-area": [
|
||||
"all"
|
||||
]
|
||||
},
|
||||
"requiredKnowledge": [
|
||||
"microsoft/knowledge/web-services/set-required-api-page-properties.md",
|
||||
"microsoft/knowledge/web-services/expose-systemid-as-the-api-key.md",
|
||||
"microsoft/knowledge/style/api-page-delayedinsert-true.md",
|
||||
"microsoft/knowledge/style/api-page-entity-naming-singular-plural.md",
|
||||
"microsoft/knowledge/security/permission-set-avoid-wildcard-grants.md"
|
||||
],
|
||||
"optionalKnowledge": [
|
||||
"microsoft/knowledge/style/api-page-camelcase-properties.md",
|
||||
"microsoft/knowledge/web-services/version-apis-by-adding-not-mutating-published-versions.md"
|
||||
],
|
||||
"requiredChecks": [
|
||||
"compile",
|
||||
"tests",
|
||||
"review"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "fix-filtered-batch-processing",
|
||||
"title": "Fix a batch routine that processes only one record",
|
||||
"capabilities": [
|
||||
"bug-diagnosis-and-fix"
|
||||
],
|
||||
"expectedKind": "bug",
|
||||
"development-request": {
|
||||
"kind": "bug",
|
||||
"description": "Users report that an existing filtered batch routine updates only the first matching record. Reproduce the defect, identify why iteration stops, make the smallest safe correction, and add a regression test that selects multiple records and proves every selected record is processed.",
|
||||
"acceptance-criteria": [
|
||||
"The defect is reproduced or demonstrated by a failing regression test before the fix.",
|
||||
"The root cause is corrected without widening the supplied record filters.",
|
||||
"The routine processes every selected record with the appropriate update locking behavior.",
|
||||
"A regression test covers more than one selected record."
|
||||
]
|
||||
},
|
||||
"context": {
|
||||
"technologies": [
|
||||
"al"
|
||||
],
|
||||
"countries": [
|
||||
"w1"
|
||||
],
|
||||
"application-area": [
|
||||
"all"
|
||||
]
|
||||
},
|
||||
"requiredKnowledge": [
|
||||
"microsoft/knowledge/performance/pair-findset-with-next-loop.md",
|
||||
"microsoft/knowledge/testing/use-library-codeunits-for-test-fixtures.md"
|
||||
],
|
||||
"optionalKnowledge": [
|
||||
"microsoft/knowledge/performance/findset-true-applies-updlock-on-read.md",
|
||||
"microsoft/knowledge/performance/pass-var-record-to-preserve-partial-load-enumerator.md"
|
||||
],
|
||||
"requiredChecks": [
|
||||
"compile",
|
||||
"tests",
|
||||
"review"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
50
evaluation/development-guidance-fixtures.json
Normal file
50
evaluation/development-guidance-fixtures.json
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
{
|
||||
"version": 1,
|
||||
"skill": "microsoft/skills/development/al-development-plan.md",
|
||||
"minimumKnowledgeRecall": 1.0,
|
||||
"minimumKnowledgePrecision": 0.5,
|
||||
"cases": [
|
||||
{
|
||||
"id": "bcapps-filtered-batch-bug-plan",
|
||||
"title": "Select guidance for a filtered batch bug fix",
|
||||
"development-plan": {
|
||||
"kind": "bug",
|
||||
"request": "Fix a filtered batch routine that updates only the first matching record.",
|
||||
"root-cause": "The routine calls FindFirst and updates the current record without entering an enumerator loop.",
|
||||
"affected-files": [
|
||||
"src/Batch/UpdateSelectedEntries.Codeunit.al",
|
||||
"test/Batch/UpdateSelectedEntries.Codeunit.al"
|
||||
],
|
||||
"proposed-changes": [
|
||||
"Iterate the supplied filtered record set and update every selected entry.",
|
||||
"Add a regression test with multiple selected entries."
|
||||
],
|
||||
"test-strategy": "Establish a red test where only one of several selected records is updated, then require all selected records to be updated after the fix.",
|
||||
"acceptance-criteria": [
|
||||
"The supplied filters are preserved.",
|
||||
"Every selected record is updated.",
|
||||
"The regression test demonstrates red-to-green behavior."
|
||||
]
|
||||
},
|
||||
"context": {
|
||||
"technologies": [
|
||||
"al"
|
||||
],
|
||||
"countries": [
|
||||
"w1"
|
||||
],
|
||||
"application-area": [
|
||||
"all"
|
||||
]
|
||||
},
|
||||
"requiredKnowledge": [
|
||||
"microsoft/knowledge/performance/pair-findset-with-next-loop.md",
|
||||
"microsoft/knowledge/performance/findset-true-applies-updlock-on-read.md",
|
||||
"microsoft/knowledge/testing/use-library-codeunits-for-test-fixtures.md"
|
||||
],
|
||||
"optionalKnowledge": [
|
||||
"microsoft/knowledge/performance/pass-var-record-to-preserve-partial-load-enumerator.md"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue