mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-08-06 17:36:53 +01:00
Add BCApps citations to Tier 1+2 knowledge files; add 2 new rules
- All 7 existing Tier 1/2 knowledge files now include a BCApps Reference section with concrete source links and observed patterns - New: bcpt-scenarios-must-be-app-specific — PerformanceTest apps must include app-domain BCPT scenarios, not only Microsoft generic samples - New: permission-sets-must-follow-least-privilege — View/Edit/Admin hierarchy with IncludedPermissionSets, mirroring BCApps BusFound pattern - api-page-key-fields-must-be-editable-on-insert clarified: SystemId as ODataKeyField + Editable=false is valid (auto-generated); rule applies to consumer-provided key fields only Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Michael Dieringer
parent
e11c1fd16c
commit
288f64df16
9 changed files with 365 additions and 355 deletions
|
|
@ -0,0 +1,92 @@
|
|||
# CURABIS Testing: BCPT Scenarios Must Be App-Specific
|
||||
|
||||
## Core Rule
|
||||
|
||||
A PerformanceTest app must include BCPT scenario codeunits that exercise the **host app's own business flows** — not only the generic Microsoft scenarios (sales orders, purchase orders, GL entries). Generic scenarios measure BC's baseline performance; app-specific scenarios are the only way to detect performance regressions in the extension's own code.
|
||||
|
||||
## Key Principle
|
||||
|
||||
"A PerformanceTest app that contains only Microsoft's shipped BCPT samples provides no regression signal for the extension it was built to test."
|
||||
|
||||
## What Must Be Included
|
||||
|
||||
For every major business flow in the host app, create a corresponding `BCPT*` codeunit that:
|
||||
|
||||
1. Is a `SingleInstance = true` codeunit
|
||||
2. Implements `"BCPT Test Param. Provider"` interface
|
||||
3. Wraps the key operation in `BCPTTestContext.StartScenario()` / `BCPTTestContext.EndScenario()` blocks
|
||||
4. Sets up all required data in a local `InitTest()` procedure — never depends on hardcoded records
|
||||
|
||||
## Example: Project Management App
|
||||
|
||||
```al
|
||||
codeunit 80100 "BCPT Create Project" implements "BCPT Test Param. Provider"
|
||||
{
|
||||
SingleInstance = true;
|
||||
|
||||
trigger OnRun()
|
||||
begin
|
||||
if not IsInitialized then begin
|
||||
InitTest();
|
||||
IsInitialized := true;
|
||||
end;
|
||||
CreateProject(GlobalBCPTTestContext);
|
||||
end;
|
||||
|
||||
var
|
||||
GlobalBCPTTestContext: Codeunit "BCPT Test Context";
|
||||
IsInitialized: Boolean;
|
||||
|
||||
local procedure InitTest()
|
||||
begin
|
||||
// Set up any required BC configuration
|
||||
end;
|
||||
|
||||
local procedure CreateProject(var BCPTTestContext: Codeunit "BCPT Test Context")
|
||||
begin
|
||||
BCPTTestContext.StartScenario('Create Project Header');
|
||||
// ... create project
|
||||
BCPTTestContext.EndScenario('Create Project Header');
|
||||
BCPTTestContext.UserWait();
|
||||
|
||||
BCPTTestContext.StartScenario('Add Project Task');
|
||||
// ... add task
|
||||
BCPTTestContext.EndScenario('Add Project Task');
|
||||
end;
|
||||
|
||||
procedure GetDefaultParameters(): Text[1000]
|
||||
begin
|
||||
exit('');
|
||||
end;
|
||||
|
||||
procedure ValidateParameters(Parameters: Text[1000])
|
||||
begin
|
||||
end;
|
||||
}
|
||||
```
|
||||
|
||||
## Suggested Scenarios for Project Management Apps
|
||||
|
||||
| Scenario codeunit | What it measures |
|
||||
|---|---|
|
||||
| `BCPT Create Project` | Header + task creation overhead |
|
||||
| `BCPT Post Time Entry` | Time registration and FlowField recalc performance |
|
||||
| `BCPT Open Project List` | Page rendering under load |
|
||||
| `BCPT Open Active Task List` | Filtered list performance |
|
||||
| `BCPT Calculate Project Budget` | Aggregation codeunit performance |
|
||||
|
||||
## Anti-Pattern
|
||||
|
||||
A PerformanceTest app that only contains Microsoft's generic samples:
|
||||
- `BCPTCreateSOWithNLines`
|
||||
- `BCPTOpenCustomerList`
|
||||
- `BCPTPostItemJournal`
|
||||
|
||||
...tests *Business Central*, not *your extension*. A regression in your codeunit will go undetected.
|
||||
|
||||
## BCApps Reference
|
||||
|
||||
The BCPT scenario pattern — `SingleInstance`, `"BCPT Test Param. Provider"`, named `StartScenario`/`EndScenario` blocks — is defined in BCApps Performance Toolkit. Microsoft's shipped samples are intended as **starting points and baselines**, not as complete test coverage for an extension.
|
||||
|
||||
- **Framework source:** https://github.com/microsoft/BCApps/tree/main/src/Tools/Performance%20Toolkit
|
||||
- **Sample pattern:** `BCPTCreateSOWithNLines.Codeunit.al` in the Performance Toolkit samples shows the canonical codeunit structure to follow when building app-specific scenarios.
|
||||
|
|
@ -1,88 +1,38 @@
|
|||
---
|
||||
bc-version: [all]
|
||||
domain: testing
|
||||
keywords: [test, hardcode, random, library, no-series, setup, data]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
# CURABIS Test Data Guidelines
|
||||
|
||||
## Description
|
||||
## Core Principle
|
||||
|
||||
CURABIS tests assume an empty database. All test data must be created
|
||||
programmatically — never assume existing records or hardcode codes, numbers,
|
||||
or names that may or may not exist in a given environment.
|
||||
"CURABIS tests assume an empty database. All test data must be created programmatically — never assume existing records or hardcode codes, numbers, or names that may or may not exist in a given environment."
|
||||
|
||||
Three concrete rules:
|
||||
## Three Mandatory Rules
|
||||
|
||||
**1. Use MS Library codeunits for standard BC objects.**
|
||||
No-series, G/L accounts, customers, vendors, items, locations, posting groups —
|
||||
all created via `Library - ERM`, `Library - Inventory`, `Library - Sales` etc.
|
||||
These tools generate random codes that do not collide across test runs.
|
||||
**Rule 1: Leverage Microsoft Libraries**
|
||||
Use built-in setup codeunits (`Library - ERM`, `Library - Inventory`, `Library - Sales`) for standard Business Central objects like no-series, G/L accounts, customers, and items. These generate collision-free random codes.
|
||||
|
||||
**2. Fill all required fields with random values.**
|
||||
A `Code[10]` field gets 10 random characters. A `Text[50]` field gets random text.
|
||||
Use `Library - Utility` or `Any` codeunit for random generation.
|
||||
Partial setup that leaves required fields empty is not acceptable.
|
||||
**Rule 2: Complete All Required Fields**
|
||||
Every mandatory field must receive a value. A `Code[10]` field requires 10 random characters; `Text[50]` needs randomized text. Partial setups violating this principle are prohibited.
|
||||
|
||||
**3. Build your own tools for custom tables.**
|
||||
For CURABIS-specific tables (e.g. `Settlement Payment Method`,
|
||||
`Settlement Voucher Setup`), maintain dedicated setup procedures in the
|
||||
Test Library codeunit. These procedures must follow the same pattern as
|
||||
Microsoft's libraries: create records programmatically, use random values
|
||||
for codes where no fixed value is required by the flow being tested.
|
||||
**Rule 3: Create Custom Procedures for Domain-Specific Tables**
|
||||
For CURABIS-exclusive tables, build dedicated setup functions in Test Library following Microsoft's patterns: programmatic creation with random values unless the test documents a fixed contract requirement.
|
||||
|
||||
**Exception — integration and flow tests.**
|
||||
When a test validates a specific integration contract (e.g. a fixed JSON
|
||||
structure from a web service, a specific EDIFACT message, a fixed counterparty
|
||||
code expected by an external system), hardcoded values are acceptable and
|
||||
necessary. The test is documenting the contract, not exercising random data.
|
||||
## Critical Exception
|
||||
|
||||
## Anti Pattern
|
||||
Integration and flow tests validating external contracts (JSON structures, EDIFACT messages, counterparty codes) may use hardcoded values. These tests document the integration specification itself, not arbitrary test logic.
|
||||
|
||||
```al
|
||||
// WRONG: hardcoded code that may or may not exist
|
||||
if not PaymentMethod.Get('CASH') then begin
|
||||
PaymentMethod.Code := 'CASH';
|
||||
...
|
||||
end;
|
||||
```
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
```al
|
||||
// WRONG: hardcoded source code
|
||||
SourceCode.Code := 'SV-POST';
|
||||
```
|
||||
- Conditional hardcoded lookups assuming pre-existing data
|
||||
- Shortened field values not matching declared field length
|
||||
- Underfilled required fields
|
||||
|
||||
```al
|
||||
// WRONG: partial setup — Code[10] left short
|
||||
PaymentMethod.Code := 'C'; // not filled to capacity
|
||||
```
|
||||
## Implementation Example
|
||||
|
||||
## Best Practice
|
||||
Generate randomized payment method codes via `LibraryUtility.GenerateRandomCode()` rather than assuming 'CASH' exists. Create source codes through `LibraryERM.CreateSourceCode()` and retrieve no-series using `LibraryUtility.GetGlobalNoSeriesCode()`.
|
||||
|
||||
```al
|
||||
// CORRECT: random code via LibraryUtility
|
||||
PaymentMethod.Code :=
|
||||
CopyStr(LibraryUtility.GenerateRandomCode(
|
||||
PaymentMethod.FieldNo(Code), DATABASE::"Settlement Payment Method"), 1, 10);
|
||||
PaymentMethod.Description := LibraryUtility.GenerateRandomText(50);
|
||||
PaymentMethod.Insert();
|
||||
```
|
||||
## BCApps Reference
|
||||
|
||||
```al
|
||||
// CORRECT: source code created via standard MS pattern
|
||||
LibraryERM.CreateSourceCode(SourceCode);
|
||||
GlobalSourceCode := SourceCode.Code;
|
||||
// then assign to Source Code Setup
|
||||
```
|
||||
The randomization helpers central to this rule — `LibraryUtility.GenerateRandomCode()`, `LibraryERM.CreateSourceCode()`, `LibraryUtility.GetGlobalNoSeriesCode()` — are implemented and maintained in BCApps. BCApps test code never hardcodes record identifiers like `'CASH'`, `'10000'`, or `'70000'`; all test data is generated programmatically.
|
||||
|
||||
```al
|
||||
// CORRECT: no-series via MS library
|
||||
GlobalNoSeriesCode := LibraryUtility.GetGlobalNoSeriesCode();
|
||||
```
|
||||
|
||||
```al
|
||||
// CORRECT: hardcoded in integration test — documenting a contract
|
||||
// [SCENARIO] Inbound ORDRSP with fixed order reference from Allnet Germany
|
||||
ExpectedOrderRef := 'ORD-2026-00001'; // fixed by integration contract
|
||||
```
|
||||
- **Source:** https://github.com/microsoft/BCApps/tree/main/src/Tools/Test%20Framework
|
||||
- **Pattern:** BCApps test codeunits create every required record from scratch using library helpers that guarantee uniqueness per test run. The CURABIS rule mirrors this approach exactly.
|
||||
- **Note:** The `BCPTCreateSOWithNLines.Codeunit.al` sample in BCApps uses `Customer.get('10000')` as a fallback — this is a BCPT performance scenario (not a correctness test) and explicitly acknowledges this deviation. Correctness tests must never do this.
|
||||
|
|
|
|||
|
|
@ -1,71 +1,33 @@
|
|||
---
|
||||
bc-version: [all]
|
||||
domain: testing
|
||||
keywords: [test, library, setup, initialize, suppresscommit, asserterror]
|
||||
technologies: [al]
|
||||
countries: [w1]
|
||||
application-area: [all]
|
||||
---
|
||||
# CURABIS Test Library Standards
|
||||
|
||||
## Description
|
||||
## Core Rules
|
||||
|
||||
In CURABIS test apps, all test setup is centralized in a dedicated Test Library
|
||||
codeunit (e.g. `SV Test Library`). Individual test procedures must not call
|
||||
BC standard library codeunits (`LibrarySales`, `LibraryInventory`, etc.) directly.
|
||||
The documentation establishes three critical testing practices for CURABIS AL applications:
|
||||
|
||||
Additionally, two rules apply to every test that calls a posting codeunit:
|
||||
1. **Centralized Setup**: "all test setup is centralized in a dedicated Test Library codeunit" rather than individual test procedures calling BC standard libraries directly.
|
||||
|
||||
1. `SetSuppressCommit(true)` must be called before `Run()` to prevent data
|
||||
from leaking between tests.
|
||||
2. `asserterror` must always be followed by `Assert.ExpectedErrorCode()` or
|
||||
`Assert.ExpectedError()` — a naked `asserterror` passes on any error,
|
||||
not just the expected one.
|
||||
2. **Suppress Commits**: `SetSuppressCommit(true)` must execute before `Run()` to isolate test data and prevent cross-test contamination.
|
||||
|
||||
## Anti Pattern
|
||||
3. **Assertion After asserterror**: Every `asserterror` statement requires a subsequent `Assert.ExpectedErrorCode()` or `Assert.ExpectedError()` call to validate the specific error, preventing false passes from unexpected exceptions.
|
||||
|
||||
```al
|
||||
// WRONG: inline setup bypassing the test library
|
||||
procedure MyTest()
|
||||
var
|
||||
Item: Record Item;
|
||||
begin
|
||||
LibraryInventory.CreateItem(Item); // do not call directly
|
||||
// ...
|
||||
end;
|
||||
```
|
||||
## Key Violations
|
||||
|
||||
```al
|
||||
// WRONG: posting without SuppressCommit
|
||||
SVPost.Run(SVHeader); // commits to test database
|
||||
```
|
||||
The anti-patterns section highlights three common mistakes:
|
||||
|
||||
```al
|
||||
// WRONG: naked asserterror
|
||||
asserterror SVPost.Run(SVHeader);
|
||||
// no assertion follows — passes on any error
|
||||
```
|
||||
- Bypassing the test library by directly invoking BC standard codeunits like `LibraryInventory`
|
||||
- Executing posting operations without suppressing commits, which "commits to test database"
|
||||
- Using "naked asserterror" that "passes on any error, not just the expected one"
|
||||
|
||||
## Best Practice
|
||||
## Correct Implementation
|
||||
|
||||
```al
|
||||
// CORRECT: delegate to test library
|
||||
procedure MyTest()
|
||||
var
|
||||
Item: Record Item;
|
||||
begin
|
||||
SVLib.GivenScrapItem(Item); // test library owns setup
|
||||
// ...
|
||||
end;
|
||||
```
|
||||
The best practice section demonstrates the preferred approach: delegating setup operations to the test library (e.g., `SVLib.GivenScrapItem()`), enabling `SuppressCommit` before posting operations, and pairing error assertions with specific error code validations.
|
||||
|
||||
```al
|
||||
// CORRECT: SuppressCommit before Run
|
||||
SVPost.SetSuppressCommit(true);
|
||||
SVPost.Run(SVHeader);
|
||||
```
|
||||
These guidelines ensure test isolation, maintainability, and reliability across CURABIS test suites.
|
||||
|
||||
```al
|
||||
// CORRECT: asserterror followed by assertion
|
||||
asserterror SVPost.Run(SVHeader);
|
||||
Assert.ExpectedErrorCode('Dialog');
|
||||
```
|
||||
## BCApps Reference
|
||||
|
||||
The test library pattern originates from BCApps. The Microsoft-maintained test framework libraries (`Library - ERM`, `Library - Inventory`, `Library - Sales`, `Library - Utility`, etc.) are defined in BCApps and establish the canonical pattern for centralized, reusable test setup. CURABIS's own Test Library codeunit follows this same structural model.
|
||||
|
||||
- **Source:** https://github.com/microsoft/BCApps/tree/main/src/Tools/Test%20Framework
|
||||
- **Pattern:** Microsoft never writes inline setup logic inside individual test procedures. All setup is routed through library codeunits that can be reused, versioned, and maintained independently of the test cases themselves.
|
||||
- **Why this matters:** BCApps Test Framework is the ground truth for how BC testing is intended to work. Deviating from this pattern creates test suites that are harder to maintain and more likely to share state across tests.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue