Microsoft/bcquality
1
0
Fork 0
mirror of https://github.com/microsoft/BCQuality.git synced 2026-10-06 15:16:56 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

Skaerp (Edison-evidens): carve-out for dokumenterede smoke-tests

Browse source
This commit is contained in:
Michael Dieringer 2026-08-13 14:10:10 +02:00
parent 2de05cc65a
commit a4c68a4751
1 changed files with 94 additions and 80 deletions
Download patch file Download diff file Expand all files Collapse all files

174
custom/knowledge/testing/given-blocks-must-cover-full-precondition-chain.md
View file

@ -1,80 +1,94 @@
--- ---
bc-version: [all] bc-version: [all]
domain: testing domain: testing
keywords: [given, test-setup, posting, report, request-page, precondition, dataset, completeness] keywords: [given, test-setup, posting, report, request-page, precondition, dataset, completeness]
technologies: [al] technologies: [al]
countries: [w1] countries: [w1]
application-area: [all] application-area: [all]
--- ---
# GIVEN blocks must cover the full precondition chain, not just the primary record # GIVEN blocks must cover the full precondition chain, not just the primary record
## Description ## Description
A `[GIVEN]` block ([[test-feature-scenario-tags]]) is only correct if it sets A `[GIVEN]` block ([[test-feature-scenario-tags]]) is only correct if it sets
up every precondition the code under test actually reads — not just the one up every precondition the code under test actually reads — not just the one
record the scenario is "about." For most master-data tests, creating the record the scenario is "about." For most master-data tests, creating the
primary record is enough. For **posting routines** and **reports**, it primary record is enough. For **posting routines** and **reports**, it
usually is not, and an incomplete `[GIVEN]` produces a test that passes for usually is not, and an incomplete `[GIVEN]` produces a test that passes for
the wrong reason (it never truly exercises the logic being claimed) or the wrong reason (it never truly exercises the logic being claimed) or
fails with an unrelated setup error that has nothing to do with the fails with an unrelated setup error that has nothing to do with the
scenario. scenario.
Observed repeatedly across CURABIS AL projects (2026-08): AI-written test Observed repeatedly across CURABIS AL projects (2026-08): AI-written test
cases for posting and report scenarios create the primary document or cases for posting and report scenarios create the primary document or
record but skip the setup records the posting/report logic silently record but skip the setup records the posting/report logic silently
depends on, producing tests that either don't compile against a clean depends on, producing tests that either don't compile against a clean
company, or that "pass" without ever reaching the assertion the scenario company, or that "pass" without ever reaching the assertion the scenario
claims to prove. claims to prove.
## Posting Scenarios ## Posting Scenarios
A `[GIVEN]` block feeding a posting routine must account for: A `[GIVEN]` block feeding a posting routine must account for:
- The posting-relevant setup records the routine reads (e.g. `Sales & - The posting-relevant setup records the routine reads (e.g. `Sales &
Receivables Setup`, `General Ledger Setup`, `Inventory Setup`) — Receivables Setup`, `General Ledger Setup`, `Inventory Setup`) —
whichever ones the specific posting path touches. whichever ones the specific posting path touches.
- Number series, or an explicitly assigned document number if number - Number series, or an explicitly assigned document number if number
series aren't part of the scenario. series aren't part of the scenario.
- The full posting-group chain the document requires (e.g. Customer - The full posting-group chain the document requires (e.g. Customer
Posting Group + Gen. Business/Product Posting Group + VAT Posting Posting Group + Gen. Business/Product Posting Group + VAT Posting
Setup combination) — a missing link here fails with a G/L-account error Setup combination) — a missing link here fails with a G/L-account error
that has nothing to do with what the test claims to verify. that has nothing to do with what the test claims to verify.
- Dimension defaults, if the scenario's data flow requires them (see - Dimension defaults, if the scenario's data flow requires them (see
[[dimension-support-must-follow-dimensionmanagement-wiring-pattern]]). [[dimension-support-must-follow-dimensionmanagement-wiring-pattern]]).
- An explicit date (not an implicit reliance on the sandbox's current - An explicit date (not an implicit reliance on the sandbox's current
system date) when the posting path is date-sensitive. system date) when the posting path is date-sensitive.
## Report Scenarios ## Report Scenarios
A `[GIVEN]` block feeding a report test must include **both** a record A `[GIVEN]` block feeding a report test must include **both** a record
that should be included by the report's filters/dataset, **and** a record that should be included by the report's filters/dataset, **and** a record
that should be excluded — otherwise the test cannot distinguish "the that should be excluded — otherwise the test cannot distinguish "the
filter works" from "the filter does nothing." Also account for: filter works" from "the filter does nothing." Also account for:
- Request page parameters the report's `OnPreDataItem`/`OnAfterGetRecord` - Request page parameters the report's `OnPreDataItem`/`OnAfterGetRecord`
logic branches on — a `[GIVEN]` that never sets a parameter the report logic branches on — a `[GIVEN]` that never sets a parameter the report
reads is testing the report's default behavior only, not the scenario. reads is testing the report's default behavior only, not the scenario.
- Any FlowField/calculated data the report's dataset relies on, since - Any FlowField/calculated data the report's dataset relies on, since
those are easy to leave at zero/blank and get a report that "runs" but those are easy to leave at zero/blank and get a report that "runs" but
never touches the logic under test. never touches the logic under test.
## Review Checklist **Carve-out: smoke tests don't need an excluded record.** A test whose
own `[SCENARIO]`/name explicitly claims only "the report runs without
1. For a posting test: does `[GIVEN]` set up every posting group / setup error" — not filter correctness — is not violating this rule by setting
record the routine will read, or only the primary document? up a single matching record with no excluded counterpart. The
2. For a report test: does `[GIVEN]` include at least one record that included/excluded pairing requirement applies to tests that claim to
should be filtered *out*, not only records that should appear? verify filtering or dataset logic; it does not retroactively apply to a
3. Does `[GIVEN]` set dates/parameters explicitly wherever the code under test that never made that claim. What *is* required either way: the test's
test branches on them, rather than relying on sandbox defaults? own scope must say which claim it's making — an undocumented single-record
4. If posting or report execution fails with a setup-related error `[GIVEN]` on a report test is ambiguous (is this an incomplete filter test,
unrelated to the scenario's own claim, that is a `[GIVEN]` gap — fix or a deliberate smoke test?), and that ambiguity is the actual violation,
the precondition, don't work around the error in `[WHEN]`/`[THEN]`. not the missing excluded record by itself.
## Source ## Review Checklist
Consultant feedback relayed by Michael Dieringer, 2026-08-13: AI-written 1. For a posting test: does `[GIVEN]` set up every posting group / setup
test cases for posting and report scenarios repeatedly under-specify record the routine will read, or only the primary document?
`[GIVEN]` conditions, requiring manual correction ("bare spørge den om at 2. For a report test that claims to verify filtering or dataset logic:
huske at tænke over GIVEN condition"). Sharpens [[test-feature-scenario-tags]] does `[GIVEN]` include at least one record that should be filtered
and [[test-setup-must-use-library-codeunit]], which cover comment structure *out*, not only records that should appear? (A test explicitly scoped
and setup centralization but not precondition completeness for these two as a run-without-error smoke test is exempt — see Report Scenarios.)
scenario types. 3. Does `[GIVEN]` set dates/parameters explicitly wherever the code under
test branches on them, rather than relying on sandbox defaults?
4. If posting or report execution fails with a setup-related error
unrelated to the scenario's own claim, that is a `[GIVEN]` gap — fix
the precondition, don't work around the error in `[WHEN]`/`[THEN]`.
## Source
Consultant feedback relayed by Michael Dieringer, 2026-08-13: AI-written
test cases for posting and report scenarios repeatedly under-specify
`[GIVEN]` conditions, requiring manual correction ("bare spørge den om at
huske at tænke over GIVEN condition"). Sharpens [[test-feature-scenario-tags]]
and [[test-setup-must-use-library-codeunit]], which cover comment structure
and setup centralization but not precondition completeness for these two
scenario types.