Microsoft/bcquality
1
0
Fork 0
mirror of https://github.com/microsoft/BCQuality.git synced 2026-08-06 17:36:53 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Add new action skills for AL testing and documentation

Browse source 
- Introduced `al-test-writer` to generate AL test codeunits for production objects based on TDD principles.
- Added `al-userguide-test-writer` to create test codeunits from user guide steps, mapping actions and assertions.
- Implemented `bc-extension-test-guide` to generate a comprehensive TEST_GUIDE.md for Business Central extensions, covering various categories.
- Created `bc-webclient-runner` to automate UI testing of the Business Central web client, capturing screenshots and asserting UI states.
- Developed `page-scripting-e2e` to produce a recording plan for Page Scripting, ensuring a structured approach to browser-level testing.
This commit is contained in:
Tharanga Chandrasekara 2026-06-07 12:26:47 +12:00
parent 822cae1b27
commit 07140e2223
76 changed files with 4353 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

37
custom/knowledge/integration/accept-async-work-instead-of-synchronous-wait-loops.bad.al Normal file
View file

@ -0,0 +1,37 @@
// Anti-pattern: the inbound handler BLOCKS the request thread, Sleep-polling an external
// service until it completes. The caller's connection is held open for the whole wait,
// and concurrent requests pile up on pinned threads.
codeunit 50123 "Inbound Intake Bad"
{
// Called on the request path. It does not return until the remote work is done,
// so its runtime is entirely dictated by a system BC does not control.
procedure Accept(Payload: Text): Text
var
Client: HttpClient;
Response: HttpResponseMessage;
JobId: Text;
Done: Boolean;
begin
JobId := StartRemoteJob(Client, Payload);
// BAD: Sleep inside a loop on the request thread. This single request now holds
// its thread and session slot for the full duration of the remote job.
repeat
// When the downstream is SLOW: this blocks for seconds or minutes. The caller's
// HTTP connection times out long before the loop ends, and the work it kicked
// off is orphaned with no staged row recording it.
Sleep(2000);
// When the downstream is DOWN: this Get blocks until its own timeout, every
// iteration, making a slow failure even slower.
Client.Get(StrSubstNo('https://svc.contoso.com/jobs/%1', JobId), Response);
Done := IsComplete(Response);
until Done;
// Under load: each in-flight request pins a thread here. A handful of slow calls
// exhaust the request slots and BC starts rejecting healthy callers too. One slow
// dependency becomes a site-wide outage.
exit('completed'); // by now the original caller has almost certainly timed out
end;
}

87
custom/knowledge/integration/accept-async-work-instead-of-synchronous-wait-loops.good.al Normal file
View file

@ -0,0 +1,87 @@
// Best practice: accept the work, STAGE it, return 202 Accepted with a status URL, and
// free the request thread immediately. A background processor finishes the slow work;
// the caller polls the status URL and watches Status advance. No thread is ever pinned
// to a downstream system's latency.
codeunit 50120 "Inbound Intake"
{
// Called from the API insert trigger / webhook receiver. It returns the URL the HTTP
// layer puts in the Location header alongside a 202 Accepted.
procedure Accept(Payload: Text; ExternalRef: Text[100]) StatusUrl: Text
var
IntegrationMessage: Record "Integration Message";
begin
IntegrationMessage.Init();
IntegrationMessage."Message ID" := CreateGuid();
IntegrationMessage.Direction := IntegrationMessage.Direction::Inbound;
IntegrationMessage."External Reference" := ExternalRef;
// New means "accepted, not yet processed". The background processor picks it up;
// the caller sees it move to In Progress, then Resolved or Failed.
IntegrationMessage.Status := IntegrationMessage.Status::New;
IntegrationMessage."Correlation ID" := CopyStr(DelChr(LowerCase(Format(CreateGuid())), '=', '{}'), 1, 40);
IntegrationMessage.SetRequest(Payload);
// The ONLY expensive thing on the request path is this Insert. The moment it
// returns, the request thread is free to serve the next caller.
IntegrationMessage.Insert(true);
// Point the caller at the staged row. The HTTP layer maps this to
// 202 Accepted + a Location header; the caller polls it for completion.
exit(StrSubstNo('/api/contoso/integration/v1.0/integrationMessages(%1)', IntegrationMessage."Message ID"));
end;
}
// Read-only status endpoint the caller polls. No blocking, no Sleep, no remote call:
// it just projects the current state of the staged row.
page 50121 "Integration Message Status"
{
PageType = API;
APIPublisher = 'contoso';
APIGroup = 'integration';
APIVersion = 'v1.0';
EntityName = 'integrationMessage';
EntitySetName = 'integrationMessages';
SourceTable = "Integration Message";
Editable = false; // a status endpoint never mutates; it only reports
layout
{
area(Content)
{
repeater(Group)
{
field(id; Rec."Message ID") { }
// The field the caller polls. New -> In Progress -> Resolved / Failed.
field(status; Rec.Status) { }
field(errorMessage; Rec."Error Message") { } // populated only on Failed
}
}
}
}
codeunit 50122 "Inbound Processor"
{
TableNo = "Job Queue Entry";
// Runs in the background, NOT on the request thread. This is where the slow work lives,
// so the caller's connection is never held open for it.
trigger OnRun()
var
IntegrationMessage: Record "Integration Message";
begin
IntegrationMessage.SetRange(Direction, IntegrationMessage.Direction::Inbound);
IntegrationMessage.SetRange(Status, IntegrationMessage.Status::New);
if IntegrationMessage.FindSet() then
repeat
IntegrationMessage.Status := IntegrationMessage.Status::"In Progress";
IntegrationMessage.Modify(true);
Commit(); // make In Progress visible to a polling caller at once
Process(IntegrationMessage); // the slow part: runs here, off the request path
until IntegrationMessage.Next() = 0;
end;
local procedure Process(var IntegrationMessage: Record "Integration Message")
begin
// ... do the real work; on success set Status::Resolved and store the response,
// on failure set Status::Failed and stamp Error Message. The caller's next poll sees it.
end;
}

32
custom/knowledge/integration/accept-async-work-instead-of-synchronous-wait-loops.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [async, sleep, polling, http-202, status-url, api-handler, request-thread]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Accept async work instead of synchronous wait loops
## Description
An inbound API handler or webhook receiver that kicks off external work and then `Sleep`-and-polls until it finishes holds the request thread for the entire wait, and with it the session slot and any locks the handler has taken. The handler is now blocked on work it does not own and cannot speed up. Web requests into Business Central have a finite server-side budget, so a wait measured in seconds or minutes does not produce a slow-but-correct answer: the connection times out, the caller gets an error, and the work it triggered is orphaned with no record that it ever started.
The damage compounds under load and under exactly the external conditions you cannot control. When the downstream system is slow, each in-flight request pins a thread, so a handful of slow calls exhaust the available request slots and healthy callers start getting rejected too: one slow dependency becomes a site-wide outage. When the downstream system is down, every request blocks for the full timeout before failing, turning a fast failure into a slow one and multiplying the thread pressure. The handler must never block on work it does not own. Accept the request, persist it, answer immediately, and let the caller check back.
## Best Practice
Split acceptance from completion. Stage the request as an Integration Message, return `202 Accepted` with a status URL that points at that staged row, and let a background processor do the slow work. The request thread is freed the instant the row is written, so throughput is bounded by how fast you can insert rows, not by how slow the downstream system is. The mechanism that makes the caller whole is the status URL plus the Status field: the caller polls a read-only API page over the Integration Message keyed by its Message ID and watches Status move from New to In Progress to Resolved or Failed, reading the final response from the same row. This applies to any inbound path where completion is not guaranteed to be immediate. See `accept-async-work-instead-of-synchronous-wait-loops.good.al`.
The trade-off is that the caller must be willing to poll (or accept a callback), which is a contract you state up front with the 202 and the Location header. For work that genuinely answers in the same call, a synchronous response is fine; reach for staging the moment completion depends on a system you do not control.
## Anti Pattern
An inbound handler that contains a `Sleep` inside a `repeat ... until` or `while` loop that re-queries an external service for completion before returning. The detection signal: `Sleep(` together with a loop and an `HttpClient` call inside an API page trigger, a webhook codeunit, or any procedure on the request path; equivalently, a handler whose return value depends on a remote status it polls in-line. The consequence is that the request blocks for the full duration of external work, the caller's connection times out, and concurrent requests pile up on pinned threads until the service stops accepting new ones. The fix is to stage the request and return 202 with a status URL. See `accept-async-work-instead-of-synchronous-wait-loops.bad.al`.
## See also
- `park-long-running-work-on-a-status-url.md`
- `stage-every-integration-message.md`
- `propagate-a-correlation-id-across-every-hop.md`

33
custom/knowledge/integration/deduplicate-inbound-messages-with-an-idempotency-check.bad.al Normal file
View file

@ -0,0 +1,33 @@
// Anti-pattern: insert on every call with no idempotency check, keyed on a brand-new GUID.
// A retried or re-fetched delivery is processed again as a fresh message, so a duplicating
// source produces duplicate documents and double-applied side effects.
codeunit 50142 "Inbound Dedup Bad"
{
procedure Stage(ExternalRef: Text[100]; MsgType: Code[40]; Payload: Text): Guid
var
IntegrationMessage: Record "Integration Message";
begin
// BAD: no SetRange on External Reference + Type, no Get, no lookup of any kind.
// The source system's stable id is captured on the row but never used to detect a repeat.
IntegrationMessage.Init();
// BAD: the only "identity" is a fresh GUID. If anyone later "dedups" on Message ID,
// it can never match, because every insert mints a new one. This is the illusion of a
// dedup key that can never actually fire.
IntegrationMessage."Message ID" := CreateGuid();
IntegrationMessage.Direction := IntegrationMessage.Direction::Inbound;
IntegrationMessage."External Reference" := ExternalRef;
IntegrationMessage.Type := MsgType;
IntegrationMessage.Status := IntegrationMessage.Status::New;
IntegrationMessage.SetRequest(Payload);
// When the source RETRIES (a webhook that did not see our ack, a restart re-send, an
// overlapping poll window): this runs again with the same ExternalRef and stages a
// second message. Downstream it becomes a second sales order and a second posting.
// The duplicate volume scales with how aggressively the source retries.
IntegrationMessage.Insert(true);
exit(IntegrationMessage."Message ID");
end;
}

67
custom/knowledge/integration/deduplicate-inbound-messages-with-an-idempotency-check.good.al Normal file
View file

@ -0,0 +1,67 @@
// Best practice: deduplicate on the source system's stable id (External Reference + Type)
// BEFORE staging, backed by a unique key. A replay returns the prior result instead of
// being processed again. The internal Message ID is never the dedup key, because it is
// freshly generated per insert and so could never match a repeat.
table 50140 "Integration Message"
{
DataClassification = CustomerContent;
fields
{
field(1; "Message ID"; Guid) { Caption = 'Message ID'; }
field(2; Direction; Enum "Integration Direction") { Caption = 'Direction'; }
field(3; "Type"; Code[40]) { Caption = 'Type'; }
field(4; Status; Enum "Integration Status") { Caption = 'Status'; }
// The source-controlled stable id. This, not Message ID, is what dedup keys on.
field(5; "External Reference"; Text[100]) { Caption = 'External Reference'; }
}
keys
{
key(PK; "Message ID") { Clustered = true; }
// UNIQUE idempotency key: a second concurrent insert of the same delivery fails at
// the database, so dedup holds even under a race, not only on the explicit lookup.
key(Idempotency; "External Reference", "Type") { Unique = true; }
}
}
codeunit 50141 "Inbound Dedup"
{
procedure Stage(ExternalRef: Text[100]; MsgType: Code[40]; Payload: Text): Guid
var
Existing: Record "Integration Message";
IntegrationMessage: Record "Integration Message";
begin
// The idempotency lookup: a single indexed read on the unique key.
Existing.SetRange("External Reference", ExternalRef);
Existing.SetRange(Type, MsgType);
if Existing.FindFirst() then begin
case Existing.Status of
Existing.Status::Resolved:
// Already processed. Return the prior result; do NOT do the work again.
exit(Existing."Message ID");
Existing.Status::"In Progress":
// A run is already handling this exact external reference. Reject the
// second one rather than process the same message concurrently.
Error('Message %1 of type %2 is already in progress', ExternalRef, MsgType);
end;
// Any other prior state (for example Failed): return the existing row so the
// resolution flow handles it, instead of minting a duplicate.
exit(Existing."Message ID");
end;
// No prior message exists: stage a genuinely new one.
IntegrationMessage.Init();
IntegrationMessage."Message ID" := CreateGuid(); // internal id, never the dedup key
IntegrationMessage.Direction := IntegrationMessage.Direction::Inbound;
IntegrationMessage."External Reference" := ExternalRef;
IntegrationMessage.Type := MsgType;
IntegrationMessage.Status := IntegrationMessage.Status::New;
IntegrationMessage.SetRequest(Payload);
// If a concurrent request slipped past the lookup, the unique key makes THIS Insert
// fail rather than create a duplicate. Either way, the side effect runs at most once.
IntegrationMessage.Insert(true);
exit(IntegrationMessage."Message ID");
end;
}

32
custom/knowledge/integration/deduplicate-inbound-messages-with-an-idempotency-check.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [idempotency, deduplication, external-reference, inbound, replay, in-progress, unique-key]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Deduplicate inbound messages with an idempotency check
## Description
Duplicate inbound messages are a guarantee, not an edge case. A source system re-fetches and resends after a restart, a webhook platform fires a retry because it did not see your acknowledgement in time, a poll window overlaps a previous one, a load balancer replays a request. Every one of these delivers a message you have already seen, and the source genuinely believes it is doing the right thing by retrying. The receiver, not the sender, is responsible for recognising the repeat, because only the receiver knows what it has already processed.
The mechanism that makes recognition possible is the source system's own stable identifier. Before staging an inbound message, look it up by that identifier plus the message type. If a matching message is already Resolved, return its stored response and do nothing else, because the work is already done. If a matching message is In Progress, wait or reject rather than start a second concurrent run against the same external reference. Only when there is no match do you stage a new message and process it. Skip this check and a slow or duplicating external system turns every replay into real work: duplicate sales orders, double postings, duplicate outbound side effects that ripple to yet more systems.
## Best Practice
Deduplicate on `External Reference + Type`, the stable id the source system controls, and back it with a unique key on `(External Reference, Type)` so the lookup is a single indexed read and a concurrent duplicate insert fails at the database rather than racing through. Never deduplicate on the internal Message ID: that GUID is generated fresh on every insert, so it never matches a replay and gives you the illusion of a dedup check that can never fire. On a Resolved hit return the stored response so the caller sees the same answer it would have seen the first time; on an In Progress hit reject or back off so two runs do not process the same external reference at once; only on no hit do you insert and process. See `deduplicate-inbound-messages-with-an-idempotency-check.good.al`.
The trade-off is one indexed read on the ingest path, which is cheap, and a unique key that will reject a genuine duplicate insert, which is the point. Pair this with outbound idempotency keys (see `send-an-idempotency-key-on-every-outbound-call.md`) so the same flow is protected against duplicates on the way out as well as on the way in.
## Anti Pattern
An inbound handler that inserts a new Integration Message on every call without first checking for an existing one, or that deduplicates on the internal Message ID instead of the source's External Reference. The detection signal: an `Insert` of an inbound message with no prior `SetRange`/`Get` on `External Reference` and `Type`, a `CreateGuid()` used as the dedup key, or a dedup lookup keyed on `Message ID`. The consequence is that a retried or re-fetched delivery is processed as a brand-new message, so a duplicating source produces duplicate documents and double-applied side effects, and the volume scales with how aggressively the source retries. The fix is a lookup on `External Reference + Type` before any insert, backed by a unique key. See `deduplicate-inbound-messages-with-an-idempotency-check.bad.al`.
## See also
- `send-an-idempotency-key-on-every-outbound-call.md`
- `use-a-framing-record-for-inbound-polling.md`
- `stage-every-integration-message.md`

36
custom/knowledge/integration/make-failed-integration-messages-manually-resolvable.bad.al Normal file
View file

@ -0,0 +1,36 @@
// Anti-pattern: failed messages are PURGED, and the only "retry" mints a NEW message. The
// payload and error context needed to diagnose the failure are destroyed, and the new id breaks
// idempotency so the receiver double-applies the side effect.
codeunit 50221 "Failed Cleanup Bad"
{
procedure PurgeFailed()
var
IntegrationMessage: Record "Integration Message";
begin
// BAD: deleting failed rows means a fix needs a code change and a redeploy, because there
// is no editable row for ops to correct and re-run. Every data-level failure becomes an
// engineering incident. Worse, the payload and the error that explain WHAT failed are gone,
// so diagnosis after the fact is impossible.
IntegrationMessage.SetRange(Status, IntegrationMessage.Status::Failed);
IntegrationMessage.DeleteAll(true);
end;
procedure RetryFailed(SourceRef: Text[100]; MsgType: Code[40]; Payload: Text)
var
NewMessage: Record "Integration Message";
begin
// BAD: a manual retry that creates a BRAND-NEW message with a fresh Message ID. The
// idempotency key is derived from the Message ID, so a new id means a new key, and the
// receiver sees this as a new request rather than a repeat of the failed one. If the
// original attempt had partly landed (a charge captured, a shipment booked), this retry
// applies the side effect a SECOND time. The correct fix is to re-run the EXISTING message.
NewMessage.Init();
NewMessage."Message ID" := CreateGuid();
NewMessage."External Reference" := SourceRef;
NewMessage.Type := MsgType;
NewMessage.Status := NewMessage.Status::New;
NewMessage.SetRequest(Payload);
NewMessage.Insert(true);
end;
}

75
custom/knowledge/integration/make-failed-integration-messages-manually-resolvable.good.al Normal file
View file

@ -0,0 +1,75 @@
// Best practice: a resolution page over failed messages. Ops corrects the payload and flips
// Status to New; the processor re-runs the SAME Message ID under the SAME idempotency key, so
// the fix reprocesses without creating a duplicate. The shape mirrors what Microsoft ships for
// E-Document: editable staging, a resolution page, a status enum, retry actions, and an audit trail.
page 50220 "Integration Resolution"
{
PageType = List;
SourceTable = "Integration Message";
// Failed messages stay EDITABLE so ops can correct the payload without a developer or a deploy.
Editable = true;
SourceTableView = where(Status = const(Failed));
Caption = 'Integration Resolution';
layout
{
area(Content)
{
repeater(Group)
{
// The id is shown but NOT editable: re-running must reuse it so the idempotency
// key (derived from this id) stays the same and the retry cannot double-apply.
field("Message ID"; Rec."Message ID") { Editable = false; }
field("External Reference"; Rec."External Reference") { Editable = false; }
// The error context ops needs to diagnose the failure. Read-only: it is history.
field("Error Message"; Rec."Error Message") { Editable = false; }
field("Retry Count"; Rec."Retry Count") { Editable = false; }
// The payload ops actually edits to fix a malformed or mis-mapped message.
field(Request; Rec.GetRequest()) { }
field("Resolution Note"; Rec."Resolution Note") { } // audit of what was decided
}
}
}
actions
{
area(Processing)
{
action(Resolve)
{
Caption = 'Resolve';
trigger OnAction()
begin
// Re-run the SAME message: same Message ID, therefore same idempotency key.
// The processor reprocesses the corrected payload, and the receiver recognises
// the repeat, so a side effect that partly landed is not applied a second time.
Rec.Status := Rec.Status::New;
Rec.Modify(true);
end;
}
action(ConfirmByException)
{
Caption = 'Confirm by Exception';
trigger OnAction()
begin
// Accept as handled with NO retry (for example the work was completed manually
// out of band). The audit record is kept so the decision is traceable.
Rec.Status := Rec.Status::Resolved;
Rec."Resolution Note" := 'Confirmed by exception';
Rec.Modify(true);
end;
}
action(Reassign)
{
Caption = 'Reassign';
trigger OnAction()
begin
// Route to another handler/queue without losing the message or its history.
Rec."Assigned To" := PickHandler();
Rec.Modify(true);
end;
}
}
}
}

32
custom/knowledge/integration/make-failed-integration-messages-manually-resolvable.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [manual-resolution, failed-message, resolution-page, edocument, confirm-by-exception, ops, audit]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Make failed integration messages manually resolvable
## Description
Automation cannot fix every failure. A malformed payload from a source that changed its format, a mapping gap for a product that was set up wrong, a one-off data problem on a single document: these are not transient and no amount of retrying resolves them, because the data itself is the problem. When automation cannot recover, a human has to be able to step in, and the design must let them do it without a developer and a deployment. If the only way to fix a stuck message is to change code and ship a release, then every data-level failure becomes an engineering incident, and the backlog of stuck messages grows while it waits for the next deployment window.
Manual resolution is therefore a first-class part of the integration design, not an afterthought bolted on once something breaks. Failed Integration Messages must stay editable so operations can correct the payload and re-run the same message, and the re-run has to preserve identity so it does not undo the very guarantees the happy path relied on. The shape worth copying is the one Microsoft already ships for E-Document: inbound staging, a resolution page, a status enum, retry actions, and an audit trail, so the experience is familiar to anyone who has resolved an electronic document and the audit story is already understood.
## Best Practice
Keep Failed messages editable. Operations corrects the payload on the row and flips Status back to New, and the processor re-runs the same Message ID under the same idempotency key, so the correction reprocesses without creating a duplicate and without double-applying a side effect that may already have partly landed. Provide a resolution page exposing the payload and the error, with three actions: Resolve (re-run the same message after a fix), Confirm-by-Exception (accept the message as handled with no retry, keeping the audit record so the decision is traceable), and Reassign (route the message to another handler or queue). The mechanism that makes re-run safe is reusing the existing Message ID rather than minting a new one, because the idempotency key is derived from that id (see `send-an-idempotency-key-on-every-outbound-call.md`), so a human-driven retry is as safe against duplicates as an automated one. Mirror the E-Document shape so the experience and the audit trail are familiar. See `make-failed-integration-messages-manually-resolvable.good.al`.
The trade-off is keeping failed rows around and editable rather than purging them, which costs storage and demands a resolution UI, in exchange for a system where a data problem is an operations task rather than an engineering deployment.
## Anti Pattern
Failed messages that are read-only or auto-deleted, so a fix means a code change and redeploy, or a manual retry that mints a new Message ID and so loses the idempotency guarantee. The detection signal: a Failed status with no editable payload and no resolution page, a purge or cleanup job that `DeleteAll`s failed rows, or a manual retry path that calls `CreateGuid()` to create a fresh message instead of re-running the existing one. The consequences are that every data failure becomes a deployment (read-only or deleted rows), the payload and error context needed to diagnose it are gone (deletion), or the retry double-applies the side effect because the receiver sees a new request rather than a repeat (new Message ID). The fix is editable failed rows, a resolution page with retry actions, and a re-run that reuses the same Message ID. See `make-failed-integration-messages-manually-resolvable.bad.al`.
## See also
- `deduplicate-inbound-messages-with-an-idempotency-check.md`
- `send-an-idempotency-key-on-every-outbound-call.md`
- `stage-every-integration-message.md`

66
custom/knowledge/integration/monitor-external-event-subscription-health.good.al Normal file
View file

@ -0,0 +1,66 @@
// Best practice: a scheduled monitor lists the LIVE external event subscriptions and diffs
// them against the EXPECTED set. Any expected subscription that BC has silently dropped
// (because the subscriber returned a non-408/429/5xx response) raises an alert and telemetry.
table 50180 "Expected Event Subscription"
{
DataClassification = SystemMetadata;
fields
{
// The expected set lives in configuration, so registering an integration also registers
// its monitoring expectation. The two cannot drift apart.
field(1; "Event Name"; Text[100]) { Caption = 'Event Name'; }
field(2; "Notification URL"; Text[250]) { Caption = 'Notification URL'; }
}
keys { key(PK; "Event Name", "Notification URL") { Clustered = true; } }
}
codeunit 50181 "Subscription Health Monitor"
{
TableNo = "Job Queue Entry";
// Runs as a Job Queue entry on a schedule (for example hourly). It must run actively:
// a dropped subscription is indistinguishable from a quiet feed, so silence cannot be trusted.
trigger OnRun()
begin
CheckHealth();
end;
procedure CheckHealth()
var
Expected: Record "Expected Event Subscription";
Live: List of [Text];
begin
// GET api/microsoft/runtime/v1.0/externaleventsubscriptions and project each live
// subscription to a comparable key.
Live := FetchLiveSubscriptions();
if Expected.FindSet() then
repeat
// The diff: an expected subscription missing from the live list was dropped by
// the platform with no notification. That is an incident, not a warning.
if not Live.Contains(SubscriptionKey(Expected."Event Name", Expected."Notification URL")) then
RaiseMissingSubscriptionAlert(Expected);
until Expected.Next() = 0;
end;
local procedure RaiseMissingSubscriptionAlert(Expected: Record "Expected Event Subscription")
var
Dimensions: Dictionary of [Text, Text];
begin
// Telemetry carries the event name and URL so operations can re-register it AND can read,
// from the telemetry timeline, roughly when delivery stopped.
Dimensions.Add('eventName', Expected."Event Name");
Dimensions.Add('notificationUrl', Expected."Notification URL");
Session.LogMessage('INT0001', 'External event subscription missing', Verbosity::Warning,
DataClassification::SystemMetadata, TelemetryScope::ExtensionPublisher, Dimensions);
// ... and raise an operational alert (email, Teams, ticket) so a human acts before the gap grows.
end;
local procedure SubscriptionKey(EventName: Text; NotificationUrl: Text): Text
begin
exit(StrSubstNo('%1|%2', EventName, NotificationUrl));
end;
}

28
custom/knowledge/integration/monitor-external-event-subscription-health.md Normal file
View file

@ -0,0 +1,28 @@
---
bc-version: [all]
domain: integration
keywords: [subscription-health, monitor, external-business-event, silent-drop, alert, job-queue, telemetry]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Monitor external event subscription health
## Description
Business Central removes an external business event subscription when the subscriber's notification endpoint returns anything other than 408, 429, or a 5xx response. A 404 because the consumer redeployed to a new URL, a 401 because a token expired, a 400 because a proxy mangled the request: any of these tells the platform the endpoint is permanently unable to accept the notification, so it stops trying and drops the subscription. This is reasonable platform behaviour, but there is no built-in alert when it happens. The subscription simply disappears and notifications stop flowing.
The reason this is dangerous is that a dropped subscription is indistinguishable from a quiet feed. If nothing has happened to raise the event lately, no notifications would arrive anyway, so the absence of traffic looks normal. The gap is typically discovered only when someone downstream asks why they stopped receiving events, by which point the integration has been silently broken for hours or days and there may be a backlog of business activity that was never communicated. Because the platform will not tell you, the only way to catch a drop is to check for it actively and on a schedule.
## Best Practice
Run a monitor job on a schedule (a Job Queue entry, for example hourly) that lists the current external event subscriptions from the `externaleventsubscriptions` endpoint and compares them against the set the integration expects to exist. Keep the expected set in a small configuration table so that registering an integration also registers its monitoring expectation, and the two never drift apart. The mechanism is the diff: for every expected subscription that is absent from the live list, raise an operational alert and emit telemetry carrying the event name and notification URL, so operations can re-register it before the gap grows and can see, from the telemetry timeline, roughly when delivery stopped. Treat a missing subscription as an incident, not a warning to be filtered out. See `monitor-external-event-subscription-health.good.al`.
The trade-off is one scheduled read of the subscription list per interval plus a small table of expectations, which is a negligible cost against the alternative of a multi-day silent outage discovered by a downstream complaint.
## See also
- `prefer-business-events-over-handwritten-retry-loops.md`
- `version-business-events-and-keep-payloads-stable.md`
- `propagate-a-correlation-id-across-every-hop.md`

33
custom/knowledge/integration/never-call-external-services-from-posting.bad.al Normal file
View file

@ -0,0 +1,33 @@
// Anti-pattern: the posting subscriber calls an external service INLINE, inside the
// posting transaction. The post now holds document and ledger locks until the remote
// endpoint answers, and a remote failure rolls the whole post back.
codeunit 50112 "Post Shipment Notifier Bad"
{
[EventSubscriber(ObjectType::Codeunit, Codeunit::"Sales-Post", 'OnAfterPostSalesDoc', '', false, false)]
local procedure NotifyShipmentInline(var SalesHeader: Record "Sales Header")
var
Client: HttpClient;
Content: HttpContent;
Response: HttpResponseMessage;
begin
Content.WriteFrom(BuildShipmentJson(SalesHeader));
// BAD: HttpClient.Post runs INSIDE the posting transaction. The locks taken by
// Sales-Post on the header, the lines, and the related ledger entries stay held
// for the entire round trip to the WMS.
//
// When the WMS is SLOW: every other user posting a sales document queues behind
// these locks. One slow endpoint serialises the whole team's posting.
//
// When the WMS is DOWN: this call blocks until the HTTP timeout fires, then throws.
// The throw propagates out of the posting transaction and the entire post ROLLS BACK.
// The shipment physically left the warehouse, but there is now no posted document
// and nothing was staged, so there is nothing to retry and nothing to inspect.
Client.Post('https://wms.contoso.com/api/shipments', Content, Response);
// Even on a success that is not actually success: a network blip after the WMS
// committed but before BC saw the response leaves the two systems disagreeing,
// with no staged row recording that the notification was attempted.
end;
}

73
custom/knowledge/integration/never-call-external-services-from-posting.good.al Normal file
View file

@ -0,0 +1,73 @@
// Best practice: the posting subscriber only STAGES an outbound message and returns.
// The HttpClient.Send happens later, in a Job Queue codeunit, outside the posting lock.
// The row is inserted inside the posting transaction, so it exists only if the post
// committed; the callout runs in a separate transaction where a remote outage can only
// delay delivery, never roll back a posted shipment.
codeunit 50110 "Post Shipment Notifier"
{
// Subscriber on the real posting publisher. It runs while posting locks are held,
// so it must do nothing that can block: no HTTP, no second remote call, just an Insert.
[EventSubscriber(ObjectType::Codeunit, Codeunit::"Sales-Post", 'OnAfterPostSalesDoc', '', false, false)]
local procedure StageShipmentNotification(var SalesHeader: Record "Sales Header")
var
IntegrationMessage: Record "Integration Message";
begin
// Only notify on an actual posted shipment, not on every posted document kind.
if not SalesHeader.Ship then
exit;
IntegrationMessage.Init();
IntegrationMessage."Message ID" := CreateGuid();
// Direction Outbound + Status New is exactly what the Job Queue processor queries for.
IntegrationMessage.Direction := IntegrationMessage.Direction::Outbound;
IntegrationMessage.Status := IntegrationMessage.Status::New;
IntegrationMessage.Type := 'SHIPMENT-NOTIFY';
// Carry the document ANCHOR, not a live handle. The processor re-reads detail later.
IntegrationMessage."Document No." := SalesHeader."No.";
IntegrationMessage."External Reference" := SalesHeader."External Document No.";
// Correlation id threads this notification to the rest of the flow's log lines.
IntegrationMessage."Correlation ID" := CopyStr(DelChr(LowerCase(Format(CreateGuid())), '=', '{}'), 1, 40);
// Insert participates in the posting transaction: the row lives only if the post commits,
// and rolls back cleanly with the post if posting fails. No remote system is touched here.
IntegrationMessage.Insert(true);
end;
}
codeunit 50111 "Outbound Sender"
{
// Runs as a Job Queue entry, well after posting has committed and released its locks.
// Nothing it does can lengthen a posting lock window, because there is no longer a post in flight.
procedure SendNew()
var
IntegrationMessage: Record "Integration Message";
begin
IntegrationMessage.SetRange(Direction, IntegrationMessage.Direction::Outbound);
IntegrationMessage.SetRange(Status, IntegrationMessage.Status::New);
if IntegrationMessage.FindSet() then
repeat
// Each row is its own short unit of work. A slow endpoint stalls delivery
// of THIS message only; it cannot stall anyone's posting.
SendOne(IntegrationMessage);
until IntegrationMessage.Next() = 0;
end;
local procedure SendOne(var IntegrationMessage: Record "Integration Message")
var
Client: HttpClient;
Content: HttpContent;
Response: HttpResponseMessage;
begin
Content.WriteFrom(BuildShipmentJson(IntegrationMessage));
if Client.Post(GetEndpoint(IntegrationMessage), Content, Response) and Response.IsSuccessStatusCode() then begin
IntegrationMessage.Status := IntegrationMessage.Status::Resolved;
IntegrationMessage.Modify(true);
end else begin
// A failure here is recorded on the row and retried later. The posted shipment
// is already durable, so a WMS outage never costs us the posting.
IntegrationMessage."Retry Count" += 1;
IntegrationMessage."Error Message" := CopyStr(GetLastErrorText(), 1, 2048);
IntegrationMessage.Modify(true);
end;
end;
}

32
custom/knowledge/integration/never-call-external-services-from-posting.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [posting, httpclient, callout, job-queue, locks, subscriber, rollback]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Never call external services from posting
## Description
A posting routine runs as one database transaction and holds write locks on the document header, the document lines, and every ledger and entry table it touches until that transaction commits. Calling an external service from inside that routine, or from a posting subscriber such as `OnAfterPostSalesDoc`, `OnAfterPostPurchaseDoc`, or `OnAfterFinalizePosting`, binds the lifetime of those locks to the response time of a system Business Central does not control. The remote endpoint, not BC, now decides how long the locks are held.
The failure mode is concrete and it gets worse under exactly the conditions you cannot prevent. When the external system is slow, the posting transaction stays open and every other user who needs those records waits behind it, so one sluggish endpoint serialises an entire team's posting. When the external system is down, the call blocks until the HTTP timeout fires and then throws, and because the throw happens inside the posting transaction the whole post rolls back: the shipment that physically left the warehouse now has no posted document, and nothing was staged to retry. When the external system is healthy but the network blips, you get an uncertain failure on a transaction that may already have committed downstream. The remote call has to leave the posting transaction entirely.
## Best Practice
Stage the outbound work instead of sending it inline. From the posting subscriber, write one Integration Message row (Direction Outbound, Status New) that carries the document anchor (for example the posted document number) and whatever payload the receiver needs, then return immediately so posting commits on local state alone. A background Job Queue codeunit reads the staged rows by Status and performs the actual `HttpClient.Send` outside any posting lock. The mechanism that makes this safe is the commit boundary: the row is inserted in the posting transaction, so it exists only if the post succeeded, and the callout runs in a separate later transaction where a remote outage delays delivery without ever touching the posting locks or the posted document. See `never-call-external-services-from-posting.good.al`.
This applies to every posting and posting-adjacent path, inbound or outbound. The one nuance worth knowing: firing an `[ExternalBusinessEvent]` from a posting subscriber is not a violation, because that is not an HTTP call and the platform delivers it post-commit (see `prefer-business-events-over-handwritten-retry-loops.md`). The trade-off of staging is added latency and one more table, which is the point: you are trading immediacy for a posting path that cannot be held hostage.
## Anti Pattern
A posting routine or a posting-event subscriber that calls `HttpClient.Send` directly, or that invokes a client codeunit which does. The detection signal a reviewer or agent can match: an `HttpClient`, `HttpRequestMessage`, `HttpContent`, or REST/JSON client reference inside a `Codeunit "*-Post"`, or inside a subscriber bound to `OnAfterPostSalesDoc`, `OnAfterPostPurchaseDoc`, `OnAfterFinalizePosting`, `OnBeforePost*`, or any publisher on a posting codeunit. The consequence is that posting locks are now held for the full remote round trip, and a remote failure rolls back a post that should have been durable. The fix is structural: move the callout into a Job Queue processor that reads staged rows. See `never-call-external-services-from-posting.bad.al`.
## See also
- `stage-every-integration-message.md`
- `prefer-business-events-over-handwritten-retry-loops.md`
- `propagate-a-correlation-id-across-every-hop.md`

32
custom/knowledge/integration/park-long-running-work-on-a-status-url.bad.al Normal file
View file

@ -0,0 +1,32 @@
// Anti-pattern: a Job Queue tight loop that Sleep-polls the status URL, with the retry count in
// a local variable that resets on restart. The loop pins a Job Queue slot for the entire
// external wait, and the counter never survives long enough to drive real backoff or give-up.
codeunit 50202 "Long Running Bad"
{
procedure Start(var IntegrationMessage: Record "Integration Message")
var
Client: HttpClient;
Response: HttpResponseMessage;
Location: array[1] of Text;
RetryCount: Integer; // BAD: lost on restart; this state belongs on the message row
begin
Client.Post('https://svc.contoso.com/api/jobs', BuildContent(IntegrationMessage), Response);
Response.Headers().GetValues('Location', Location);
// BAD: a tight Sleep-poll loop INSIDE the Job Queue handler. This single flow now holds a
// Job Queue worker slot for the whole external wait. A flow that can take hours starves
// every other job behind it, because the slot is occupied doing nothing but sleeping.
repeat
Sleep(5000);
// BAD: RetryCount is a local. Every BC restart resets it to zero, so backoff and the
// give-up threshold below never behave correctly across a restart: the loop effectively
// starts over, having forgotten how long it has already been waiting.
RetryCount += 1;
Client.Get(Location[1], Response);
until IsComplete(Response) or (RetryCount > 1000);
// The flow lives only in this session. If BC recycles the session mid-wait, the work is
// orphaned: nothing parked it, so nothing will ever resume it.
end;
}

59
custom/knowledge/integration/park-long-running-work-on-a-status-url.good.al Normal file
View file

@ -0,0 +1,59 @@
// Best practice: on 202 Accepted, PARK the message as Awaiting Reply with the status URL on
// the row, then let a scheduled poll resume it. Retry count and last error live on the MESSAGE,
// so a resume after a restart still knows how often it has tried and why it last failed.
codeunit 50200 "Long Running Start"
{
procedure Start(var IntegrationMessage: Record "Integration Message")
var
Client: HttpClient;
Response: HttpResponseMessage;
Headers: HttpHeaders;
Location: array[1] of Text;
begin
Client.Post('https://svc.contoso.com/api/jobs', BuildContent(IntegrationMessage), Response);
// 202 means "accepted, answer later". Treat it as a DEFERRAL, not a failure to retry and
// not a completion. Re-sending the request here would duplicate work the service already took.
if Response.HttpStatusCode() = 202 then begin
Response.Headers().GetValues('Location', Location);
// Store the status URL on the row and park it. The flow now lives in the database,
// not in this session, so it survives the session ending.
IntegrationMessage."Status URL" := CopyStr(Location[1], 1, 250);
IntegrationMessage.Status := IntegrationMessage.Status::"Awaiting Reply";
IntegrationMessage.Modify(true);
end;
// Start returns immediately. Work that waits more than ~30s belongs to external
// orchestration (a Logic App / Durable Function) or a brief scheduled poll, NEVER a
// Job Queue tight loop. The Job Queue owns short, BC-bounded units of work.
end;
}
codeunit 50201 "Long Running Resume"
{
TableNo = "Job Queue Entry";
// Runs on a schedule. Each invocation does a quick pass over parked rows and returns; it does
// not sit and wait. A flow that is still pending simply gets picked up again next run.
procedure ResumeAwaiting()
var
IntegrationMessage: Record "Integration Message";
Client: HttpClient;
Response: HttpResponseMessage;
begin
IntegrationMessage.SetRange(Status, IntegrationMessage.Status::"Awaiting Reply");
if IntegrationMessage.FindSet() then
repeat
if Client.Get(IntegrationMessage."Status URL", Response) and IsComplete(Response) then
Complete(IntegrationMessage)
else begin
// Retry/last-error state lives ON THE MESSAGE, not in a variable. A resume in
// a different session after a restart still sees the true attempt count.
IntegrationMessage."Retry Count" += 1;
IntegrationMessage."Error Message" := CopyStr(LastError(Response), 1, 2048);
IntegrationMessage.Modify(true);
end;
until IntegrationMessage.Next() = 0;
end;
}

32
custom/knowledge/integration/park-long-running-work-on-a-status-url.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [long-running, http-202, status-url, awaiting-reply, orchestration, retry-state, durable-function]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Park long-running work on a status URL
## Description
Some external work does not answer in the call that starts it. The service accepts the request, returns `202 Accepted` with a status URL in the Location header, and finishes minutes or hours later. This is a correct and common pattern on the remote side, and Business Central has to handle it correctly on its side, which means avoiding two opposite mistakes. Blocking on the work until it finishes is wrong, because it pins a session for the whole wait (the no-synchronous-wait-loop rule). Firing the request and forgetting it is also wrong, because the answer arrives later with nothing in BC tracking that it is owed.
The right shape treats the 202 as a deferral, not a failure or a completion. On receiving it, park the Integration Message as Awaiting Reply with the status URL stored on the row, and let a separate scheduled poll resume the flow when the answer is ready. The request and the eventual confirmation are two states of one message sharing a correlation id, not two unrelated events. The detail that makes a parked flow survivable is where its retry state lives: retry count and last error belong on the message row, not in a codeunit variable, because a variable resets on restart and a flow that can wait hours will almost certainly outlive the session that started it.
## Best Practice
On a 202, read the Location header, store it on the message as the status URL, and set Status to Awaiting Reply. A scheduled poll reads Awaiting Reply rows, queries each status URL, and advances the message to Resolved when the work is done or records the failure when it is not. Keep retry count and last error on the message, so a resumed poll, possibly running in a different session after a restart, knows how many times it has tried and why it last failed. The mechanism that keeps the Job Queue healthy is the separation between parking and polling: the Job Queue owns short, BC-bounded units of work, so when the wait exceeds roughly 30 seconds the waiting belongs to external orchestration (a Logic App or a Durable Function) that calls back or that BC polls briefly, never a Job Queue tight loop holding a slot for hours. See `park-long-running-work-on-a-status-url.good.al`.
The trade-off is an extra status field and a poll job, which buys you a flow that resumes correctly across restarts and never monopolises a worker slot.
## Anti Pattern
Treating a 202 as a failure and retrying the original request, holding the work in a Job Queue tight loop that `Sleep`-polls the status URL, or keeping retry count in a codeunit variable that resets on restart. The detection signal: a 202 branch that re-sends the original request, a `Sleep` poll loop over a status URL inside a Job Queue handler, or retry/last-error state held in a local or global variable rather than on the Integration Message. The consequences are duplicate work (re-sending a request the service already accepted), a Job Queue slot pinned for the entire external wait (so a flow that waits hours starves other jobs), and a retry counter that resets to zero every restart so backoff and give-up logic never work. The fix is to park as Awaiting Reply with the status URL and resume via a scheduled poll, with all retry state on the row. See `park-long-running-work-on-a-status-url.bad.al`.
## See also
- `accept-async-work-instead-of-synchronous-wait-loops.md`
- `propagate-a-correlation-id-across-every-hop.md`
- `split-multi-step-flows-into-staged-job-queue-entries.md`

35
custom/knowledge/integration/prefer-business-events-over-handwritten-retry-loops.bad.al Normal file
View file

@ -0,0 +1,35 @@
// Anti-pattern: a hand-written HTTP retry loop for a fire-and-forget notification. It
// reimplements platform retry, backoff, and durability (usually less correctly), couples
// delivery to BC staying up for the life of the loop, and runs inline on the caller's thread.
codeunit 50162 "Shipment Notifier Bad"
{
procedure NotifyShipmentReleased(DocumentNo: Code[20])
var
Client: HttpClient;
Content: HttpContent;
Response: HttpResponseMessage;
Attempt: Integer;
begin
Content.WriteFrom(BuildJson(DocumentNo));
// BAD: a hand-rolled retry loop for a one-way notification an external business event
// would carry. Everything in this loop is something the platform already does for free.
for Attempt := 1 to 5 do begin
if Client.Post('https://wms.contoso.com/api/events', Content, Response) then
// BAD: status classification by hand. A real implementation must distinguish
// 408/429/5xx (retry) from 4xx (give up), and this one does not even try.
if Response.IsSuccessStatusCode() then
exit;
// BAD: hand-rolled backoff. The platform's external-event delivery already retries
// with backoff for up to ~36 hours and persists the state across restarts.
Sleep(Attempt * 2000);
end;
// BAD: the loop lives entirely in this session. If the WMS is DOWN for the whole window
// the notification is lost; and if BC RESTARTS mid-loop, the retry state is gone and the
// notification is silently lost with no record that it was ever attempted. Nobody is
// alerted, so the gap is found only when downstream is discovered to be out of sync.
end;
}

36
custom/knowledge/integration/prefer-business-events-over-handwritten-retry-loops.good.al Normal file
View file

@ -0,0 +1,36 @@
// Best practice: declare an external business event and fire it from a thin subscriber.
// The platform owns retry (408/429/5xx, up to ~36h) and backoff; delivery is asynchronous
// and post-commit, so the notification is sent only if the firing transaction commits.
codeunit 50160 "Shipment Events v1"
{
// [ExternalBusinessEvent], not [BusinessEvent]: the externally deliverable flavour that
// external subscribers can register against. Parameters are a minimal DTO of identifiers,
// never the BC record (see version-business-events-and-keep-payloads-stable).
[ExternalBusinessEvent('ShipmentReleased', 'Shipment released', 'Raised when a warehouse shipment is posted', EventCategory::Sales)]
procedure OnShipmentReleased_v1(DocumentNo: Code[20]; ExternalRef: Text[100])
begin
// Body is intentionally empty: the platform raises and delivers this; we only declare it.
end;
}
codeunit 50161 "Shipment Event Firer"
{
[EventSubscriber(ObjectType::Codeunit, Codeunit::"Whse.-Post Shipment", 'OnAfterPostWhseShipment', '', false, false)]
local procedure FireShipmentReleased(var WhseShptHeader: Record "Warehouse Shipment Header")
var
Events: Codeunit "Shipment Events v1";
begin
// Safe to fire from the posting path, even though an HttpClient.Send here would NOT be:
// - this is not an HTTP call, so it holds no lock open on a remote round trip;
// - the platform queues delivery and sends it only AFTER this transaction commits;
// - if posting rolls back, the event is never sent, so nothing leaks on failure.
Events.OnShipmentReleased_v1(WhseShptHeader."No.", WhseShptHeader."External Document No.");
end;
}
// External subscribers register themselves with no AL change, by POSTing to
// api/microsoft/runtime/v1.0/externaleventsubscriptions
// with eventName, appId, notificationUrl and clientState. Adding a consumer is configuration,
// not code. (External business events are available from runtime 11 and still preview; confirm
// the surface against current docs before relying on it.)

32
custom/knowledge/integration/prefer-business-events-over-handwritten-retry-loops.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [business-events, external-business-event, retry, outbound, post-commit, subscription, notification]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Prefer business events over handwritten retry loops
## Description
For outbound notification, the kind of "something happened, tell whoever is interested" message that does not need an inline answer, a hand-written AL HTTP retry loop reimplements machinery the platform already ships. To be correct, that loop must own exponential backoff, classify which status codes are worth retrying (408, 429, and 5xx) versus which are permanent, persist its retry state so it survives a restart, and stay alive long enough to exhaust its attempts. Most hand-rolled loops get at least one of these wrong, and the failure is silent: a notification is simply lost and nobody notices until a downstream system is found to be out of sync.
An `[ExternalBusinessEvent]` hands all of that to the platform. Business Central retries the delivery on 408, 429, and 5xx responses for up to roughly 36 hours, persists the delivery state itself, and lets external subscribers register without any AL change. Just as important, delivery is asynchronous and post-commit: the platform sends the notification only after the firing transaction commits, and never sends it if that transaction rolls back. That is why firing an event from a posting or release path is safe even though calling `HttpClient` from the same path is not. Prefer the event over the loop wherever a subscriber can register for it.
## Best Practice
Declare an `[ExternalBusinessEvent('name', 'Display', 'Desc', Category)]` whose parameters are a minimal DTO of identifiers (a document number, an external reference) rather than a record, and fire it from a thin subscriber on the real event such as release or post. The mechanism that makes this both reliable and safe is the platform's delivery model: the event is queued in the committing transaction, so it exists only if the business action succeeded, and the platform then owns retry and backoff against the registered notification URLs. External subscribers self-serve by POSTing to `api/microsoft/runtime/v1.0/externaleventsubscriptions` with the event name, app id, notification URL, and client state, so adding a consumer is a configuration step, not a code change. External business events are available from runtime 11 and are still labelled preview, so confirm the surface against current docs before relying on it. See `prefer-business-events-over-handwritten-retry-loops.good.al`.
The trade-off and its boundary: business events are for fire-and-forget notification, not for request/response where you need an answer in the same call. If the caller must act on a returned value, this is the wrong tool; stage an outbound message and call with an idempotency key instead.
## Anti Pattern
A custom AL codeunit that loops over `HttpClient.Send` with `Sleep` backoff to deliver a notification that an external business event could carry. The detection signal: a retry loop counting attempts around an outbound POST, classifying 429/5xx by hand, with `Sleep`-based backoff, where the payload is a one-way notification rather than a request needing an inline reply. The consequence is twofold: delivery is coupled to Business Central staying up for the life of the loop (a restart mid-loop loses the notification with no record), and you have reimplemented, usually less correctly, the retry, backoff, and durability the platform already provides. The fix is to declare an external business event and fire it from a thin subscriber. See `prefer-business-events-over-handwritten-retry-loops.bad.al`.
## See also
- `version-business-events-and-keep-payloads-stable.md`
- `monitor-external-event-subscription-health.md`
- `never-call-external-services-from-posting.md`

32
custom/knowledge/integration/propagate-a-correlation-id-across-every-hop.bad.al Normal file
View file

@ -0,0 +1,32 @@
// Anti-pattern: a new id minted at a downstream hop, and an outbound call with no correlation
// header. Nothing ties the inbound message, this processing step, and the external system's
// logs together. Tracing a failure becomes correlation-by-timestamp guesswork.
codeunit 50192 "Outbound Step Bad"
{
procedure Send(var IntegrationMessage: Record "Integration Message")
var
Client: HttpClient;
Request: HttpRequestMessage;
Headers: HttpHeaders;
Response: HttpResponseMessage;
LocalTrace: Guid;
begin
// BAD: a brand-new id, unrelated to IntegrationMessage."Correlation ID". The entry point
// already minted the flow's trace id; minting another one here breaks the chain just as
// thoroughly as having none, because the two halves now log different identifiers.
LocalTrace := CreateGuid();
// This trace value appears in no other component's logs, so it joins to nothing.
LogStep('sending', Format(LocalTrace));
Request.SetRequestUri('https://svc.contoso.com/api/orders');
Request.Method := 'POST';
Request.GetHeaders(Headers);
// BAD: no Correlation-Id header at all. The receiver logs the call under its own ids, and
// there is no shared value to join the external system's logs back to this flow. When this
// POST fails three hops into a busy system, reconstructing what happened is archaeology.
Client.Send(Request, Response);
end;
}

54
custom/knowledge/integration/propagate-a-correlation-id-across-every-hop.good.al Normal file
View file

@ -0,0 +1,54 @@
// Best practice: mint the correlation id ONCE at the entry point and carry it unchanged on
// every staged message, event payload, queue header, and outbound call. Log it at every step,
// so one filter pulls the entire flow across BC, the queue, and the external system.
codeunit 50190 "Inbound Entry"
{
// The boundary: this is the ONLY place a correlation value is created. Everything downstream
// reads it, never regenerates it.
procedure Receive(Payload: Text)
var
IntegrationMessage: Record "Integration Message";
begin
IntegrationMessage.Init();
IntegrationMessage."Message ID" := CreateGuid();
// Generated once, here, at the entry point. This is the trace id for the whole flow.
IntegrationMessage."Correlation ID" := NewCorrelationId();
IntegrationMessage.SetRequest(Payload);
IntegrationMessage.Insert(true);
// Log it on the very first step, so even the inbound receipt is part of the trace.
LogStep('received', IntegrationMessage."Correlation ID");
end;
local procedure NewCorrelationId(): Code[40]
begin
exit(CopyStr(DelChr(LowerCase(Format(CreateGuid())), '=', '{}'), 1, 40));
end;
}
codeunit 50191 "Outbound Step"
{
// A downstream hop. It READS the correlation id off the message it was handed; it does not
// mint a new one, because that would split the flow into two untraceable halves.
procedure Send(var IntegrationMessage: Record "Integration Message")
var
Client: HttpClient;
Request: HttpRequestMessage;
Headers: HttpHeaders;
Response: HttpResponseMessage;
begin
Request.SetRequestUri('https://svc.contoso.com/api/orders');
Request.Method := 'POST';
Request.GetHeaders(Headers);
// Carry the SAME id onto the outbound call as a header. The receiver logs it too, so the
// external system's logs can be joined back to the BC side by this one value.
Headers.Add('Correlation-Id', IntegrationMessage."Correlation ID");
Client.Send(Request, Response);
// Same id logged on this hop. Request and confirmation rows share it, so a status query
// or failure investigation pulls the whole chain with a single filter.
LogStep('sent', IntegrationMessage."Correlation ID");
end;
}

32
custom/knowledge/integration/propagate-a-correlation-id-across-every-hop.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [correlation-id, tracing, propagation, queue-header, telemetry, end-to-end, trace]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Propagate a correlation id across every hop
## Description
A single integration flow touches many components in sequence: an inbound message arrives, a row is staged, a business event fires, a queue entry is picked up, an outbound call goes out, a confirmation comes back. Each component logs its own activity, but unless one identifier is threaded through all of them, those log lines are isolated islands. When something fails three hops in, reconstructing what happened means correlating by timestamp and hope, guessing which inbound message produced which outbound call, which is slow at the best of times and nearly impossible when the system is busy and many flows are interleaved.
A correlation id solves this by giving every component in one flow the same trace identifier to log and to pass along. It is the second most load-bearing field on the Integration Message after its own key, because it is what turns a pile of disconnected log entries into a single traceable story. The discipline that matters is that it is set exactly once, at the point where the flow enters Business Central, and then carried unchanged everywhere downstream. The most common failure is not the absence of a correlation id but its regeneration: a downstream step that mints a fresh id breaks the chain just as thoroughly as having none, because now two halves of the same flow log different identifiers.
## Best Practice
Generate the correlation id once at the entry point, the webhook receiver, the poll handler, or the first staged message, and never regenerate it downstream. Carry it on every subsequent Integration Message, every event payload, every queue message header, and every outbound HTTP request as a header such as `Correlation-Id`, and log it at every step. The mechanism that pays off is that the request row and its eventual confirmation row, and every step in between, all carry the one identifier, so a status query or a failure investigation pulls the entire chain, across Business Central, the queue, and the external system, with a single filter. Read the id from the incoming message rather than creating a new one; the only `CreateGuid` for a correlation value lives at the entry point. See `propagate-a-correlation-id-across-every-hop.good.al`.
The cost is one field carried and one header set per hop, which is trivial; the payoff is that incident response goes from archaeology to a single filtered query.
## Anti Pattern
Generating a new id at each hop, or not carrying the id onto outbound calls and queue headers at all, so each component logs an unrelated identifier. The detection signal: a `CreateGuid()` producing a correlation value inside a downstream processor or outbound step rather than reading the id from the incoming message, an outbound `HttpClient` request or event payload that omits the correlation header, or log statements that emit a locally minted trace value. The consequence is that no two components share a trace id, so tracing a failure across the flow requires correlating by timestamp and guesswork, and the external system's logs can never be joined back to the BC side at all. The fix is one id minted at the entry point and read, never regenerated, by every hop after it. See `propagate-a-correlation-id-across-every-hop.bad.al`.
## See also
- `stage-every-integration-message.md`
- `park-long-running-work-on-a-status-url.md`
- `monitor-external-event-subscription-health.md`

37
custom/knowledge/integration/send-an-idempotency-key-on-every-outbound-call.bad.al Normal file
View file

@ -0,0 +1,37 @@
// Anti-pattern: a fresh key per attempt (and the loop would be just as broken with no key).
// Every retry looks like a brand-new request, so after an uncertain failure the receiver
// applies the side effect AGAIN. A flaky payment service produces duplicate charges exactly
// when it is least healthy.
codeunit 50151 "Outbound Caller Bad"
{
procedure Send(var IntegrationMessage: Record "Integration Message")
var
Client: HttpClient;
Request: HttpRequestMessage;
Content: HttpContent;
Headers: HttpHeaders;
Response: HttpResponseMessage;
Attempt: Integer;
begin
for Attempt := 1 to 3 do begin
Content.WriteFrom(IntegrationMessage.GetRequest());
Request.Content := Content;
Request.Method := 'POST';
Request.SetRequestUri('https://pay.contoso.com/api/charges');
Request.GetHeaders(Headers);
// BAD: a new GUID on every attempt. The key is supposed to let the receiver
// recognise a retry, but a value that changes each time is functionally NO key:
// attempt 2 and attempt 3 each look like a completely new charge request.
Headers.Add('Idempotency-Key', Format(CreateGuid()));
// The dangerous case is the UNCERTAIN failure. If attempt 1 actually reached the
// service and captured the payment, but the response was lost to a timeout, then
// Send returns false here and the loop retries. Attempt 2 carries a different key,
// so the service captures the payment a SECOND time. The customer is charged twice.
if Client.Send(Request, Response) and Response.IsSuccessStatusCode() then
exit;
end;
end;
}

47
custom/knowledge/integration/send-an-idempotency-key-on-every-outbound-call.good.al Normal file
View file

@ -0,0 +1,47 @@
// Best practice: every outbound call carries Idempotency-Key = the Integration Message GUID.
// The key is created once when the message is staged and never changes, so the first call
// and every retry (Job Queue or operator-driven) send the SAME key. A well-behaved receiver
// collapses them into a single side effect and returns the original response.
codeunit 50150 "Outbound Caller"
{
procedure Send(var IntegrationMessage: Record "Integration Message")
var
Client: HttpClient;
Request: HttpRequestMessage;
Content: HttpContent;
ContentHeaders: HttpHeaders;
RequestHeaders: HttpHeaders;
Response: HttpResponseMessage;
begin
Content.WriteFrom(IntegrationMessage.GetRequest());
// Content-Type belongs on the content headers, not the request headers.
Content.GetHeaders(ContentHeaders);
if ContentHeaders.Contains('Content-Type') then
ContentHeaders.Remove('Content-Type');
ContentHeaders.Add('Content-Type', 'application/json');
Request.Content := Content;
Request.Method := 'POST';
Request.SetRequestUri('https://pay.contoso.com/api/charges');
Request.GetHeaders(RequestHeaders);
// THE key line. The value is the staged Message ID, which is stable for the life of
// the message. Calling Send again for the same row sends this exact same value, so the
// payment service sees the retry as a repeat of one charge and captures money once.
RequestHeaders.Add('Idempotency-Key', StableKey(IntegrationMessage));
// After an UNCERTAIN failure (timeout, dropped connection, 502) the Job Queue will
// re-run this message. Because the key is unchanged, the retry is safe: no double charge.
Client.Send(Request, Response);
IntegrationMessage.RecordResult(Response);
end;
// The key is derived purely from the durable message id. Nothing here changes between
// attempts: no CreateGuid, no timestamp, no attempt counter.
local procedure StableKey(IntegrationMessage: Record "Integration Message"): Text
begin
exit(DelChr(LowerCase(Format(IntegrationMessage."Message ID")), '=', '{}'));
end;
}

32
custom/knowledge/integration/send-an-idempotency-key-on-every-outbound-call.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [idempotency-key, outbound, http-header, retry, message-guid, side-effect, uncertain-failure]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Send an idempotency key on every outbound call
## Description
When Business Central calls an external system, some failures are certain (a 400 that clearly rejected the request) but the dangerous ones are uncertain: a socket timeout, a dropped connection, a 502 from a gateway in front of a service that may have processed the request anyway. After an uncertain failure you genuinely do not know whether the work landed. You must retry to make progress, but a blind retry risks doing the side effect twice: a second payment captured, a second shipment booked, a second order placed. The uncertainty is inherent to networks and cannot be engineered away; what you can do is make the retry safe.
An idempotency key makes it safe. It is a value the caller sends so the receiver can recognise a repeat of a request it has already handled and return the original response instead of acting again. The contract is the caller's responsibility: the receiver can only deduplicate if every retry of the same logical request carries the same key. That is the crux of the rule, because the most common bug is a key that changes between attempts, which looks like a fix but is functionally no key at all. Every outbound call that has a side effect must carry a stable key.
## Best Practice
Set an `Idempotency-Key` header on every outbound request and derive its value from the Integration Message GUID, which is created once when the message is staged and never changes. Because the key lives on the staged row, every retry of that row, whether by the Job Queue minutes later or by an operator resolving a failed message days later, sends the identical key, so a well-behaved receiver collapses all of them into one side effect and returns the same response. The mechanism is the binding of the key to the durable message rather than to the attempt: a new key is minted only when a genuinely new message is created. See `send-an-idempotency-key-on-every-outbound-call.good.al`.
This pairs with re-running the same message on manual resolution (see `make-failed-integration-messages-manually-resolvable.md`): because resolution re-runs the same Message ID, it reuses the same idempotency key, so even a human-driven retry cannot double-apply. The only cost is one header per request and the discipline of never regenerating the key.
## Anti Pattern
An outbound `HttpClient` call with no idempotency header, or one that generates a fresh key per attempt (for example `CreateGuid()` or a counter inside the retry loop) so each retry looks like a brand-new request to the receiver. The detection signal: an `HttpClient.Post`/`Send` building an outbound request that has a side effect but no `Idempotency-Key` header, or a key whose source is anything that changes between attempts (a `CreateGuid()` inside the loop, a timestamp, an attempt counter). The consequence is that after an uncertain failure the retry double-applies the side effect, so a slow or flaky receiver produces duplicate payments and duplicate shipments precisely when it is least healthy. The fix is one stable key derived from the message GUID, set on every attempt. See `send-an-idempotency-key-on-every-outbound-call.bad.al`.
## See also
- `deduplicate-inbound-messages-with-an-idempotency-check.md`
- `make-failed-integration-messages-manually-resolvable.md`
- `park-long-running-work-on-a-status-url.md`

38
custom/knowledge/integration/split-multi-step-flows-into-staged-job-queue-entries.bad.al Normal file
View file

@ -0,0 +1,38 @@
// Anti-pattern: one handler runs every step in a single transaction, and a SingleInstance
// codeunit holds lookups that leak across runs. One big lock, one big rollback, and stages
// that should be independent are coupled through shared state.
codeunit 50213 "Monolithic Flow Bad"
{
procedure RunAll(var IntegrationMessage: Record "Integration Message")
begin
// BAD: fetch, transform, post, and notify all run in ONE transaction. Every lock any step
// takes is held until the final step commits, so the slowest/most contended step sets the
// lock duration for all of them.
Fetch(IntegrationMessage);
Transform(IntegrationMessage);
// If Post fails, Fetch and Transform ROLL BACK with it: their successful work is discarded
// and the whole flow must re-run from the start, redoing work that had already succeeded.
Post(IntegrationMessage);
// A transient hiccup HERE, after a perfectly good post, throws away that post too, because
// it is all one transaction. The unit of failure is the entire flow, not the failing step.
Notify(IntegrationMessage);
end;
}
codeunit 50214 "Cross Stage Cache Bad"
{
SingleInstance = true; // BAD: a global cache that survives between stage runs couples the stages
var
ItemCache: Dictionary of [Code[20], Code[20]];
procedure Lookup(ItemNo: Code[20]): Code[20]
begin
// BAD: stages that read this cache now depend on whichever earlier run populated it. They
// can no longer be retried or reordered in isolation, which is exactly the independence a
// staged split is supposed to give. Wanting a cache this global is the tell the split is wrong.
exit(ItemCache.Get(ItemNo));
end;
}

59
custom/knowledge/integration/split-multi-step-flows-into-staged-job-queue-entries.good.al Normal file
View file

@ -0,0 +1,59 @@
// Best practice: each stage is its own Job Queue entry implementing IIntegrationStage,
// dispatched from an extensible enum. Status is the cursor that records the flow's position.
// Stages share no state across runs, so each has its own short lock window and its own retry.
interface IIntegrationStage
{
procedure Run(var IntegrationMessage: Record "Integration Message");
procedure NextStatus(): Enum "Integration Status";
}
// Extensible: adding a stage is ONE new codeunit plus ONE enum value, with no orchestrator change.
enum 50210 "Integration Stage" implements IIntegrationStage
{
Extensible = true;
value(0; Fetch) { Implementation = IIntegrationStage = "Stage Fetch"; }
value(1; Transform) { Implementation = IIntegrationStage = "Stage Transform"; }
value(2; Post) { Implementation = IIntegrationStage = "Stage Post"; }
}
codeunit 50211 "Stage Transform" implements IIntegrationStage
{
procedure Run(var IntegrationMessage: Record "Integration Message")
var
ItemCache: Dictionary of [Code[20], Code[20]];
begin
// The cache is local to THIS run. It is created here and gone when Run returns, so it
// cannot couple this stage to any other. If a lookup were hot enough to want a GLOBAL
// cache, that would be the signal the split is in the wrong place.
TransformPayload(IntegrationMessage, ItemCache);
// Only this stage's work is in scope, so its lock window is short and it commits on its own.
end;
procedure NextStatus(): Enum "Integration Status"
begin
// Advances the cursor to the next stage. A failure here rolls back ONLY this stage;
// Fetch stays committed and the flow resumes from Transform, not from the start.
exit("Integration Status"::Post);
end;
}
codeunit 50212 "Stage Dispatcher"
{
TableNo = "Job Queue Entry";
// Each invocation runs ONE stage as its own Job Queue entry, advances Status, then the next
// stage runs as a separate entry. No step holds a lock across another step's work.
procedure RunStage(var IntegrationMessage: Record "Integration Message"; Stage: Enum "Integration Stage")
var
StageImpl: Interface IIntegrationStage;
begin
StageImpl := Stage;
StageImpl.Run(IntegrationMessage);
// Status is the cursor: it records where the flow is up to, so resuming is just reading
// the next stage. There is no separate per-stage row to reconcile.
IntegrationMessage.Status := StageImpl.NextStatus();
IntegrationMessage.Modify(true);
end;
}

32
custom/knowledge/integration/split-multi-step-flows-into-staged-job-queue-entries.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [staged-pipeline, job-queue, iintegrationstage, interface, lock-window, no-shared-state, rollback]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Split multi-step flows into staged job queue entries
## Description
A multi-step integration flow (fetch, transform, post, notify) handled by one big codeunit in one transaction is one big lock and one big rollback. Every lock the flow takes anywhere along the chain is held until the final step commits, so the slowest or most contended step sets the lock duration for all of them, and a failure in the last step discards the successful work of every earlier step. A transient hiccup while notifying then throws away a posting that was perfectly good, and the whole flow re-runs from the start, redoing work that had already succeeded.
Splitting the flow into stages, each its own Job Queue entry, changes the unit of failure and the unit of locking. Every stage gets its own short lock window and its own retry policy, and a failed stage rolls back only its own work, leaving earlier stages committed and the flow free to resume from where it stopped. The flow advances stage by stage with the message Status as the cursor: Status records the position, so resuming is just reading the next stage to run, and there is no separate per-stage row to reconcile. The discipline that keeps the stages genuinely independent is that they share no state across runs.
## Best Practice
Make each stage its own Job Queue entry with its own short lock window and retry. Have stages implement a common `IIntegrationStage` interface dispatched from an extensible enum, so adding a stage is one new codeunit plus one enum value with no change to the orchestrator. Use Status as the cursor that records the flow's position; do not create a new row per stage, because the message is the flow and its Status is where it is up to. The rule that makes the split real is shared state: a stage may cache item, customer, or location lookups within a single run (a Dictionary that is created and discarded inside one invocation), but never across stages and never globally, because cross-run cache is exactly the coupling the split exists to remove. A useful tell is that if a lookup is hot enough to tempt you toward a global cache, the split is in the wrong place. See `split-multi-step-flows-into-staged-job-queue-entries.good.al`.
The trade-off is more moving parts (an interface, an enum, several codeunits) in exchange for short lock windows, per-stage retry, and a flow that resumes instead of restarting. That trade is worth making once a flow has more than one step that can fail independently.
## Anti Pattern
One handler that runs every step in a single transaction, or stages that pass data through a global or cross-invocation cache. The detection signal: a single codeunit whose `Run`/`OnRun` does fetch, transform, post, and notify in sequence in one transaction, or a `SingleInstance` codeunit or other long-lived holder caching lookups that survive between stage runs. The consequence of the monolith is one lock window covering the whole chain and one rollback that discards all prior work when any step fails; the consequence of the shared cache is that stages which should be independent are coupled, so they can no longer be retried or reordered in isolation. The fix is one Job Queue entry per stage behind an interface, Status as the cursor, and per-run-only caching. See `split-multi-step-flows-into-staged-job-queue-entries.bad.al`.
## See also
- `park-long-running-work-on-a-status-url.md`
- `stage-every-integration-message.md`
- `use-a-framing-record-for-inbound-polling.md`

47
custom/knowledge/integration/stage-every-integration-message.bad.al Normal file
View file

@ -0,0 +1,47 @@
// Anti-pattern: the inbound API insert trigger enriches and posts inline.
// The HTTP request now holds posting locks and depends on two remote systems
// (the pricing service and the caller) staying responsive. A slow pricing call
// or a posting error surfaces to the caller as a request timeout, and the whole
// transaction rolls back: the message is lost, with no staged row to retry.
page 50100 "Sales Order Intake API"
{
PageType = API;
APIPublisher = 'contoso';
APIGroup = 'integration';
APIVersion = 'v1.0';
EntityName = 'salesOrderIntake';
EntitySetName = 'salesOrderIntakes';
SourceTable = "Sales Header";
DelayedInsert = true;
layout
{
area(Content)
{
repeater(Group)
{
field(externalNo; Rec."External Document No.") { }
field(sellToCustomerNo; Rec."Sell-to Customer No.") { }
}
}
}
trigger OnInsertRecord(BelowxRec: Boolean): Boolean
var
SalesPost: Codeunit "Sales-Post";
PricingClient: Codeunit "External Pricing Client";
begin
// BAD: a second remote call, inside the request that is creating the row.
// If the pricing service is slow, the caller's HTTP request blocks on it.
Rec.Validate("Unit Price", PricingClient.GetPrice(Rec."No."));
// BAD: posting inline, inside the request transaction. Posting locks are
// held for the whole HTTP round trip. A posting error rolls back the
// insert too, so there is nothing left to inspect or retry.
SalesPost.Run(Rec);
// There is no staging row. A duplicate delivery (the source retried after
// a timeout) is processed again from scratch, creating a second order.
end;
}

99
custom/knowledge/integration/stage-every-integration-message.good.al Normal file
View file

@ -0,0 +1,99 @@
// Best practice: a single staging table, a thin acceptance endpoint, and a
// background processor. The endpoint only validates and stages; it never posts
// and never calls the source system back. The Job Queue codeunit does the real
// work later, decoupled from the caller and from the remote system's uptime.
// --- The spine: one staging table for every inbound and outbound message ---
table 50100 "Integration Message"
{
DataClassification = CustomerContent;
fields
{
field(1; "Message ID"; Guid) { Caption = 'Message ID'; }
field(2; Direction; Enum "Integration Direction") { Caption = 'Direction'; }
// Type drives the dispatcher below. A new message kind is a new branch,
// not a new published API page.
field(3; "Type"; Code[40]) { Caption = 'Type'; }
field(4; Status; Enum "Integration Status") { Caption = 'Status'; }
// The source system's stable id. Drives inbound de-duplication, so it
// carries a unique key, never the internal Message ID.
field(5; "External Reference"; Text[100]) { Caption = 'External Reference'; }
field(6; "Correlation ID"; Code[40]) { Caption = 'Correlation ID'; }
field(10; Request; Blob) { Caption = 'Request'; }
field(11; Response; Blob) { Caption = 'Response'; }
field(20; "Error Message"; Text[2048]) { Caption = 'Error Message'; }
field(21; "Retry Count"; Integer) { Caption = 'Retry Count'; }
}
keys
{
key(PK; "Message ID") { Clustered = true; }
// The work key the Job Queue queries: which rows still need processing.
key(Work; Status, Direction) { }
// The idempotency key: detect a replayed inbound message at insert time.
key(Idempotency; "External Reference", "Type") { }
}
}
// --- Phase one: acceptance. Validate, stage, return. No posting here. ---
page 50100 "Integration Message API"
{
PageType = API;
APIPublisher = 'contoso';
APIGroup = 'integration';
APIVersion = 'v1.0';
EntityName = 'integrationMessage';
EntitySetName = 'integrationMessages';
SourceTable = "Integration Message";
DelayedInsert = true;
layout
{
area(Content)
{
repeater(Group)
{
field(externalReference; Rec."External Reference") { }
field(type; Rec.Type) { }
field(request; Rec.Request) { }
}
}
}
trigger OnInsertRecord(BelowxRec: Boolean): Boolean
begin
// The only work the endpoint does: stamp identity and mark the row New.
// Everything expensive happens later, in the Job Queue processor.
Rec."Message ID" := CreateGuid();
Rec.Direction := Rec.Direction::Inbound;
Rec.Status := Rec.Status::New;
end;
}
// --- Phase two: processing. Runs as a Job Queue entry, reads staged rows. ---
codeunit 50101 "Inbound Message Processor"
{
TableNo = "Job Queue Entry";
trigger OnRun()
var
IntegrationMessage: Record "Integration Message";
begin
// Read by Status, never from an HTTP call. The caller is long gone.
IntegrationMessage.SetRange(Direction, IntegrationMessage.Direction::Inbound);
IntegrationMessage.SetRange(Status, IntegrationMessage.Status::New);
if IntegrationMessage.FindSet() then
repeat
Dispatch(IntegrationMessage);
until IntegrationMessage.Next() = 0;
end;
// The Type field routes to the right handler. No giant CASE in the endpoint.
local procedure Dispatch(var IntegrationMessage: Record "Integration Message")
begin
// ... resolve a handler by IntegrationMessage.Type and run it; on
// success set Status::Resolved, on failure stamp Error Message and
// bump Retry Count so the row stays auditable and re-runnable.
end;
}

34
custom/knowledge/integration/stage-every-integration-message.md Normal file
View file

@ -0,0 +1,34 @@
---
bc-version: [all]
domain: integration
keywords: [integration, staging, integration-message, webhook, posting, decoupling]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Stage every integration message
## Description
Every message that crosses the Business Central boundary, inbound or outbound, should be written to one staging table (the Integration Message) before any business work runs against it. Posting, document creation, and notification then operate on staged data, never on a live external call. This is the single most load-bearing rule in BC integration design because it decouples the local transaction from the availability and latency of a system you do not control: an external outage delays processing, it never breaks posting or forces a rollback. A flow that skips staging couples a database transaction to a remote endpoint, so a slow or failed remote call surfaces inside BC as a request timeout, a lock held too long, or a half-finished document.
The Integration Message is a normal table whose rows carry everything a processor needs to act without calling back to the source: the external reference, the message type, a status, the request and response payloads, the correlation id, and the retry state. Inbound and outbound rows share the table and are told apart by a Direction field.
## Best Practice
Treat staging as a two-phase split. Phase one is acceptance: a webhook receiver or an API page validates the payload, writes one Integration Message row, and returns. It does no posting and makes no second remote call. Phase two is processing: a background Job Queue codeunit reads rows by Status and does the real work, fully decoupled from the original caller.
Expose the Integration Message as a single API page and let a Type field route each row to the correct dispatcher codeunit, rather than versioning a separate endpoint per source system. One endpoint plus a Type-driven dispatcher means a new message kind is a new dispatcher branch, not a new published API surface. Keep an idempotency key on the external reference so a replayed delivery is detected at insert time rather than processed twice. See `stage-every-integration-message.good.al` for the intake page, the staging table shape, and the Job Queue processor.
## Anti Pattern
A webhook handler, an API page insert trigger, or a Job Queue poll handler that posts a document inline, or that calls the external service again to enrich the message before returning. Both couple the request to live database locks and to the remote system staying up.
Detection signal for a reviewer or agent: an HTTP-triggered handler or an `OnInsertRecord`/`OnModifyRecord` trigger on an API page that calls a posting codeunit (`Codeunit "Sales-Post"`, `OnAfterPostSalesDoc`, and similar) or `HttpClient.Send` directly, instead of `Insert`-ing an Integration Message row and returning. The fix is structural: move the post and the callout into the Job Queue processor that reads staged rows. See `stage-every-integration-message.bad.al`.
## See also
- `accept-async-work-instead-of-synchronous-wait-loops.md`
- `never-call-external-services-from-posting.md`
- `deduplicate-inbound-messages-with-an-idempotency-check.md`

27
custom/knowledge/integration/use-a-framing-record-for-inbound-polling.bad.al Normal file
View file

@ -0,0 +1,27 @@
// Anti-pattern: no framing record, no watermark, no lock. Every run fetches the whole
// collection, and two overlapping Job Queue runs stage the same records twice.
codeunit 50132 "Inbound Poller Bad"
{
TableNo = "Job Queue Entry";
procedure Poll()
var
Client: HttpClient;
Response: HttpResponseMessage;
begin
// BAD: "fetch all". No last-fetch datetime feeds the request and no window cap bounds it,
// so the cost of every poll grows with the TOTAL data set, not with what is new. Records
// that were already staged and resolved are pulled again and reprocessed every run.
Client.Get('https://svc.contoso.com/api/orders', Response);
// BAD: no lock. The Job Queue can start the next run before this one finishes (a run that
// overruns its recurrence interval overlaps the following one). Both runs fetch the full
// collection concurrently and BOTH stage every order, so each order lands twice and becomes
// a duplicate document downstream.
StageAll(Response);
// There is also no watermark to advance, so even back-to-back runs cannot narrow their
// windows: there is no notion of "where we left off" anywhere in this design.
end;
}

78
custom/knowledge/integration/use-a-framing-record-for-inbound-polling.good.al Normal file
View file

@ -0,0 +1,78 @@
// Best practice: one framing record per feed. It remembers where the last run stopped,
// caps how much one run pulls, and carries a lock with a stale timeout so two overlapping
// Job Queue runs can never fetch the same window.
table 50130 "Inbound Feed Frame"
{
DataClassification = SystemMetadata;
fields
{
field(1; "Feed Code"; Code[20]) { Caption = 'Feed Code'; }
// The watermark: the next run starts here, so no run ever re-pulls old data.
field(2; "Last Fetch At"; DateTime) { Caption = 'Last Fetch At'; }
// The cap: a long-quiet feed catches up over several bounded runs instead of one huge pull.
field(3; "Max Window (Hours)"; Integer) { Caption = 'Max Window (Hours)'; }
// Opaque continuation token from the source's paged API, when it offers one.
field(4; "Cursor"; Text[250]) { Caption = 'Cursor'; }
// The lock: a second concurrent run sees this set and backs off.
field(5; "Locked"; Boolean) { Caption = 'Locked'; }
// Stamped when the lock is taken, so a crashed run's lock can be reclaimed after a timeout.
field(6; "Locked At"; DateTime) { Caption = 'Locked At'; }
}
keys { key(PK; "Feed Code") { Clustered = true; } }
}
codeunit 50131 "Inbound Poller"
{
TableNo = "Job Queue Entry";
procedure Poll(FeedCode: Code[20])
var
Frame: Record "Inbound Feed Frame";
WindowEnd: DateTime;
begin
Frame.Get(FeedCode);
// Acquire the lock first. If another run already owns this feed, exit quietly:
// overlap is the whole problem we are preventing.
if not TryAcquireLock(Frame) then
exit;
// Bounded incremental window: from the watermark up to a capped end. Never "fetch all".
WindowEnd := CapWindow(Frame."Last Fetch At", Frame."Max Window (Hours)");
FetchAndStage(Frame, Frame."Last Fetch At", WindowEnd);
// Advance the watermark and cursor so the NEXT run starts exactly where this one ended.
// Sequential runs therefore never overlap their windows.
Frame."Last Fetch At" := WindowEnd;
Frame."Cursor" := NextCursor();
ReleaseLock(Frame);
end;
local procedure TryAcquireLock(var Frame: Record "Inbound Feed Frame"): Boolean
begin
// If the lock is held AND fresh, someone is actively polling: do not steal it.
if Frame."Locked" and (CurrentDateTime() - Frame."Locked At" < GetStaleTimeoutMs()) then
exit(false);
// Either free, or stale (a previous run crashed without releasing). Reclaim it.
Frame."Locked" := true;
Frame."Locked At" := CurrentDateTime();
Frame.Modify(true);
Commit(); // make the lock durable before the long fetch starts
exit(true);
end;
local procedure ReleaseLock(var Frame: Record "Inbound Feed Frame")
begin
Frame."Locked" := false;
Frame.Modify(true);
end;
local procedure GetStaleTimeoutMs(): Integer
begin
exit(300000); // 5 minutes: longer than a healthy run, short enough to recover from a crash
end;
}

32
custom/knowledge/integration/use-a-framing-record-for-inbound-polling.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [polling, framing-record, cursor, lock, stale-lock, incremental-window, overlap]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Use a framing record for inbound polling
## Description
When Business Central polls an external paged API on a schedule, two facts about the schedule create problems that a naive poll handler ignores. First, the handler needs durable memory of where it left off, because the Job Queue run that fetched the last window is gone by the time the next one starts. Second, the Job Queue can overlap: a run that takes longer than its recurrence interval is still working when the next run begins, so two runs are live at once. A framing record is the small per-feed table that solves both: it holds the last fetch datetime, a maximum window size, an optional cursor token, and a lock flag with a stale-lock timeout.
Without a framing record a poll fails in one of two ways, both of which get worse the busier the feed is. A handler with no last-fetch memory re-fetches the entire collection every run, so the cost of a poll grows with the total data set rather than with what is new, and resolved messages are restaged and reprocessed. A handler with no lock lets overlapping runs fetch the same window concurrently, so the same records are staged twice and downstream they become duplicate documents. The fix is the same record in both cases: bounded windows fix re-fetching, the lock fixes overlap.
## Best Practice
Keep one framing record per inbound feed and drive every poll through it. At the start of a run, acquire the lock, honouring a stale-lock timeout so a run that crashed without releasing the lock does not wedge the feed forever; if the lock is held and fresh, the run exits and lets the holder finish. Then compute a bounded window from the last fetch datetime up to a capped end (never an open-ended "everything since"), fetch exactly that window, advance the cursor and last fetch datetime, and release the lock. The mechanism that prevents double-staging is the lock plus the advancing watermark: the second overlapping run sees the lock held and backs off, and even sequential runs never overlap their windows because each one starts where the previous one's watermark ended. See `use-a-framing-record-for-inbound-polling.good.al`.
The window cap is a deliberate trade-off: capping the end datetime means a feed that has been quiet for a long time catches up over several runs rather than pulling a huge window in one go, which keeps each run bounded and each lock window short. Pair this with idempotent staging (see `deduplicate-inbound-messages-with-an-idempotency-check.md`) so that even a window boundary that overlaps slightly cannot create duplicates.
## Anti Pattern
A poll handler that fetches the full collection every run, or that has no lock so concurrent Job Queue runs fetch and stage the same records. The detection signal: a polling codeunit whose `HttpClient` call has no last-fetch datetime or cursor feeding the request (a bare "get all"), no cap on the requested window, or no lock acquisition guarding the fetch. The consequence of fetch-all is wasted work that scales with the whole data set and reprocessing of already-resolved messages; the consequence of a missing lock is duplicate staging whenever two runs overlap, which surfaces downstream as duplicate documents. The fix is a per-feed framing record with a watermark, a window cap, and a stale-aware lock. See `use-a-framing-record-for-inbound-polling.bad.al`.
## See also
- `deduplicate-inbound-messages-with-an-idempotency-check.md`
- `stage-every-integration-message.md`
- `split-multi-step-flows-into-staged-job-queue-entries.md`

30
custom/knowledge/integration/version-business-events-and-keep-payloads-stable.bad.al Normal file
View file

@ -0,0 +1,30 @@
// Anti-pattern: mutating a published signature, passing the whole record plus a secret, and
// firing without validation or failure classification. Every external subscriber breaks, and
// credentials and every table field leak into the payload.
codeunit 50173 "Order Events Bad"
{
// This event already shipped as OnOrderConfirmed(OrderNo: Code[20]). Subscribers bound to
// that signature. Editing it IN PLACE to add parameters silently breaks all of them: there
// is no compiler across the boundary, so the notification just starts arriving in the wrong
// shape and processing fails on the far side, far from this change.
[ExternalBusinessEvent('OrderConfirmed', 'Order confirmed', 'Raised on confirm', EventCategory::Sales)]
procedure OnOrderConfirmed(var SalesHeader: Record "Sales Header"; ApiKey: Text)
// BAD: the full record exposes every field of Sales Header to every subscriber and couples
// the contract to the table layout. ApiKey leaks a secret credential into the payload.
begin
end;
}
codeunit 50174 "Order Publisher Bad"
{
procedure Publish(SalesHeader: Record "Sales Header")
var
Events: Codeunit "Order Events Bad";
begin
// BAD: no validation. A header with no document number is published and then, because
// the data is permanently invalid, retried by the platform indefinitely. There is no
// transient/permanent classification, so an unfixable payload is treated like a blip.
Events.OnOrderConfirmed(SalesHeader, GetSecretApiKey());
end;
}

42
custom/knowledge/integration/version-business-events-and-keep-payloads-stable.good.al Normal file
View file

@ -0,0 +1,42 @@
// Best practice: one events codeunit per version, versioned procedure names, a minimal
// stable DTO of identifiers, validation before firing, and transient/permanent classification.
codeunit 50170 "Order Events v1"
{
// Shipped contract. Once a subscriber binds to OnOrderConfirmed_v1, this signature is FROZEN.
// Parameters are identifiers only: a subscriber calls back for detail, so no field leaks and
// no secret travels in the payload.
[ExternalBusinessEvent('OrderConfirmed', 'Order confirmed', 'Raised when a sales order is confirmed', EventCategory::Sales)]
procedure OnOrderConfirmed_v1(OrderNo: Code[20]; ExternalRef: Text[100])
begin
end;
}
codeunit 50171 "Order Event Publisher"
{
procedure Publish(SalesHeader: Record "Sales Header")
var
Events: Codeunit "Order Events v1";
begin
// Validate BEFORE firing. A header with no number cannot produce a meaningful
// notification, so this is a PERMANENT failure: fail and alert, do not publish it and
// let the platform retry an unfixable payload for 36 hours.
if SalesHeader."No." = '' then
Error('Cannot publish OrderConfirmed without a document number');
// Transient conditions (subscriber temporarily down, network blip) are NOT handled here:
// the platform's external-event delivery retries those for us. We only guard permanent ones.
Events.OnOrderConfirmed_v1(SalesHeader."No.", SalesHeader."External Document No.");
end;
}
// A breaking change does NOT edit OnOrderConfirmed_v1. It ships a NEW codeunit with a NEW
// procedure, so v1 subscribers keep receiving exactly what they bound to and new subscribers
// opt into the richer v2 shape.
codeunit 50172 "Order Events v2"
{
[ExternalBusinessEvent('OrderConfirmedV2', 'Order confirmed (v2)', 'Adds the warehouse location code', EventCategory::Sales)]
procedure OnOrderConfirmed_v2(OrderNo: Code[20]; ExternalRef: Text[100]; LocationCode: Code[10])
begin
end;
}

32
custom/knowledge/integration/version-business-events-and-keep-payloads-stable.md Normal file
View file

@ -0,0 +1,32 @@
---
bc-version: [all]
domain: integration
keywords: [business-events, versioning, dto, payload-contract, validate, transient-permanent, breaking-change]
technologies: [al]
countries: [w1]
application-area: [all]
---
# Version business events and keep payloads stable
## Description
A business event payload is a published contract. Once an external subscriber has bound to an event's name and signature, that signature is no longer yours to change quietly: adding a parameter, reordering parameters, or retyping one changes the shape the subscriber receives, and because external subscribers live outside your app there is no compiler to catch the break. The notification simply starts arriving in a shape the consumer does not expect, and the failure surfaces as malformed data or dropped processing on the far side, often long after the change shipped and far from the code that caused it.
Two disciplines keep the contract honest. The first is versioning: treat a shipped event signature as frozen, and express any change as a new procedure or a new codeunit rather than an edit to the old one, so existing subscribers keep receiving exactly what they bound to while new subscribers opt into the new shape. The second is payload hygiene: the payload must be a minimal, stable DTO of identifiers and just enough context, never the raw BC record (which exposes every field and couples the contract to the table layout) and never a secret such as an API key or token (which leaks credentials to every subscriber). A payload that must also be validated before firing, so invalid data is never published and then retried forever.
## Best Practice
Put all events for one integration version in a single events codeunit, and give a second version its own codeunit. Name each procedure with its version suffix (`OnOrderConfirmed_v1`) so the version is visible at the call site and a new version sits beside the old one rather than replacing it. Pass a small DTO of identifiers so a subscriber can call back for detail without the payload leaking fields or secrets. Validate the payload before firing, so a record that cannot produce a meaningful notification fails fast rather than being published and retried indefinitely. Classify failures as transient (network, subscriber temporarily down: let the platform retry) versus permanent (invalid data, a malformed payload: fail, alert, and consider a dead-letter path) so a permanent error is not retried for 36 hours as if it were a blip. See `version-business-events-and-keep-payloads-stable.good.al`.
The trade-off is more codeunits over time as versions accumulate, which is the correct cost: a stable contract for existing consumers is worth more than a tidy single signature that silently breaks them.
## Anti Pattern
Adding or reordering parameters on an already-published event, passing the whole BC record or secret-bearing fields as the payload, or firing without validating first. The detection signal: an edit to the signature of an existing `[ExternalBusinessEvent]` or `[BusinessEvent]` that has already shipped, a parameter typed as a full table record (`var Rec: Record ...`) on an event, a payload field that holds a key or token, or a fire with no prior validation and no transient-versus-permanent classification of the failure. The consequence is silently broken subscribers (signature change), leaked data or credentials (record or secret payload), and infinite retry of unfixable data (no validation). The fix is a new versioned procedure or codeunit, a minimal identifier DTO, validation before firing, and explicit failure classification. See `version-business-events-and-keep-payloads-stable.bad.al`.
## See also
- `prefer-business-events-over-handwritten-retry-loops.md`
- `monitor-external-event-subscription-health.md`
- `send-an-idempotency-key-on-every-outbound-call.md`