alguidelines.dev - Business Central Design Patterns – Queries

Docs: SELECT DISTINCT with Queries

Mon, 01 Jan 0001 00:00:00 +0000

By Bogdana Botez, at Microsoft Development Center Copenhagen

Abstract

This pattern explains how to perform SELECT DISTINCT by using queries in Dynamics NAV.

Description

When working with tables, sometimes a developer needs to perform a SELECT DISTINCT (also known as SELECT UNIQUE) from a table. As NAV does not provide this out of the box, we present below a way to select unique records by using queries.

Problem statement

Let’s consider the VAT Entry table as below:

The goal is to select one line for each separate document that produced VAT Entries. In other words, we want records grouped by Type, Document Type and Document No.. However, if there are multiple lines with the same value of the triad Type, Document Type and Document No. in the VAT Entry table, we only want to see one of them.

Solution

Create a new query object VAT Entry Distinct Document No., with a single DataItem sourced from VAT Entry table. Add the three desired group-by fields Type, Document Type and Document No. as columns.

To enable grouping, add one more column, with Method Type = Totals. This will automatically set the Group By checkbox to TRUE on the three precedent fields.

Note that the Group By field is read-only and trying to set it by hand will clarify that:

Running the query yields a single record per document. You can notice in the second line below for example, how the sales invoice number 103001 had 2 VAT Entries, but it shows up only once in the query:

Example

One thing is to be noted: there is a limitation to how much information you can take out from the records. For example, if we need to extract more information than just the one we already have in the columns, then the following apply: adding one more column of Method Type = None will indeed show more information, but it might affect the grouping. More details below.

The grouping is affected for example when the additional column is the VAT Entry No. In this case, this column brings up additional grouping criteria and one group for each entry number will end up being created.

You can notice that both lines of document 103001 are not visible, which contradicts the goal of SELECT DISTINCT.

The grouping is not affected when the additional column does not influence grouping (has variation identical with one of the other existing columns). For example, adding Posting Date as a new column, does not change anything because posting date is the same for all lines of a document, so the query result is identical with the initial one:

Below is the result of this query, where you can notice that the initial grouping is preserved and correct. Bonus, we can now read the Posting Date of the document too.

The grouping is also not affected when adding more columns of Method Type = Totals. However, this can only be done with columns of Decimal data type.

NAV Usages

This pattern is used in Query 19: VAT Entries Base Amt. Sum. This query is used in Report 19: VAT- VIES Declaration Tax Auth.

The pattern is also used in Query 1511: User IDs by Notification Type.

A variation using loops could be described, for C/AL development on NAV 6 where query objects are not available.

Docs: Use Queries to Detect Duplicate Records

Mon, 01 Jan 0001 00:00:00 +0000

Originally by Abshishek Ghosh and Bogdan Sturzoiu at Microsoft Development Center Copenhagen

Abstract

This pattern uses queries to create an efficient way to detect duplicate entries in a table. This is, for example, useful when trying to find out which customers or contacts have the same names, so we can merge them later.

Description

Duplicate detection has several requirements in Microsoft Dynamics NAV. One method to eliminate duplication is by defining the relevant field as the primary key. However, this method is not always practical either due to the size of the field or due to business requirements that dictate how duplicates are detected but not necessarily how they are eliminated. An example of this method is to detect contacts with the same name and take action to merge them if they are.

Before Dynamics NAV 2013, the only possibility was to iterate through the table in a loop and then create a sub-loop where another instance of the same table is filtered to check for duplicates. For example, to check for duplicate names in the Customer table, the code would look like this:

PROCEDURE HasDuplicateCustomers@26() : Boolean;
VAR
 Customer@1000 : Record 18;
 Customer2@1001 : Record 18;
BEGIN
 IF Customer.FINDSET THEN
 REPEAT
 Customer2.SETRANGE(Name,Customer.Name);
 IF Customer2.COUNT \> 1 THEN
 EXIT(TRUE);
 UNTIL Customer.NEXT = 0;
 EXIT(FALSE);
END;

This code would involve setting filters on the Customer table as many times as there are records in the table. This is an expensive operation.

Starting with Dynamics NAV 2013, we can use queries to create a more efficient implementation of the same logic.

Usage

The solution involves that you create a query to return duplicates, and then invoke it from a method that would test the value of the query to identify if duplicates were found.

Step 1 – Creating the Query

The query must be created with the table we want to search in as the dataitem.
The field we want to search for must be created as a grouped field.
Create a totaling field on the count, and add a filter for Count > 1. This ensures that only records with more than one instance of the field that we selected in the previous step are included in the query result.

Continuing with our Customer Name example, here is how the query would look:

ELEMENTS
{
{ 1 ; ;DataItem; ;
DataItemTable=Table18 }
{ 2 ;1 ;Column ; ;
DataSource=Name }
{ 3 ;1 ;Column ; ;
ColumnFilter=Count_=FILTER(\>1);
MethodType=Totals;
Method=Count }
}

Step 2 – Invoking the Query to Check for Duplicates

Now that the query is created, all we need to do is to invoke the query and check if any records are returned, which would mean that there are duplicates.

Here is an alternate implementation of the HasDuplicateCustomers method using the query that we created:

PROCEDURE HasDuplicateCustomersWithQuery@27() : Boolean;
VAR
 CustomerDuplicate@1000 : Query 70000;
BEGIN
 CustomerDuplicate.OPEN;
 EXIT(CustomerDuplicate.READ);
END;

Examples

Acc. Sched. Chart Management codeunit (762),
methods CheckDuplicateAccScheduleLineDescription and CheckDuplicateColumnLayoutColumnHeader
Analysis Report Chart Mgt. codeunit (770),
methods CheckDuplicateAnalysisLineDescription and CheckDuplicateAnalysisColumnHeader

Consequences

A new query object is needed for every type of duplicate check. This could easily explode and create a maintenance problem.

Docs: Use Queries to Replace Nested Loops

Mon, 01 Jan 0001 00:00:00 +0000

Originally by Bogdan Sturzoiu, Microsoft Development Center Copenhagen

Abstract

This pattern shows how the new query object type introduced in NAV 2013 allows you to replace costly loops when inspecting data from two or more tables.

Description

One of the core operations in a relational database is joining two or more tables. For example, you might need to extract all sales lines in the database together with information regarding the related sales header. This requires joining the Sales Header and Sales Line tables using Sales Header No. as the connecting field.;

The join operation has traditionally been done in C/AL by record looping. When NAV 2013 introduced the query object, it allowed us to produce a data set that is the result of a join operation between two or more tables. This simplifies the problem of finding related records in two tables linked through a foreign key.

Pattern Elements

Two or more tables that contain records linked through a foreign key: Table 1, Table 2, Table n.
A query object Query X, that joins Table 1, Table 2, etc. based on the connecting key.
A processing codeunit that loops through the query records (or any other code-bearing object).

Pattern Steps

Run the query on the connected tables.
Loop through the records returned by the query.
Process the records.

Figure 1. The pattern elements

Usage

Sample Problem

The Bank Acc. Reconciliation Line table (274) and the Bank Account Ledger Entry table (271) are connected through the Bank Account No. field. Identify the matching pairs of records based on having the same remaining amount and transaction date.

Solution Using Nested Loops

The classic C/AL approach is to:

Set the necessary filters on the left table, i.e. table 274.
Loop through the filtered records.
For each record in the filter, find the related records in the right table (table 271) and set the required filters on it.
For each pair of records from the left and right table, decide if they are a solution and if so, apply them to each other.

PROCEDURE MatchSingle@5(BankAccReconciliation@1003 : Record 273);
VAR
 BankAccRecLine@1005 : Record 274;
 BankAccLedgerEntry@1006 : Record 271;
 BankAccEntrySetReconNo@1007 : Codeunit 375;
BEGIN
 BankAccRecLine.SETRANGE("Bank Account No.",BankAccReconciliation."Bank Account No.");
 BankAccRecLine.SETRANGE("Statement No.",BankAccReconciliation."Statement No.");
 BankAccRecLine.SETFILTER(Difference,'<>%1',0);
 BankAccRecLine.SETRANGE(Type,BankAccRecLine.Type::"Bank Account Ledger Entry");

 IF BankAccRecLine.FINDSET THEN
 REPEAT
 BankAccLedgerEntry.SETRANGE("Bank Account No.",BankAccRecLine."Bank Account No.");
 BankAccLedgerEntry.SETRANGE(Open,TRUE);
 BankAccLedgerEntry.SETRANGE("Statement Status",BankAccLedgerEntry."Statement Status"::Open);
 BankAccLedgerEntry.SETFILTER("Remaining Amount",'<>%1',0);
 IF BankAccLedgerEntry.FINDSET THEN
 REPEAT
 IF (BankAccRecLine.Difference = BankAccLedgerEntry."Remaining Amount") AND
 (BankAccRecLine."Transaction Date" = BankAccLedgerEntry."Posting Date") THEN
 BankAccEntrySetReconNo.ApplyEntries(
 BankAccRecLine,BankAccLedgerEntry, Relation::"One-to-One");
 UNTIL BankAccLedgerEntry.NEXT = 0;
 UNTIL BankAccRecLine.NEXT = 0;
END;

Solution Using Query

The new query-based approach involves:

Define a query that returns the full filtered join of tables 271 and 274.
Loop through the records returned by the query.
For each query record, decide if it represents a solution and then connect the two table records that formed it through an application.

PROCEDURE MatchSingle@5(BankAccReconciliation@1003 : Record 273);
VAR
 BankRecMatchCandidates@1001 : Query 1252;
 BankAccEntrySetReconNo@1007 : Codeunit 375;
BEGIN
 BankRecMatchCandidates.SETRANGE(Rec_Line_Bank_Account_No,
 BankAccReconciliation."Bank Account No.");
 BankRecMatchCandidates.SETRANGE(Rec_Line_Statement_No,
 BankAccReconciliation."Statement No.");

 IF NOT BankRecMatchCandidates.OPEN THEN
 EXIT;

 WHILE BankRecMatchCandidates.READ DO BEGIN
 BankAccLedgerEntry.GET(BankRecMatchCandidates.Entry_No);
 BankAccRecLine.GET(BankAccRecLine."Statement Type"::"Bank Reconciliation",
 BankRecMatchCandidates.Rec_Line_Bank_Account_No,
 BankRecMatchCandidates.Rec_Line_Statement_No,
 BankRecMatchCandidates.Rec_Line_Statement_Line_No);
 BankAccEntrySetReconNo.ApplyEntries(BankAccRecLine,BankAccLedgerEntry,
 Relation::"One-to-One");
 END;
END;

where the query 1252 is defined as:

OBJECT Query 1252 Bank Rec. Match Candidates
{
 OBJECT-PROPERTIES
 {
 Date=;
 Time=;
 Version List=;
 }
 PROPERTIES
 {
 }
 ELEMENTS
 {
 { 1 ; ;DataItem; ;
 DataItemTable=Table274;
 DataItemTableFilter=Difference=FILTER(<>0),
 Type=FILTER(=Bank Account Ledger Entry) }
 { 2 ;1 ;Column ;Rec_Line_Bank_Account_No;
 DataSource=Bank Account No. }
 { 3 ;1 ;Column ;Rec_Line_Statement_No;
 DataSource=Statement No. }
 { 4 ;1 ;Column ;Rec_Line_Statement_Line_No;
 DataSource=Statement Line No. }
 { 5 ;1 ;Column ;Rec_Line_Transaction_Date;
 DataSource=Transaction Date }
 { 6 ;1 ;Column ;Rec_Line_Difference ;
 DataSource=Difference }
 { 7 ;1 ;DataItem; ;
 DataItemTable=Table271;
 DataItemTableFilter=Remaining Amount=FILTER(<>0),
 Open=CONST(Yes),
 Statement Status=FILTER(Open);
 DataItemLink=Bank Account No.=Bank_Acc_Reconciliation_Line."Bank Account o.",
 Remaining Amount=Bank_Acc_Reconciliation_Line.Difference,
 Posting Date=Bank_Acc_Reconciliation_Line."Transaction Date" }
 { 8 ;2 ;Column ; ;
 DataSource=Entry No. }
 { 9 ;2 ;Column ;Bank_Account_No ;
 DataSource=Bank Account No. }
 { 10 ;2 ;Column ; ;
 DataSource=Posting Date }
 { 11 ;2 ;Column ; ;
 DataSource=Remaining Amount }
 { 12 ;2 ;Column ;Bank_Ledger_Entry_Open;
 DataSource=Open }
 { 13 ;2 ;Column ; ;
 DataSource=Statement Status }
 }
 CODE
 {
 BEGIN
 END.
 }

When comparing the two implementations, we notice the following advantages of using a query instead of two loops:Comparison

A query produces the Cartesian product of tables 1 and 2 faster than by looping through both of them. The advantage grows as there are more tables linked.

The query object leverages the power of SQL Server (as it basically executes a SQL query directly against the database server), and as such it only requires one trip to the database to fetch the data, instead of getting the related records on the right side of the join one by one.

A query is scalable in the sense that it allows reusing its definition by applying filters when looping through its dataset. So a generic query can be reused for a variety of purposes just by adapting the filters to the programming need. Duplicating traditional record loops in different functions, on the other side, requires extensive code copy-pasting, which can introduce code defects.
Using a query only requires one loop, whereas joining two or more tables requires multiple code loops that quickly become difficult to read and follow.
Queries are easy to create and maintain and generally provide a cleaner design.

NAV Specific Examples

In Microsoft Dynamics NAV 2013 R2, we can see the query object used in the bank account reconciliation matching algorithm. The object is query Bank Rec. Match Candidates query (1252), and it is called by the matching engine in the Match Bank Rec. Lines codeunit (1252).

Ideas for Improvement

The query object type could be improved to allow the passing of parameters at runtime, or, in general, being built dynamically at runtime. This will remove the need for multiple static definitions of the same base query used in slightly different contexts.

alguidelines.dev - Business Central Design Patterns – Queries

Docs: SELECT DISTINCT with Queries

Abstract

Solution

Example

NAV Usages

Related Topics

Docs: Use Queries to Detect Duplicate Records

Abstract

Description

Usage

Consequences

Docs: Use Queries to Replace Nested Loops

Abstract

Description

Pattern Elements

Pattern Steps

Usage

Sample Problem

Solution Using Nested Loops

NAV Specific Examples

Ideas for Improvement