I have discovered a performance issue in the ApplyUpdates process of the DataAbstract framework. If the schema contains a self-reference the SynchronizeAutoIncs() method is called for each master record and processes all detail records of all master records. This causes a major performance problem starting at several dozens of records as the complexity is cubic.
I have prepared a small sample project to demostrate the issue. Just run it, click on the “Fill Orders” button. It will generate 25 new master records with 12 detail records for each master. The ApplyUpdate on this takes more than a minute! The same problem occurs when the records are only mass-changed (imagine a flag for the order item or a new price-list).
At first sight it seems like, only the currently stored master record (the one with the last generated AutoInc) should be processed when SynchronizeAutoIncs() is called here (all records needs to be processed for regular master-detail relations of course). Other optimization opportunity seems to be when the self-reference field is not changed in the Delta. The “masterval” is equal to the “oldmasterval” and the whole processing of the detail delta seems redundant. Is that correct?
[Window Title]
Information
[Content]
Generated 25 Orders!
Generating with UI took: 452ms.
Applying updates took: 1232ms.
[OK]
update TDABusinessProcessor.SynchronizeAutoIncs as
source code
procedure TDABusinessProcessor.SynchronizeAutoIncs(const aMasterDelta, aDetailDelta : IDADelta;
const aRelationship : TDADatasetRelationship);
var x, k, z: integer;
masterds : TDADataset;
masterfields, detailfields : TStringList;
oldmasterval, masterval, detailval : Variant;
m_index,d_index: array of integer;
begin
CheckProperties;
masterfields := TStringList.Create;
masterfields.Delimiter := ';';
detailfields := TStringList.Create;
detailfields.Delimiter := ';';
try
masterds := Schema.Datasets.DatasetByName(aMasterDelta.LogicalName);
masterfields.DelimitedText := aRelationship.MasterFields;
SetLength(m_index, masterfields.Count);
for x := 0 to (masterfields.Count-1) do
m_index[x] := aMasterDelta.IndexOfLoggedField(masterfields[x]);
detailfields.DelimitedText := aRelationship.DetailFields;
SetLength(d_index, detailfields.Count);
for x := 0 to (detailfields.Count-1) do
d_index[x] := aDetailDelta.IndexOfLoggedField(detailfields[x]);
for x := 0 to (aMasterDelta.Count-1) do begin
if not (aMasterDelta[x].Status=csResolved) then Continue;
for k := 0 to (masterfields.Count-1) do begin
if not (masterds.FieldByName(masterfields[k]).DataType in [datAutoInc, datLargeAutoInc, datInteger, datLargeInt]) then Continue;
masterval := aMasterDelta[x].NewValues[m_index[k]];
oldmasterval := aMasterDelta[x].OldValues[m_index[k]];
for z := 0 to (aDetailDelta.Count-1) do begin
if not (aDetailDelta[z].ChangeType in [ctInsert,ctUpdate]) then Continue;
detailval := aDetailDelta[z].NewValues[d_index[k]];
if (detailval<>oldmasterval) then Continue;
aDetailDelta[z].NewValues[d_index[k]] := masterval;
aDetailDelta[z].RefreshedByServer := TRUE;
end;
end;
end;
finally
masterfields.Free;
detailfields.Free;
end;
end;