DASchema with a self-reference massively slows down the ApplyUpdates process

Hello,

I have discovered a performance issue in the ApplyUpdates process of the DataAbstract framework. If the schema contains a self-reference the SynchronizeAutoIncs() method is called for each master record and processes all detail records of all master records. This causes a major performance problem starting at several dozens of records as the complexity is cubic.

I have prepared a small sample project to demostrate the issue. Just run it, click on the “Fill Orders” button. It will generate 25 new master records with 12 detail records for each master. The ApplyUpdate on this takes more than a minute! The same problem occurs when the records are only mass-changed (imagine a flag for the order item or a new price-list).

At first sight it seems like, only the currently stored master record (the one with the last generated AutoInc) should be processed when SynchronizeAutoIncs() is called here (all records needs to be processed for regular master-detail relations of course). Other optimization opportunity seems to be when the self-reference field is not changed in the Delta. The “masterval” is equal to the “oldmasterval” and the whole processing of the detail delta seems redundant. Is that correct?

Regard from Prague,
Jaroslav

daSample.zip (685.5 KB)

Thanks, logged as bugs://79315

bugs://79315 got closed with status fixed.

Nowadays it generates

[Window Title]
Information

[Content]
Generated 25 Orders!
Generating with UI took: 452ms.
Applying updates took: 1232ms.

[OK]

update TDABusinessProcessor.SynchronizeAutoIncs as

source code
procedure TDABusinessProcessor.SynchronizeAutoIncs(const aMasterDelta, aDetailDelta : IDADelta;
                                  const aRelationship : TDADatasetRelationship);
var x, k, z: integer;
    masterds : TDADataset;
    masterfields, detailfields : TStringList;
    oldmasterval, masterval, detailval : Variant;
    m_index,d_index: array of integer;
begin
  CheckProperties;
  masterfields := TStringList.Create;
  masterfields.Delimiter := ';';

  detailfields := TStringList.Create;
  detailfields.Delimiter := ';';

  try
    masterds := Schema.Datasets.DatasetByName(aMasterDelta.LogicalName);

    masterfields.DelimitedText := aRelationship.MasterFields;
    SetLength(m_index, masterfields.Count);
    for x := 0 to (masterfields.Count-1) do
      m_index[x] := aMasterDelta.IndexOfLoggedField(masterfields[x]);

    detailfields.DelimitedText := aRelationship.DetailFields;
    SetLength(d_index, detailfields.Count);
    for x := 0 to (detailfields.Count-1) do
      d_index[x] := aDetailDelta.IndexOfLoggedField(detailfields[x]);

    for x := 0 to (aMasterDelta.Count-1) do begin
      if not (aMasterDelta[x].Status=csResolved) then Continue;

      for k := 0 to (masterfields.Count-1) do begin
        if not (masterds.FieldByName(masterfields[k]).DataType in [datAutoInc, datLargeAutoInc, datInteger, datLargeInt]) then Continue;

        masterval := aMasterDelta[x].NewValues[m_index[k]];
        oldmasterval := aMasterDelta[x].OldValues[m_index[k]];

        for z := 0 to (aDetailDelta.Count-1) do begin
          if not (aDetailDelta[z].ChangeType in [ctInsert,ctUpdate]) then Continue;

          detailval := aDetailDelta[z].NewValues[d_index[k]];
          if (detailval<>oldmasterval) then Continue;

          aDetailDelta[z].NewValues[d_index[k]] := masterval;
          aDetailDelta[z].RefreshedByServer := TRUE;
        end;
      end;
    end;
  finally
    masterfields.Free;
    detailfields.Free;
  end;
end;

Hello Evgeny,

your fix is working well… thanks for quick response.

Regards,
Jaroslav