Issue with DataTable streaming from delphi to .NET

Data Abstract
,
1

I am streaming data but I notice that sometimes no rows are received.
The 18 columns from the schema are correctly detected

Can you analyse the attached binary data and tell me how many rows are in there?
The data buffer is ~30KB so I think the rows should be in there
I have tried reading in .NET using 2 methods (see below)

    var streamer = new RemObjects.DataAbstract.Bin2DataStreamer(tabledata, RemObjects.DataAbstract.StreamerInitialization.ReadFromBeginning);
          var dataTable = new System.Data.DataTable(queryinfo.tablename);
          streamer.ReadDataTable(queryinfo.tablename, dataTable, true, true);
          var deltas = new RemObjects.DataAbstract.Delta(dataTable);
          streamer.ReadDelta(deltas);
          streamer.FinalizeStreamer();

and

          var arrData = tabledata.ToArray();
          var stream = new System.IO.MemoryStream(tabledata.ToArray());
          System.IO.File.WriteAllBytes("testrotabledata.bin", arrData);
          streamer = new RemObjects.DataAbstract.Bin2DataStreamer();
          streamer.ReadDataTable(stream, queryinfo.tablename, dataTable, true, true);
          streamer.FinalizeStreamer();

FYI: when I set a breakpoint after the first FinalizeStreamer the streamer Position is only 13765

testrotabledata.zip (2.2 KB)

FYI: I write with

    streamer:=TDABin2DataStreamer.Create(nil);
    try
      tabledata:=Binary.Create();
      streamer.SendReducedDelta:=True;  // FVC : Does not work
      streamer.Initialize(tabledata,aiWrite);
      DataTable.LogicalName:=queryinfo.tablename;   // FVC: Needed for fetched tables (DataTransfer) because the LogicalName was 'Table'
      streamer.WriteDataset(DataTable,[woSchema,woRows],-1); // FVC: WriteDataset without stream as first argument does not call Initialize and Finalize
      if not options.Contains(OPTCLOUDDATA_INSERTONLY) then
      begin
        //DataTable.Delta.GetDelta().ReducedDelta:=True;
        //DataTable.HasReducedDelta:=True;
        //streamer.DeltaCount
        streamer.WriteDelta(DataTable);              // FVC: 03/2022 At present deltas are not supported yet, see https://talk.remobjects.com/t/upload-datatable-to-server-from-delphi-to-net/26374/9
      end;
      //FVC: This WriteDataset calls Initialize and Finalize        streamer.WriteDataset(tabledata,DataTable,[woSchema,woRows],-1);
    finally
      streamer.Finalize();
      FreeAndNil(streamer);
    end;
3

Hi,

you have:

  • Datasets:
    • [0] DataTransfer - 0 rows
  • Deltas
    • [0] Table - 10 delta changes

Note: you can investigate content of file by yourself in Delphi checking

  • streamer.DatasetCount / streamer.DatasetNames
  • streamer.DeltaCount / streamer.DeltaNames
4

So there are 0 rows in the 30kb binary data I send, correct?
So what you are saying is that the stream writer is doing something wrong.
This is very unexpected.

When I step through my writer code te datatable contains 10 rows and the tablename is ‘DataTable’.

Is there a bug in my writer code?
Is it allowed to do a WriteDataset followed by a WriteDelta?
How can we proceed to investigate the issue?
My writer code is below

streamer:=TDABin2DataStreamer.Create(nil);
try
  tabledata:=Binary.Create();
  streamer.SendReducedDelta:=False;  // FVC : True does not work
  streamer.Initialize(tabledata,aiWrite);
  DataTable.LogicalName:=queryinfo.tablename;   // FVC: Needed for fetched tables (DataTransfer) because the LogicalName was 'Table'
  streamer.WriteDataset(DataTable,[woSchema,woRows],-1); // FVC: WriteDataset without stream as first argument does not call Initialize and Finalize
  if not options.Contains(OPTCLOUDDATA_INSERTONLY) then
  begin
    //DataTable.Delta.GetDelta().ReducedDelta:=True;
    //DataTable.HasReducedDelta:=True;
    //streamer.DeltaCount
    streamer.WriteDelta(DataTable);              // FVC: 03/2022 At present deltas are not supported yet, see https://talk.remobjects.com/t/upload-datatable-to-server-from-delphi-to-net/26374/9
  end;
  //FVC: This WriteDataset calls Initialize and Finalize        streamer.WriteDataset(tabledata,DataTable,[woSchema,woRows],-1);
finally
  streamer.Finalize();
  FreeAndNil(streamer);
end;
5

Hi,

Stream contains fields declarations, 0 rows and 10 delta changes.

looks like you haven’t call table.First; before saving table to stream.
btw, you can use table.SaveToStream for saving table to stream.

review that code and you find some important things can have influence like current position, filtering, etc

6

This works, thanks a lot

1 Like