DataAbstract Load Large Dataset Out Of Memory Error

Data Abstract
1

Hi,
My Environment: Delphi 2010
RO/DA: 8.1.87.1147
Db:Firebird 2.5
Connection Driver: AnyDAC?AuxDriver=IB;Server=localhost;Database=DATA.FDB;UserID=SYSDBA;Password=masterkey;Protocol=TCPIP;Characterset=utf8;

I need load a Large DataSet as the packet size limit. I use LoadFromRemoteSource(True) to retrieve all data.

But I get error “out of memory” in the middle of loading data.

and debug it stop at

procedure TDAMemDataTable.SetWideString(NativeBuf: Pointer; FldNo: integer;
const Value: Unicodestring);
begin

fMemDataset.SetWideString(NativeBuf, FldNo, Value);
end;

I create a test case show the error.

and call stack of the function for it.

Out-Of_Memory.zip (101.8 KB)

As our system is very large , I just copy the logic and reproduce the case by repeat load the large dataset.

Please help as our end customer is complain about it.

2

you haven’t attached your DB so I can’t reproduce this error :frowning:
according to your table structure you have a lot of string data that consumes memory.
I can suggest to set AutoCompactRecords property (by default it is false).

1 Like
3

Do you need all these fields in the same time on clients?
review our Dynamic Select feature.

4

Hi,
Lucky during the data preparation for the testing sending to you, I find the problem

One of field FixWdideChar The size go to 65536 , which is auto defined by schema Modeler

Please refer to capture schema definition. It should be 1 in size but it go wrong to 65536.

I change to all the size go to 1 then all is correct.

5

I can’t reproduce this problem with simple script like

CREATE TABLE TEST (
    NEW_FIELD  CHAR(1)
);
6

I will check out the structure and send you the script can reproduce the problem later . Thank you