How to speedup extremely slow TDAMemDataTable.Loadschema / RemoteDataAdapter.FillSchema

How can we speedup our code by speeding up
TAMemDataTable.Loadschema / RemoteDataAdapter.FillSchema
or by reducing the number of calls to TAMemDataTable.Loadschema

Our TDADataTable have LoadSchema in the constructor :sometimes (or always?) we need this (e.g. when parameters are used)

We use a new DARemoteDataAdapter instance per table (because we have multiple TRORemoteServices (from multiple threads)

DARemoteDataAdapter.CacheSchema:=True does not seem to do a lot
The TRORemoteService also has CacheRODL=True which also does not do a lot.

FYI:I remember we discussed this a long time ago when we reported that DA4 was a lot slower than DA3 and then you answered the XML format was the cause.

What is today the best way to speedup our application by reducing the number of LoadSchema calls or by making the calls faster?

FYI: An indication of how slow it is
Opening a window takes 27 seconds, of which 22,166s in 10 loadschema calls

Note: If we know when the call can be completely avoided it might help although this would probably still leave us ugly code.

You’reloadiomg the schema from the remote server, I understand correctly? If so that is never, ever something you will want to to in the main thread (and this, in the constructor of a form) as by definition, any network access will take an indeterministic amount of time that’s too long for min().

What you’ll want to do is move that that downloads into a background thread and sync it back to main once done (possibly showing appropriate UI while it’s in progress).

Hello Marc,

What exactly gets loaded with LoadSchema ? The scheme for the entire service?
I think our main problem is that we load too much (an entire scheme) for each instance of each DA table (=way too many times).

How can we avoid that?
How can we create many DA tables and attach a scheme we downloaded in the background thread instead of calling loadscheme?

The only thing I see now that might work is instead of creating a TDARemoteDataAdapter per TDAMemDataTabe instance is sharing this.
However, we have had issues with this.
At some point we only created one TDARemoteDataAdapter per thread but then we ran into issues with the multiple services (schemas).
Something we did not try yet was a single shared TDARemoteDataAdapter per thread per service.
Will that work? Does this have known issues or caveats?

FYI: I am still interested in speeding up the LoadSchema call itself.
Alternatively what do you suggest to improve responsiveness. At startup create a pool of TDARemoteDataAdapter with several instances for each of our services
that can then be assigned and released when used multithreaded?
Seems like a lot of extra work to use the RO framework efficiently

Regards,
Frederic

One extra piece of information: Our client and server run in the same (wired) LAN. In my case even on the same machine. So network speed and lags are not a factor and no big concern of ours. We use GetData for large amounts of data without an issue. Our schemes contain entire parts of our database (e.g. 50 tables) so they are not exactly small but still it seems GetData (LoadFromRemoteSource) deserializes much more efficiently than LoadScheme.

Hi,

Can you describe why you call table.LoadSchema manually?
If you open table with zero fields, schema will be loaded automatically with row data.
Can you describe why you are using personal RDA for each table?
Data Abstract v4 behavior is designed for using one RDA for all tables. as a result, you can pass array of tables in these calls:
RDA.Fill, RDA.FillSchema, RDA.ApplyUpdates

also you can use async version of these calls: BeginFill/EndFill, BeginFillSchema/EndFillSchema, BeginApplyUpdates/EndApplyUpdates,

Hi

  1. We have a little framework for all our DA tables.
    This adds necesary initialization when creating the tables. Some tables need the LoadSchema because there are parameters. It is error prone to disable this, the person modifying a DA schema server side will forget about this when adding a parameter.

  2. We use personal RDA because we had issues when using multithreading.

  • Can a single RDA be attached to 2 DA tables that are used concurrently from separate threads?
  • Furthermore: Can a single RDA be attached to 2 DA tables that have different services (schemes)?

FYI: I am trying the approach of a RDA instance per thread/service. This looks like a speed gain. I however still have to be carefull because not all of our DA tables use the same getdata method and this is a parameter which is also shared via the RDA

you can group your initial initialization per service, so one server call initializes RDA with service schema (RDA.ReadSchema) and then you can assign loaded schema to all tables that uses the same service.

if they use RDA async calls (Begin*/End*) - yes.

no, RDA can work only with one specified service.

It doesn’t really matter, on principle. Downloading a single byte from the network is not something that should be done in the main thread :wink: