Microsoft ML.NET dependencies not being added to /Debug folder

pthompson · April 13, 2021, 10:22am

Hi,

Not quite sure where this one sits, but I think it’s a RemObjects issue ( Elements 10.0.0.2575)) and not a Microsoft one. I have two versions of a .Net Classic console application. The first is in Microsoft C# and the other in Oxygene. The application is a first investigation of ML.NET , making use of sample .Net Core code, translated in to classic .Net.

The C# version copies a number of native DLL dependencies into the debug folder that the Oxygene version does not. The C# application works and the Oxygene one fails (due to missing MklImports.dll).

If I manually copy the same 5 dll’s into the Oxygene debug folder, the application works.
The missing dll’s are CpuMathNative.dll, LdaNative.dll, libiomp5md.dll, MklImports.dll and MklProxyNative.dll. These are shown in the screenshot below. The attached project is the Oxygene version and should build…

As I’d like to investigate ML.NET further (in Oxygene) I’d like to get past this issue. Can you advise ?

TimeSeriesForecastOXY-master.zip (363.1 KB)

mh · April 13, 2021, 12:32pm

Are your references to the ML .dlls set to Copy Local? that’s what governs which dlls are copied to output and which aren’t…

pthompson · April 13, 2021, 1:11pm

Hi Marc,

Yes, as per the screenshot (in my original post)- you can see all the ML dll’s are copied into the debug folder. The ones that don’t get copied are those highlighted. In my C# equivalent project, those dll’s are copied in addition to the ML ones. There are differences in the project references, although the projects are identical in terms of function .

mh · April 13, 2021, 1:35pm

Hmm. are those directly referenced by the project, or where do they come from? I don’t see those in the list of references in your new screenshot (Oxygene or VC#), so where do they magically come from in the C# project? Can I see the .csproj and .elements files for this?

That’s fine. Microsoft.CSharp is specific to VC#, and yu won’t need (but feel free to dd it if you like). mscorlib is always referenced, its the core .NET library. I’m guessing VC#@ just hides it from you.

I’m guessing System.Net.Http is just in the C# project redundantly and not used, again nothing I would worthy about.

pthompson · April 13, 2021, 2:00pm

The extra dll’s aren’t referenced directly. From a little reading and by their name, I believe they are native dll’s and not managed. Three of the files are from …\packages\Microsoft.ML.Mkl.Redist.1.5.5\runtimes\win-x64\native and in the C# version of the project are copied to the debug folder on build.

Having started with an Oxygene project and running into " Unable to load DLL ‘MklImports’: The specified module could not be found. (Exception from HRESULT: 0x8007007E)" I created a C# version which worked and it was at that point I noticed the extra native dll’s being copied into the debug folder.

My Oxygene version of the project is attached on the first post in the chain - it should just need the packages adding - but I don’t think you run VS on a windows PC ?

mh · April 13, 2021, 2:02pm

Oh, those are NuGet packages? I could not tell that from the screenshot. Can I see the (ideally complete, or act least one that will copile clean for me, can be without any real code) Oxygene project and have a look? it seems that our NuGet processing might be missing a step for the native .dlls…

thanx,
marc

pthompson · April 13, 2021, 2:11pm

Yes, sorry they are NuGet packages :-

(You need the Microsoft.ML - the next three are added automatically as dependencies, then the Micorosoft.ML.TimeSeries needs adding).

I’m not quite clear on what you’d like me to send you… The project with the packages is several hundred Mb. Without the packages, it’s what is in the zip file. The code is very simple - just a few lines of main program, a couple of very simple data holding classes and sample data file - energy_hourly.csv.

mh · April 13, 2021, 3:55pm

My apologies; I missed the zip in the original post. gift it now, and will have a look in the next couple of hours, and hopefully be able to fix this.

—marc

mh · April 13, 2021, 5:06pm

Oe problem is that your project has hardcoded references to the dlls, rather than the NuGet packages:

    <Reference Include="Microsoft.ML">
      <HintPath>D:\Temp\Rubbish\TimeSeriesForecastOXY\packages\Microsoft.ML.1.5.5\lib\netstandard2.0\Microsoft.ML.dll</HintPath>
      <Private>True</Private>
    </Reference>

instead of

    <NuGetReference Include="Microsoft.ML:*">
      <Private>True</Private>
    </NuGetReference>

With that, out tool chain only sees the dlls as dumb references and it cannot do any additional handling for the full package. That said, even after fixing this (I’ll attaching the new project) this is not working as it should, I’ll ned to add some extra logic to handle the nativeassets folders in the package. I’ll look at getting that addressed for this week’s build, 2621 on Friday.

thanx,
marc

TimeSeriesForecastOXY.elements (5.4 KB)

mh · April 13, 2021, 5:46pm

Fixed:

               Final Output for 'TimeSeriesForecastOXY' target 'Echoes':
                 libCpuMathNative.dylib (/Users/mh/Downloads/TimeSeriesForecastOXY-master/TimeSeriesForecastOXY/Bin/Debug/libCpuMathNative.dylib)
                 libLdaNative.dylib (/Users/mh/Downloads/TimeSeriesForecastOXY-master/TimeSeriesForecastOXY/Bin/Debug/libLdaNative.dylib)
                 libMklImports.dylib (/Users/mh/Downloads/TimeSeriesForecastOXY-master/TimeSeriesForecastOXY/Bin/Debug/libMklImports.dylib)
                 libMklProxyNative.dylib (/Users/mh/Downloads/TimeSeriesForecastOXY-master/TimeSeriesForecastOXY/Bin/Debug/libMklProxyNative.dylib)
                 Microsoft.ML.Core.dll (/Users/mh/Downloads/TimeSeriesForecastOXY-master/TimeSeriesForecastOXY/Bin/Debug/Microsoft.ML.Core.dll)
                 Microsoft.ML.Core.xml (/Users/mh/Downloads/TimeSeriesForecastOXY-master/TimeSeriesForecastOXY/Bin/Debug/Microsoft.ML.Core.xml)
...

pthompson · April 13, 2021, 5:46pm

Thanks Marc. As you can see from the hint path, it’s not production code, so no big rush…

When creating both the Oxygene and C# versions, I followed the same process - just adding in the packages. I did not hard code any references. I can see in the Oxygene project file that the dll’s are explicitly referenced, and the C# ones are relative :-

<Reference Include="Microsoft.ML.Transforms, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL">
  <HintPath>..\packages\Microsoft.ML.1.5.5\lib\netstandard2.0\Microsoft.ML.Transforms.dll</HintPath>
</Reference>

Just just be sure of that, I’ve just created another sample Oxygene console application. All I have done is create from the wizard, and add in the ML package. The project file contains hard coded references :-

<Reference Include="Microsoft.ML">
  <HintPath>D:\Temp\rubbish\Sample1\packages\Microsoft.ML.1.5.5\lib\netstandard2.0\Microsoft.ML.dll</HintPath>
  <Private>True</Private>
</Reference>

So, would that would point to something in your build chain not making them relative ?

Also of interest, when adding the ML package, the project acquires a lot of “package.config files” :-

Looking into the individual files, it looks like the packages.config is the final one, with the first “copy” being the starting position and the final copy(12) having all the packages listed - each one adding one more of the dependant packages pulled in when the single ML package is added. This also happened in my original Oxygene application (but not C#) - I just remove the copies. Probably a separate issue, but worth mentioning ?

mh · April 13, 2021, 5:58pm

@viktoriad: are new EBuild-stylc package references not hook dup in VS yet?

pthompson:

When creating both the Oxygene and C# versions, I followed the same process - just adding in the packages. I did not hard code any references. I can see in the Oxygene project file that the dll’s are explicitly referenced, and the C# ones are relative :-
<Reference Include="Microsoft.ML.Transforms, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL">
  <HintPath>..\packages\Microsoft.ML.1.5.5\lib\netstandard2.0\Microsoft.ML.Transforms.dll</HintPath>
</Reference>

pthompson · April 19, 2021, 8:14am

As I said, no big rush on resolving this, but ML.NET is definitely a direction I’m pursuing.

Just for interest, here’s a graph showing Melbourne temperatures (in blue) for a 10 year period, and the results of ML.NET ForecastBySsa predictions laid over the top (in red).

Our customer retail data will need work to improve it in order to get anything near this level of accuracy, but from just a few lines of Oxygene code, this should be an interesting area to look at.

mh · April 19, 2021, 12:10pm

Friday’s build should have the fix for deployment; if your package references are done right. Since VS doesn’t have this hooked up properly, I suggest this:

remove all the NuGet refs form you project
open it in Water and add them back, there

after that you can keep working in VS, if you prefer, it will recognize and build with the new/proper package refrences.

very cool!