Yes. Indeed need those TF_String_XXXX functions from encoding/deconding TensorFlow data. I believe those inline functions would enhance runtime performance for large-scale data - that is the reason they are introduced.
Is there any literature (documentation, blog) providing more details on Island fx and the importer, especially the functionalities to interop with C?
Apart from DLL, this importer also works with static library, and C obj file, right?
I think this is a great feature (like the capability of importing C inline functions). I would like to add the information to a research paper that I am working on.
The importer just turns the “.h” into binary data in there “.fx”. it’s really unrelated to where the implementation comes from (.dll, .lib or object) — aside from the LinkLibrary setting, which gets encoded in the .fx to tell the build chain what binaries to pass to the linker.
Right now, the only official documentation we have is what’s at FXGen.
Is this finalized? I downloaded the latest, and having the following method calling inline functions TF_TString_Init and TF_TString_Copy, but got Internal Error. Let me know if you need a new test case.
method ScalarStringTensor(const aStr: ^AnsiChar; aStatus: ^TF_Status): ^TF_Tensor;
begin
const TF_TSTRING_SIZE: Integer = 24;
var strlen := lstrlenA(aStr);
result := TF_AllocateTensor(TF_DataType.TF_STRING, nil, 0, TF_TSTRING_SIZE);
var data := ^TF_TString(TF_TensorData(result));
TF_TString_Init(data);
TF_TString_Copy(data, aStr, strlen);
end;
Severity Code Description Project File Line Suppression State
Error (E0) Internal error: System.InvalidOperationException: Not valid type for GEP
at LLVM.BitCode.LLVMModule.InstGEP(LLVMBaseValue aVal, LLVMBaseValue[] aArgs)
at RemObjects.Elements.Island.IslandOutput.GenerateArrayAccessExpression(Expression aExpr, Boolean aAddress, Boolean aSkipBounds)
at RemObjects.Elements.Island.IslandOutput.GenerateExpression(Expression aExpr, Boolean aAddress)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(Statement aSt)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(Statement aSt)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(ScopeStatement asT)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(Statement aSt)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(Statement aSt)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(ScopeStatement asT)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(Statement aSt)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(Statement aSt)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(ScopeStatement asT)
at RemObjects.Elements.Island.IslandOutput.GenerateStatement(Statement aSt)
at RemObjects.Elements.Island.IslandOutput.GenerateMethod(IMethodInfo ameth)
at RemObjects.Oxygene.Code.CombinedParsedType.ForAllMethods(Func2 action) at RemObjects.Elements.Island.IslandOutput.GenerateParsedType(IMutableParsedType aType) at RemObjects.Elements.Island.IslandOutput.<GenerateExecutable>b__19(IParsedType a) at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext()
at RemObjects.Elements.Island.IslandOutput.GenerateExecutable()
at RemObjects.Oxygene.Code.Compiler.Compiler.GenerateExecutable()
at RemObjects.Oxygene.Code.Compiler.Compiler.Compile() ApiHelpers C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\RemObjects Software\Elements\RemObjects.Elements.Island.Windows.targets 70
I downloaded the latest 2618 - I still have the following warning for the IMPORT project, on Windows?
inline function “TF_swap32” has unsupported element: include/tensorflow/core/platform/ctstring_internal.h (33) unknown identifier _byteswap_ulong [C:\DEVL
so when you import now, you have 1 or 2 .h files extra with custom content right?
What needs to happen now is that we need to provide these ourselves, as our Islandrtl does not have everything the c library has (I only added stuff on demand). What I think the sanest approach here is, is in stdio.h or stdlib.h (the empty files) you add the signatures for these functions, like:
extern unsigned long _byteswap_ulong ( unsigned long val );
Then it imports and you can provide this function in your own code with:
[SymbolName('_byteswap_ulong')]
class method _byteswap_ulong(l: Cardinal): Cardinal;
begin
exit (l shr 24) or (l shl 24) or ((l shr 8) and $ff00)or ((l shl 8) and $ff0000);
// note i wrote this from memory; it might be wrong!
end;