Island memory manager question/passing Struct to C lib

wuping · December 20, 2019, 4:47am

I have a TensorFlow sample application in Island/Oxygene - and somehow I ended up in an exception which gave me a Window error code indicating heap corruption.

I am scratching my head. Maybe this is a stupid question - will by any chance Island memeory manager interfere with tensorflow’s C memory manager?

The sample is at here, TensorInfo:

github.com

wxinix/TensorFlow-Island/blob/master/src/TensorInfo.pas

// MIT License
// Copyright (c) 2019-2020 Wuping Xin.
//
// Permission is hereby  granted, free of charge, to any  person obtaining a copy
// of this software and associated  documentation files (the "Software"), to deal
// in the Software  without restriction, including without  limitation the rights
// to  use, copy,  modify, merge,  publish, distribute,  sublicense, and/or  sell
// copies  of  the Software,  and  to  permit persons  to  whom  the Software  is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE  IS PROVIDED "AS  IS", WITHOUT WARRANTY  OF ANY KIND,  EXPRESS OR
// IMPLIED,  INCLUDING BUT  NOT  LIMITED TO  THE  WARRANTIES OF  MERCHANTABILITY,
// FITNESS FOR  A PARTICULAR PURPOSE AND  NONINFRINGEMENT. IN NO EVENT  SHALL THE
// AUTHORS  OR COPYRIGHT  HOLDERS  BE  LIABLE FOR  ANY  CLAIM,  DAMAGES OR  OTHER
// LIABILITY, WHETHER IN AN ACTION OF  CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE  OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.

This file has been truncated. show original

ck · December 20, 2019, 5:42am

No the GC has it’s “own” memory (it uses virtualalloc to allocate a buffer for GC memory).

Given that I know nothing about tensor flow, can you elaborate a bit as to what fails and where? (and if it’s needed for me to run this, what I need to run this?)

wuping · December 20, 2019, 1:10pm

@ck
Many thanks

The github repository includes a list of Elements/Island projects. If you could clone that repository, and run the project: ./projects/TensorInfo, you would see the exception - it was caught by Element, and the error code indicates it was a Windows Heap Corruption error.

ck · December 20, 2019, 1:44pm

So it runs and triggers an access violations:

it prints: Tensor: input_4 inputs: 0 outputs: 1

Fails inside TF_GraphGetTensorNumDims:
Exception thrown at 0x00007FFDA2CBB120 (tensorflow.dll) in TensorInfo.exe: 0xC0000005: Access violation reading location 0x0000000200000092.

i think I know what’s going on here. Windows passes structs differently, and we use the sysv way of passing this.

RemObjectsSoftware · December 20, 2019, 1:44pm

Thanks, logged as bugs://83682

ck · December 20, 2019, 2:01pm

is this what i should expect @wuping?

Tensor: input_4 inputs: 0 outputs: 1
dims: 3Output: 0 type: TF_FLOAT [-1, 5, 12]
Tensor: output_node0 inputs: 1 outputs: 1
Input: 0 type: TF_FLOAT
dims: 2Output: 0 type: TF_FLOAT [-1, 4]

RemObjectsSoftware · December 20, 2019, 2:02pm

bugs://83682 got closed with status fixed.

wuping · December 20, 2019, 2:16pm

@ck Cool！ You are the Man! Yes - that is the expected results.

So this is some sort of LLVM struct ABI issue?

ck · December 20, 2019, 2:43pm

Not exactly. I’m defining how things are passed. The windows abi differed in this but we never encountered this issue before.

mh · December 20, 2019, 3:42pm

the fix will be in 2465, coming later today.

wuping · December 20, 2019, 6:37pm

@ck @mh
Thank you - I download the 2465 build. The “heap corruption” issue is solved, but the VS2019 debugger (Windows/Island/x64) seems stopped working. Previously, the debugger at least worked to some extent.

The debugger works as expected for non-tensorflow projects (i.e., when there is no linking to TensorFlow), but
The debugger stops working if the program is linked to TensorFlow. By “stops working”, I mean, it won’t hit ANY breaking points, while just go all the way to run and exit (as if the “Start without Debugging” mode).

Any heads up?

UPDATE: it seems build 2465 has debugger totally broken, not working on VS2019/Water. It has nothing to do with whether linking to TensorFlow or not.

ck · December 20, 2019, 6:57pm

Hrmm I didn’t notice that. I’ll ask my colleague @elenap to look at it next week.

RemObjectsSoftware · December 20, 2019, 7:29pm

Thanks, logged as bugs://83687

wuping · December 22, 2019, 12:42am

Double-check and Update - The Island/Windows debugger is totally broken with build 2465

On both VS2019 and Water, it doesn’t work at all. No breaking points are hit. It has nothing to do with TensorFlow.
This can be replicated easily with a boilerplate project (see attached):
ConsoleApplication.7z (4.2 KB)

RemObjectsSoftware · January 6, 2020, 9:24am

bugs://83687 got closed with status fixed.