What is the memory/type model? and why doesn't the website explain it?

[davidj (David Jeske)]

I just landed on this website today… I’m reading about this elements compiler environment decoupling languages and platforms, and the obvious unanswered question is…

what the heck is the memory model / GC / typing / runtime situation for this cross-compiling? and why doesn’t the website say anything about it?

• Is Swift on windows an ARC system talking to a .NET runtime? Is it compiling swift into CLR?
• Is C# on Cocoa running on a GC runtime? Or is it C# with ARC? Does it ARC all C# objects?
• Is Java for Windows Native on a GC runtime? Or some Java/C hybrid with unsafe unmanaged memory?
• Does Google Go for CLR/JVM compiling Go to bytecode? If so, does it support row polymorphic structural typing? and if so, how? Is it using dynamic for everything? Or have some other way to map structural typing into these non-structural typing runtimes?

When talking about this kind of cross-compiling… this memory model (and type-model) information is essential. Seems like it should be answered up front, right on the Platforms Page. Yet there is zero mention of it. This makes me believe the platform is targeted at less sophisticated developers who don’t understand these details matter. Is that the case? Or is there a solid story here?

This docs page on Storage Modifiers is the first inkling of an answer I’ve been able to find… where it suggests the ARC storage keywords are available in all languages. However, this doesn’t really explain how this hybrid memory model / cross-compiling actually works. Is there some obvious page that has some details?

Here you can see my related post asking why Elements-Go produces different output than Google Go.

Doesn't match Google Go

I’m trying to understand what Elements-Go is. I put a simple Go program into Water, compiling to CLR, and the output is not correct. package RemTest1 import ( “fmt” ) type User struct { name String } func (user User) String() String { return fmt.Sprintf(“User: name = %s”, user.name) } func main() { user := User{name: “foo”} fmt.Println(user) // User: name = foo } In Google Go, this program outputs “User: name = foo” In Water, compiling for CLR, this program outputs “{f…

[mh (marc hoffman)]

Hi David,

@davidj

I just landed on this website today… I’m reading about this elements compiler environment decoupling languages and platforms, and the obvious unanswered question is…

The basics of this should be covered on our documentation site, specifically at ARC vs. GC. In essence, we use GC on all platforms except Cocoa, which uses ARC. On the platforms backed by our native Island compiler backend ("Island" Compiler Back-end), you can also manually override the lifecycle support and implement your own, but that’s a very advanced topic not sure anyone should ever worry about.

@davidj

• Is Swift on windows an ARC system talking to a .NET runtime? Is it compiling swift into CLR?
• Is C# on Cocoa running on a GC runtime? Or is it C# with ARC? Does it ARC all C# objects?
• Is Java for Windows Native on a GC runtime? Or some Java/C hybrid with unsafe unmanaged memory?
• Does Google Go for CLR/JVM compiling Go to bytecode? If so, does it support row polymorphic structural typing? and if so, how? Is it using dynamic for everything? Or have some other way to map structural typing into these non-structural typing runtimes?

None and all of these. Elements completely separates the language from the platform you are compiling for. The language is just the syntax. If you build a project for a .NET target, you are compiling to CLR (IL byte code) and will use GC, regardless of what language(s) you use (in fact, you can mix them in the same project, even in the same class, with partial classes).

GThe target platform drives all the semantics, such as the memory model, the platform APIs you have available, etc. Not the language. See these two block posts for some more background:

• Platform vs. Language
• Mixing Languages

Hope this helps, and I’ll be happy to answer any further questions you may have. I will also look at surfacing this information even more Monien toy on the main website and in the docs.

Thanks,
Marc

[davidj (David Jeske)]

@mh

Elements completely separates the language from the platform you are compiling for. The language is just the syntax.

I think this is a very important bit of information to put on the Languages page… Because this means the “Elements” version of the language will not have the same behavior as the language on it’s native runtime, merely the same Syntax.

In memory management, for example, existing C#/Java code which expects a garbage collector and makes cyclic references, is going to leak memory when running on ARC.

And while memory-management is one of the most obvious behavioral differences, it’s not the only one.

How are C# (or Go) structs mapped onto the JVM, which has no structs or value-type arrays? If they are mapped onto classes, how are the copy vs reference semantics handled? Do C# structs on JVM still carry their “copy on assignment” behavior? Or do they now act like classes with reference semantics?

What about separate compilation units (DLLs, Assemblies, .so)? How do “foreign” language constructs get mapped into these separate compilation units for mapping into other languages?

How is tail-call optimization handled? CLR supports it (partially), JVM doesn’t. Native code can do anything, so it depends what your compiler and back-end emit.

What about Google Go’s stackless/heap-stacks? For goroutines to be lightweight, all Go code needs to use heap-allocated stack-frames, which on CLR/JVM means making all Go methods async… is that happening in Elements?

These differences should be discussed somewhere as “behavioral differences of Elements languages when running on foreign platforms”.

[mh (marc hoffman)]

@davidj

In memory management, for example, existing C#/Java code which expects a garbage collector and makes cyclic references, is going to leak memory when running on ARC.

Memory management is not an aspect of the language, but of the platform, see Platform vs. Language again. For example, there’s literally dozens of languages that target the CLR or the JVM, they will all use Garbage Collection, because that is a feature of the platform, not of the language. For example, Pascal on .NET (e.g. Oxygene) uses GC, while classic Pascal such as Delphi or Borland/Turbo Pascal thatbtarget DOC or Win32 use(d) manual memory management. These behavior semantics are driven by the platform, not by the language. The language is “just” the syntax.

As a crude analogy — if you visit Japan, whether you speak Japanese or English, you still will need to drive on the left side of the road.

@davidj

How are C# (or Go) structs mapped onto the JVM, which has no structs or value-type arrays?

Struct support is provided transparently on the JVM to work in all languages like you would expect Struct to work. I can get the someone from the compiler team to give you a deep dive how this is handled on the lowest level, if you need (that’s outside of my area if expertise), but the relevant part is that if you use a Struct, it behaves like a value type.

@davidj

These differences should be discussed somewhere as “behavioral differences of Elements languages when running on foreign platforms”.

Define “foreign platforms” — that terminology itself implies the wrong approach to the subject, IMHO. But this topic among others, covers relevant platform differences. But neither (or all, depending on how you wanna play it) platform is foreign to the Elements languages…

[davidj (David Jeske)]

I find the idea of Elements/Water interesting. Much like seeing the magic of Haxe transpile into a bunch of targets, seeing the “magic” of two languages in the same project side-by-side is really interesting.

However, implying that because the language syntax is supported everything will just be okay, even when behavior is different… is somewhere between confusing and misleading.

It’s obvious Elements is willing to deliver a “language syntax” with behavior that differs from the main existing implementations of those languages. And this isn’t a problem. In fact, it’s more of an inevitability of the approach.

However, It would be nice if the docs and marketing were upfront about how this thing works, so we could understand how we might have to work with it to get a desired set of results.

---

As for the definition of the term “language”…

I personally consider the “C# language” to be what is defined in ISO 23270 / ECMA-334 document titled “The C# Language Specification”. I think most would agree.

This document outlines the requirements of the C# Language automatic memory management system, and ARC can not satisfy those requirements. Which means a C# compiler to an ARC memory manager is not the C# language – because it will not correctly run C# programs. (unless you consider using infinitely more memory acceptable)

I think the word the blog post should be using is “syntax” not “language”. However, this is just a pointless semantics argument. Whoever’s words we use, we need to be able to talk about things clearly…

Let’s just talk about “syntax” and “behavior”.

---

Myself and others have lots of C#/Java/Go code with memory-cycles that will not work under an ARC system. Others have lots of go-routine code that will not work under a system with heavyweight stacks.

The manual says:

RemObjects Go is a 100% compatible implementation of the Go language supported for all Elements platforms — .NET, Cocoa, Android, Java, Windows, Linux and WebAssembly.

In my world, using ARC under Mac-native is not 100% compatible with existing code. And if you are not providing lightweight heap stacks for Go then it’s also not 100% compatible with existing code.

So what is this “100% compatibility claim”? That it’s 100% compatible syntax parsing, even if it won’t run the same? I fail to see how that’s useful. And this isn’t just theoretical. Even my trivial ~20 line Go test program doesn’t run the same on Elements as on Google Go.

Here’s another ambitious claim…

An iOS or macOS app [written in any syntax] will look, feel and be as fast as any app written in Xcode.

Please explain what we’re dealing with here…

How does Java for MacOS native map native structs and packed native struct-arrays to Java language constructs?

Is there a new struct keyword (thus making the language a Java-superset like J#) or is there a functionality limitation (thus making Java-for-native-macos incapable of being as fast as native code). Likewise, if it’s being compiled to static native ARC code, what parts of standard Java libraries work and don’t work?

Likewise for Go and Go-routines. Is Gold/Go for JVM using lightweight heap-stacks? Or is this a case where it’s “technically Go syntax, but memory consumption is massively more for many threads”?

---

Looking forward to any clarity…

[ck (Carlo Kok)]

Just want to add a bit on the platform mapping. With Java and the clr we are limited to the platform itself for some features. As you mentioned Java doesn’t have structs. Neither Java not .net support stacks the way go does them.

The first is solved by taking structs. Structs are mapped to classes with compiler controlled semantics. This makes them act exactly like golang does.

The second problem isn’t solved yet but we have a good plan there which will work via a micro await support. This is a lot like async await in .net but all methods become sort of like micro tasks. It works quite well but isn’t ready for production yet. So that will be for v2.

I hope that explains where we are and where we are going.

What we currently have is a quite well working framework that matches go in most ways. But blocking io does block and require multiple threads currently.

The big problem with mapping on top of other platforms is always that you have to work with their constraints.

If you want to see how the microawait stuffs works, it’s already in echoes.dll and islandrtl.

[davidj (David Jeske)]

Thanks for the detailed technical reply. This does explain something about where you are going.

@ck

The big problem with mapping on top of other platforms is always that you have to work with their constraints.

I understand this well, and I don’t expect Elements not to have constraints.

It seems like the “language vs platform” blog post is on the one hand misleading and on the other hand selling Elements short.

Elements is not implementing standard-Java or standard-C#, because that would be insufficient to run properly on these vastly different platform targets.

Elements seems like a multi-facade language system. It’s not “Java on Cocoa”… it’s “Elements on Cocoa”, where Elements has syntax-facades that resemble Java, and C#, and Pascal, and Go. But to say those facades are 100% compatible with some other implementation is both over-promising, and under-promising. Because in reality one needs a Java superset (Indigo) to be effective on CocoaNative or CLR. (just like J# was a superset on CLR)

---

As for the technical details… I hope the docs get improve to clarify some of the un-answered questions…

I understand and see the Iodine __struct declaration, and page on Structs. What happens for arrays of __struct? Can I make them? I can guess at how I would handle this in Cocoa/ARC and CLR, but they would be just guesses.

As for Go, I can imagine Go being compiled to CLR async/await, though perhaps you are implying a different and custom compiler transformation when you say “micro await”. Though allowing people to use existing blocking .NET code libraries inside a go-style cooperative threading environment becomes very error-prone and problematic. Any code that accidentally calls something that blocks will block all cooperative threads. This is why “stackless” environments like Google-Go and Stackless-python tend to be worlds unto their own. I’m curious to see what your “v2” solution looks like for Go.

I found all of this because I was looking for Swift for Windows and landed on Silver. I think the case for extending the ARC based Silver onto other platforms is a bit easier to wrap my head around, since it won’t create problems if the core expectation is Swift-on-ARC. Since GC would only do a better job (though with GC pauses, obviously).

[ck (Carlo Kok)]

Swift is a different level of thing though. Swift had so many things that just can’t be done on jvm or .net that you can hit things. But with go we have the whole runtime working, so compatibility is very high.

As for micro await. Async await works with the presumption that you are going to yield and involves creating classes at every level. Micro await uses a value type, and avoids allocations. It also presumes to not yield, but supports it. That said, yes, like in go, if you do hit a path that can’t yield, you block that thread. (I’m fairly sure go hits that too)

[davidj (David Jeske)]

@ck

As for micro await. Async await works with the presumption that you are going to yield and involves creating classes at every level. Micro await uses a value type, and avoids allocations. It also presumes to not yield, but supports it.

I don’t understand how a solution presuming to not yield (which I assume means yields are slow?) would work with goroutine code, since it tends to yield very very often. Moreso, without efficient goroutines, there is little reason to use the Go language, since it doesn’t yet have parametrics.

Generally lightweight threads fall into two categories… (a) “stackless” heap-allocated frames, and (b) “stackful” stack-relocating techniques.

(a) async/await is a stackless heap-allocated frame solution, implemented into explicit classes within the compiler when requested. (as opposed to implicitly handling all functions this way, as in Stackless Python)

(b) The experimental “Implementing Coroutines for .NET” is a stackful relocation technique, based on windows fibers. Though it’s fragile and using undocumented APIs.

Here is another useful article on coroutine implemetations…

https://blog.panicsoftware.com/coroutines-introduction/

[ck (Carlo Kok)]

@davidj

I don’t understand how a solution presuming to not yield (which I assume means yields are slow?) would work with goroutine code, since it tends to yield very very often. Moreso, without efficient goroutines, there is little reason to use the Go language, since it doesn’t yet have parametrics.

No the big difference here is in overhead of the structure required to yield (Task vs MicroTask). It’s not slow, but I don’t want to do a heap allocation per call. The micro task logic avoids that in the cases when not yielding (so really any call tree that doesn’t yield); only when something does yield, it gets moved to the heap and restored when done.

We can’t use (b) because it’s Windows only, and experimental.

(a) will match in behavior of Go though, the only time you would need to be careful is when calling foreign functions that can’t yield, but afaik Go can’t yield on functions that it doesn’t know about either.

[davidj (David Jeske)]

The new go v2 system sounds like the C/C++ “stackfull” stack copying coroutine techniques. Sounds interesting.