On Monday, 28 September 2020 at 15:44:44 UTC, Steven
Schveighoffer wrote:
On 9/28/20 8:57 AM, Bastiaan Veelo wrote:
I am glad to have found the cause of the breakage finally, but
it won't be easy to find a generic solution...
Obviously, this isn't a real piece of code, but there is no way
around this. You have to align your pointers. The other option
is to not use the GC and use manual memory management.
If this is a compatibility thing between D and Pascal, and you
absolutely have to have the same layout, is there a way to
adjust the structure in Pascal? Like put the elements that
misalign the pointers at the end of the structure?
Another totally drastic approach would be to supply your own
even-more-conservative GC which will scan misaligned pointers.
Probably going to hurt performance quite a bit. You might be
able to get away with marking only certain blocks as having
misaligned pointers, but you will have to scan all the stacks
with this assumption.
Some more information about the setup you are using might help
(I'm assuming D and Pascal are using the same memory in the
same process, otherwise this wouldn't be a problem). In
particular, where does the data come from, and how malleable is
it in your system? Are there times where references to the D
data only exist in Pascal?
-Steve
Thanks a lot for thinking with me. I’m not linking any Pascal
objects, so I don’t need to maintain binary compatibility in
memory; Only compatibility of data files. The problem arises when
those files are read using memory mapped files, from which
structs are memcpy’d over. This is of course the result of
machine translation of the current Pascal implementation.
Manual memory management is an option and would be
straightforward in principle, as we’ve done that for ages. The
only thing is that this memory cannot contain other allocations
on the GC heap, such as strings or other slices, unless they are
both aligned and their root is registered.
Fixing the alignment in Pascal is possible in principle, but any
old files would then need to first be processed by the last
Pascal version of the programs, which we then would need to keep
around indefinitely. There would also be issues when we port from
32 bit to 64 bit.
Another option could be to use 1-byte aligned structs for I/O,
and copy the members over in default aligned versions. But this
cannot be part of the automated transcompilation.
Thanks for suggesting a custom gc, which I had not thought of.
I’m leaning towards ditching the memory mapped I/O on the D end,
and replace it by regular serialisation/deserialisation. That
will be a manual rewrite though, which is a bit of bummer as
memory mapped files are widely used in our Pascal code. But this
will probably give the best end result.
-Bastiaan.
