Re: Linker issues with struct postblit
On Sunday, 31 October 2021 at 03:51:49 UTC, James Blachly wrote:
On 10/29/21 7:10 AM, Stanislav Blinov wrote:
On Friday, 29 October 2021 at 11:05:14 UTC, Imperatorn wrote:
On Thursday, 28 October 2021 at 01:39:10 UTC, Thomas Gregory
wrote:
I am a maintainer of the
[dhtslib](https://github.com/blachlylab/dhtslib) package and
I have been running into issues with a new implementation of
reference counting we are using.
[...]
Postblit?
https://dlang.org/spec/struct.html#struct-postblit
I imagine Imperatorn is quite familiar with postblit and was
pointing out that it is strange to use postblit constructor,
which is deprecated, in a "new implementation".
The original post was long and really buried the lede, but OP
was pointing out what looks like a compiler bug: a linker error
referencing a postblit symbol only shows up when compiling with
DMD or with LDC earlier than 1.25.
Can someone give some insight?
I apologize for the long-winded question.
Postblit?
Yes, one solution would be to add a postblit.
However, the compiler should be able to generate default postblit
and it does. I am only running into issues linking, which leads
me to believe this is a compiler or language bug (fixed as of
ldc-1.25).
Is there some way to to avoid writing an explicit postblit for
every struct that uses `SafeHtslibPtr`? Many of `dhtslib`'s
structs wrap c type pointers that are reference counted and owned
by `SafeHtslibPtr` like so:
```D
// wrappedCPtr is an alias defined in another module as:
alias wrappedCPtr = SafeHtslibPtr!(c_type, destroy_c_type_fun);
// It is imported here
struct WrapperTypeName
{
/// Backing C type pointer
wrappedCPtr b;
...
}
```
Re: Linker issues with struct postblit
On 10/29/21 7:10 AM, Stanislav Blinov wrote: On Friday, 29 October 2021 at 11:05:14 UTC, Imperatorn wrote: On Thursday, 28 October 2021 at 01:39:10 UTC, Thomas Gregory wrote: I am a maintainer of the [dhtslib](https://github.com/blachlylab/dhtslib) package and I have been running into issues with a new implementation of reference counting we are using. [...] Postblit? https://dlang.org/spec/struct.html#struct-postblit I imagine Imperatorn is quite familiar with postblit and was pointing out that it is strange to use postblit constructor, which is deprecated, in a "new implementation". The original post was long and really buried the lede, but OP was pointing out what looks like a compiler bug: a linker error referencing a postblit symbol only shows up when compiling with DMD or with LDC earlier than 1.25. Can someone give some insight?
Re: Linker issues with struct postblit
On Friday, 29 October 2021 at 11:05:14 UTC, Imperatorn wrote: On Thursday, 28 October 2021 at 01:39:10 UTC, Thomas Gregory wrote: I am a maintainer of the [dhtslib](https://github.com/blachlylab/dhtslib) package and I have been running into issues with a new implementation of reference counting we are using. [...] Postblit? https://dlang.org/spec/struct.html#struct-postblit
Re: Linker issues with struct postblit
On Thursday, 28 October 2021 at 01:39:10 UTC, Thomas Gregory wrote: I am a maintainer of the [dhtslib](https://github.com/blachlylab/dhtslib) package and I have been running into issues with a new implementation of reference counting we are using. [...] Postblit?
Linker issues with struct postblit
I am a maintainer of the
[dhtslib](https://github.com/blachlylab/dhtslib) package and I
have been running into issues with a new implementation of
reference counting we are using.
Below is the implementation (which is basically our replacement
for `RefCounted`).
```d
/// Template struct that wraps an htslib
/// pointer and reference counts it and then
/// destroys with destroyFun when it goes
/// truly out of scope
struct SafeHtslibPtr(T, alias destroyFun)
if(!isPointer!T && isSomeFunction!destroyFun)
{
@safe @nogc nothrow:
/// data pointer
T * ptr;
/// reference counting
shared int* refct;
/// initialized?
bool initialized;
/// ctor that respects scope
this(T * rawPtr) @trusted return scope
{
this.ptr = rawPtr;
this.refct = cast(shared int *) calloc(int.sizeof,1);
(*this.refct) = 1;
this.initialized = true;
}
/// postblit that respects scope
this(this) @trusted return scope
{
if(initialized)atomicOp!"+="(*this.refct, 1);
}
/// allow SafeHtslibPtr to be used as
/// underlying ptr type
alias getRef this;
/// get underlying data pointer
@property nothrow pure @nogc
ref inout(T*) getRef() inout return
{
return ptr;
}
/// take ownership of underlying data pointer
@property nothrow pure @nogc
T* moveRef()
{
T * ptr;
move(this.getRef, ptr);
return ptr;
}
/// dtor that respects scope
~this() @trusted return scope
{
if(!this.initialized) return;
if(atomicOp!"-="(*this.refct, 1)) return;
if(this.ptr){
free(cast(int*)this.refct);
/// if destroy function return is void
/// just destroy
/// else if return is int
/// destroy then check return value
/// else don't compile
static if(is(ReturnType!destroyFun == void))
destroyFun(this.ptr);
else static if(is(ReturnType!destroyFun == int))
{
auto err = destroyFun(this.ptr);
if(err != 0)
hts_log_errorNoGC!__FUNCTION__("Couldn't
destroy/close "~T.stringof~" * data using function
"~__traits(identifier, destroyFun));
}else{
static assert(0, "HtslibPtr doesn't recognize
destroy function return type");
}
}
}
}
```
This can be used as such to reference count a pointer created
from the c library [htslib](https://github.com/samtools/htslib)
as such:
```d
/// bam1_t is a struct from c bindings
/// bam_destroy1 is a function to clean up a bam1_t *
/// that is created from the c bindings
alias Bam1 = SafeHtslibPtr!(bam1_t, bam_destroy1);
auto b = Bam1(bam_init1());
```
The issue presents with `SAMRecord`:
```d
struct SAMRecord
{
/// Backing SAM/BAM row record
Bam1 b;
/// Corresponding SAM/BAM header data
SAMHeader h;
```
dhtslib itself builds fine on both dmd and ldc compilers but when
it is used as a dependency it seems to have issues building on
any compiler that is not ldc > v1.24.0:
```
_D39TypeInfo_S7dhtslib3sam6record9SAMRecord6__initZ: error:
undefined reference to
`_D7dhtslib3sam6record9SAMRecord15__fieldPostblitMFNbNiNlNeZv'
```
Though I only experience this when trying to create an array of
`SAMRecord`s.
One solution I have found is using `std.array.Appender` instead
of arrays.
Another solution I have found is to define an explicit postblit
for `SAMReader`:
```
this(this)
{
this.h = h;
this.b = b;
}
```
Looking through ldc changelogs, the closest thing I could
attribute this to is this change for ldc-1.25.0:
- Struct TypeInfos are emitted into referencing object files
only, and special TypeInfo member functions into the owning
object file only. (#3491)
I suspect this is something to do with the alias'd function in
`SafeHtslibPtr`. Is there something I should be doing differently?
