Re: Initializing D runtime and executing module and TLS ctors for D libraries
On 1/24/21 2:28 AM, IGotD- wrote:
> Any threads started by druntime has proper initialization of course. Any
> thread started by any module written in another language will not do D
> the thread initialization.
And that of course has been what I've been trying to deal with. Bugs in
the uses of thread_attachThis and thread_detachThis, and most
importantly, not having a guaranteed opportunity to call
thread_detachThis (think a foreign thread dies on its own without
calling us and the runtime crashes attempting to stop a nonexisting
thread during a GC cycle) finally made me realize that D shared library
functions cannot be called on foreign threads. At least not today... Or,
they can only be used under unusual conventions like the rule below.
So, that's the golden rule: If you want to call functions of a D shared
library (I guess static library as well) you must create your thread by
our library's create_thread() function and join that thread by our
library's join_thread() function. Works like a charm!
Luckily, it is trivial to determine whether we are being called on a
foreign thread or a D thread through a module scoped bool variable...
> Since std::function cannot be
> used in a generic interface I actually use something like this,
> http://blog.coldflake.com/posts/C++-delegates-on-steroids/.
If I understand that article correctly, and by pure coincidence, the
very shared libraries that are the subject of this discussion, which I
load at run time, happen to register themselves by providing function
pointers. Like in the article, those function pointers are of template
instances, each of which know exactly what to do for their particular
types but the registry keeps opaque functions. Pseudo code:
```
// Shared library:
struct A {
// ...
}
struct B {
// ...
}
shared static this() {
register("some key",
&serializer!A,// <-- Takes e.g. void* but knows about A
&deserializer!B); // ditto for B
}
```
And one of my issues has been that module constructors not being called
when the library is loaded as a dependency of a C++ library, which is
loaded by a Python module, which is imported by another Python module. :)
As I said earlier, I solved that issue by parsing and persisting the
output of 'nm mylib.so' to identify the module ctor and to call it after
dlsym'ing. Pretty hacky but works...
Getting back to my main issue: I am about to write a mixin template
where any library's interface .d file will do the following and get the
create_thread and join_thread functions automatically:
// mylib's extern(C) functions:
// This one provides mylib_create_thread() and mylib_join_thread():
mixin LibAPI!"mylib"();
// Other extern(C) functions of the library:
extern(C)
nothrow
int foo(int) {
// ...
}
The .h file must still be maintained by hand.
Ali
Re: D meets GPU: recommendations?
On Friday, 29 January 2021 at 20:01:17 UTC, Bruce Carneal wrote: On Friday, 29 January 2021 at 17:46:05 UTC, Guillaume Piolat wrote: On Friday, 29 January 2021 at 16:34:25 UTC, Bruce Carneal wrote: The project I've been working on for the last few months has a compute backend that is currently written MT+SIMD. I would like to bring up a GPU variant. What you could do is ressurect DerelictCL, port it to BindBC, and write vanilla OpenCL 1.2 + OpenCL C. Not up to date on both, but CUDA is messier than OpenCL. I don't really know about the other possibilities, like OpenGL + compute shaders or Vulkan + compute shaders. [...] I've not looked in to OpenGL compute shaders seriously either but I did look at Vulkan compute shaders. They looked very strong with respect to future deployability but were weak in other ways so I kept them off the list. I don't think I can warp my code to fit in to glsl or similar, at least not easily. Custom neighborhood indexing, group/sub-group scheduling control and ability to manage group local memory efficiently were the main concerns, IIRC. I took another look at Vulkan compute shaders. They have come a long way. Not nearly as nice as programming in D but good enough that I'll sketch out a few kernels if I can't make headway with dcompute. Anybody with recent dcompute experience, please speak up. The code in the repository is pretty nice but it looks unfinished or at least frozen in time.
Re: What does this code snippet even do?
On 1/29/21 2:41 PM, WhatMeWorry wrote: Ali's book talks about the colon appearing for There is also "is, expression": http://ddili.org/ders/d.en/is_expr.html#ix_is_expr.is,%20expression But the is expression is so complicated. :( I defined that particular syntax as is (T : Specifier, TemplateParamList) --- quote --- identifier, Specifier, :, and == all have the same meanings as described above. TemplateParamList is both a part of the condition that needs to be satisfied and a facility to define additional aliases if the condition is indeed satisfied. It works in the same way as template type deduction. I almost understand it. ;) Ali
Re: Quick question
On Saturday, 30 January 2021 at 00:58:09 UTC, Ruby The Roobster wrote: I have question here. Is there a difference between .sizeof and .length(of a char[])? for future reference if someone stumbles on this, .sizeof is the static size of the reference, .length is the actual length of the array you pretty rarely want array.sizeof in D.
Re: Quick question
On Saturday, 30 January 2021 at 00:58:09 UTC, Ruby The Roobster wrote: I have question here. Is there a difference between .sizeof and .length(of a char[])? For example, let's say you have the following array: char[2][] members. Is it possible for members[0].sizeof == members[1].sizeof but members[0].length != members[1].length? Thanks in advance. Nevermind. This is junk. I was trying to get something to work, but then I realized doing that was impossible. Ignore this now.
Quick question
I have question here. Is there a difference between .sizeof and .length(of a char[])? For example, let's say you have the following array: char[2][] members. Is it possible for members[0].sizeof == members[1].sizeof but members[0].length != members[1].length? Thanks in advance.
Re: What does this code snippet even do?
On Friday, 29 January 2021 at 22:59:14 UTC, H. S. Teoh wrote: On Fri, Jan 29, 2021 at 10:41:33PM +, WhatMeWorry via Digitalmars-d-learn wrote: [...] This means: "does the type of 'a' have the form U[], where U is a type that implicitly converts to T?". [...] This means: "does the type of 'a' implicitly convert to T[]?". [...] [...] Colon means "implicitly converts to". U is a template parameter to an implicit template `U[]`. It's basically used for pattern-matching the LHS type to some type pattern on the RHS. The general pattern is: is(typeToBeMatched : typePattern, templateParams...) `typeToBeMatched` is treated as a argument type to be matched against `typePattern` as if it were a template with parameters `templateParams`. T I wish we could upvote answers 👍
Re: What does this code snippet even do?
On Fri, Jan 29, 2021 at 10:41:33PM +, WhatMeWorry via Digitalmars-d-learn wrote: > // The following four lines in run.lang.io > > int[] a; > alias T = long; > pragma(msg, is(typeof(a) : U[], U : T)); This means: "does the type of 'a' have the form U[], where U is a type that implicitly converts to T?". > pragma(msg, is(typeof(a) : T[])); This means: "does the type of 'a' implicitly convert to T[]?". [...] > I know about alias (T is replaced with long), pragma, is, and typeof. > But what is U and where does it come from? And what do the colons do > here? Colon means "implicitly converts to". U is a template parameter to an implicit template `U[]`. It's basically used for pattern-matching the LHS type to some type pattern on the RHS. The general pattern is: is(typeToBeMatched : typePattern, templateParams...) `typeToBeMatched` is treated as a argument type to be matched against `typePattern` as if it were a template with parameters `templateParams`. T -- Designer clothes: how to cover less by paying more.
What does this code snippet even do?
// The following four lines in run.lang.io int[] a; alias T = long; pragma(msg, is(typeof(a) : U[], U : T)); pragma(msg, is(typeof(a) : T[])); // returns true false But I'm not even sure what I'm looking at. Ali's book talks about the colon appearing for :, associative array ⬁ :, import ⬁ :, inheritance ⬁ :, label but I'm pretty sure none apply here. I know about alias (T is replaced with long), pragma, is, and typeof. But what is U and where does it come from? And what do the colons do here?
Re: D meets GPU: recommendations?
On Friday, 29 January 2021 at 17:46:05 UTC, Guillaume Piolat wrote: On Friday, 29 January 2021 at 16:34:25 UTC, Bruce Carneal wrote: The project I've been working on for the last few months has a compute backend that is currently written MT+SIMD. I would like to bring up a GPU variant. What you could do is ressurect DerelictCL, port it to BindBC, and write vanilla OpenCL 1.2 + OpenCL C. Not up to date on both, but CUDA is messier than OpenCL. I don't really know about the other possibilities, like OpenGL + compute shaders or Vulkan + compute shaders. Thanks for the pointer to Mike's bindings. If I run in to trouble plugging in with both dcompute and C++/SycL I'll take a look before falling back to CUDA. I've not looked in to OpenGL compute shaders seriously either but I did look at Vulkan compute shaders. They looked very strong with respect to future deployability but were weak in other ways so I kept them off the list. I don't think I can warp my code to fit in to glsl or similar, at least not easily. Custom neighborhood indexing, group/sub-group scheduling control and ability to manage group local memory efficiently were the main concerns, IIRC.
Re: D meets GPU: recommendations?
On Friday, 29 January 2021 at 18:23:40 UTC, mw wrote: On Friday, 29 January 2021 at 16:34:25 UTC, Bruce Carneal wrote: Guidance from experience regarding any of the above, or other, GPU possibilities would be most welcome. https://dlang.org/blog/2017/10/30/d-compute-running-d-on-the-gpu/ or Google search: https://www.google.com/search?&q=dlang+gpu Yes, there is a lot of readily available information on the possibilities that I listed. And I have read/viewed much of that, including the text and video items prominent in a simple google search. (shame on me if I ever ask for something trivially available) Current day "Guidance from *experience*..." is in another category, hence the request.
Re: D meets GPU: recommendations?
On Friday, 29 January 2021 at 16:34:25 UTC, Bruce Carneal wrote: Guidance from experience regarding any of the above, or other, GPU possibilities would be most welcome. https://dlang.org/blog/2017/10/30/d-compute-running-d-on-the-gpu/ or Google search: https://www.google.com/search?&q=dlang+gpu
Re: D meets GPU: recommendations?
On Friday, 29 January 2021 at 16:34:25 UTC, Bruce Carneal wrote: The project I've been working on for the last few months has a compute backend that is currently written MT+SIMD. I would like to bring up a GPU variant. What you could do is ressurect DerelictCL, port it to BindBC, and write vanilla OpenCL 1.2 + OpenCL C. Not up to date on both, but CUDA is messier than OpenCL. I don't really know about the other possibilities, like OpenGL + compute shaders or Vulkan + compute shaders.
Re: Why filling AA in shared library freezes execution?
On Friday, 29 January 2021 at 12:45:02 UTC, Imperatorn wrote: Anyone knows what it would take to fix it? This may help to narrow down the problem. Disable garbage collect Configuring the Garbage Collector https://dlang.org/spec/garbage.html https://stackoverflow.com/questions/472133/turning-off-the-d-garbage-collector Or change it so that there is only one thread Parallel marking https://dlang.org/spec/garbage.html
Re: Using mir to work with matrices
On Friday, 29 January 2021 at 15:35:49 UTC, drug wrote: Between is there a plan to implement some sort of static slice where the lengths of the dimensions are known in compile time? Compiler help is very useful. No. BLAS/LAPACK API's can't use compile-time information. User matrix loops can be optimized by the compiler using constants and without introducing new types. If you need a stack-allocated matrix, then a 1D stack-allocated array can be used import mir.slice.slice; double[12] payload; auto matrix = payload[].sliced(3, 4);
D meets GPU: recommendations?
The project I've been working on for the last few months has a compute backend that is currently written MT+SIMD. I would like to bring up a GPU variant. If you have experience with this sort of thing, I'd love to hear from you, either within this forum or at beerconf. In a past life I was a CUDA programmer, but I'd prefer to use something now that has at least some chance of running on most mobile devices in the future. In addition to the CUDA fallback, I've looked at SycL, dcompute and, as suggested by Ethan a couple of beerconfs back, I looked in to the D/MLIR work. That work is very nice but I didn't find the connection to GPUs. Guidance from experience regarding any of the above, or other, GPU possibilities would be most welcome.
Re: Using mir to work with matrices
On 1/29/21 4:50 PM, 9il wrote: On Tuesday, 26 January 2021 at 14:43:08 UTC, drug wrote: It is not easy to understand what mir library one should use to work with matrices. mir-glas turns out unsupported now and I try to use mir-blas. I need to reimplement my Kalman filter version to use more high dimension matrix than 4x4 plus Kronecker product. Is mir-blas recommended to work with matrices? Yes, it is wrapper around a common BLAS libraries such as OpenBLAS or Intel MKL. I've implemented the filter using mir-lapack. But then I found lubeck - I didn't try it but it seemed that it provided more high level functionality like matrix inversion (`inv` wrapper is much handy than `getrf_` and `dgetri_` combination I used directly). So my first thought was that lubeck should be more preferable but then I found there was lubeck2 so I didn't know what to think further. But I pretty well understand the situation and the reasons behind it. Thank you for you efforts! Between is there a plan to implement some sort of static slice where the lengths of the dimensions are known in compile time? Compiler help is very useful.
Re: Why filling AA in shared library freezes execution?
On Fri, Jan 29, 2021 at 12:45:02PM +, Imperatorn via Digitalmars-d-learn wrote: > On Wednesday, 27 January 2021 at 15:25:17 UTC, H. S. Teoh wrote: > > On Wed, Jan 27, 2021 at 02:39:08PM +, Adam D. Ruppe via > > Digitalmars-d-learn wrote: > > > On Wednesday, 27 January 2021 at 14:36:16 UTC, Adam D. Ruppe wrote: > > > > (btw as for me fixing it myself > > > > > > oh edit, I should point out it also requires some degree of > > > language change to match what Microsoft's C++ does. They do > > > dllimport and dllexport annotations to help the compiler generate > > > the stuff. So that prolly is another complication to actually > > > merge it. "Breaking" changes lololololol as if it can get more > > > broken than it is now. > > > > I'm surprised this stuff hasn't been fixed yet, considering Walter > > (mostly?) works on Windows. Has he never run into these issues > > before? [...] > Anyone knows what it would take to fix it? Somebody who (1) knows enough of compiler internals to be able to fix this, (2) is intimately familiar with how Windows dlls work, (3) is desperate enough to do the work himself instead of waiting for someone else to do it, and (4) is stubborn enough to push it through in spite of any resistance. T -- Never wrestle a pig. You both get covered in mud, and the pig likes it.
Re: F*cked by memory corruption after assiging value to associative array
On Friday, 29 January 2021 at 15:09:23 UTC, ShadoLight wrote: Just to confirm... I assume you just neglected to show the line in fun template function that returns the object, right? Yes, that's pseudo code with a missed return :D
Re: F*cked by memory corruption after assiging value to associative array
On Monday, 25 January 2021 at 17:11:37 UTC, frame wrote:
On Monday, 25 January 2021 at 16:54:42 UTC, vitamin wrote:
On Monday, 25 January 2021 at 16:44:40 UTC, frame wrote:
On Monday, 25 January 2021 at 16:14:05 UTC, vitamin wrote:
Yes, I directly calling on every function that returns an
object:
T fun(T)(T object) {
GC.addRoot(cast(void*) object);
}
...
extern (C) export SomeObject bar() {
return fun(new SomeObject);
}
Just to confirm... I assume you just neglected to show the line
in fun template function that returns the object, right?
Like...
T fun(T)(T object) {
GC.addRoot(cast(void*) object);
return object;
}
Re: Using mir to work with matrices
On Tuesday, 26 January 2021 at 14:43:08 UTC, drug wrote: It is not easy to understand what mir library one should use to work with matrices. mir-glas turns out unsupported now and I try to use mir-blas. I need to reimplement my Kalman filter version to use more high dimension matrix than 4x4 plus Kronecker product. Is mir-blas recommended to work with matrices? Yes, it is wrapper around a common BLAS libraries such as OpenBLAS or Intel MKL.
Re: Why filling AA in shared library freezes execution?
On Wednesday, 27 January 2021 at 15:25:17 UTC, H. S. Teoh wrote: On Wed, Jan 27, 2021 at 02:39:08PM +, Adam D. Ruppe via Digitalmars-d-learn wrote: On Wednesday, 27 January 2021 at 14:36:16 UTC, Adam D. Ruppe wrote: > (btw as for me fixing it myself oh edit, I should point out it also requires some degree of language change to match what Microsoft's C++ does. They do dllimport and dllexport annotations to help the compiler generate the stuff. So that prolly is another complication to actually merge it. "Breaking" changes lololololol as if it can get more broken than it is now. I'm surprised this stuff hasn't been fixed yet, considering Walter (mostly?) works on Windows. Has he never run into these issues before? T Anyone knows what it would take to fix it?
Re: Why filling AA in shared library freezes execution?
On Friday, 29 January 2021 at 01:23:20 UTC, Siemargl wrote: On Friday, 29 January 2021 at 00:45:12 UTC, Siemargl wrote: Then i modify program, just removing DLL, copying TestFun() in main module and it runs. Same compiler -m64 target. Ups. Sorry, I just forget copy test_dll.dll inside VM :-) So, program runs in Win7, but hangs after printing i:64511 I downgrade DMD to 2.090.1 + MSVC2013 libs and problem disappears. But 2.092 + MSVC2013 libs also hangs. Not every time, but You should really try to use a debugger to see what error is thrown in first chance. It also helps to identify a possible hidden problem that is not reproducable on other machines.
Re: F*cked by memory corruption after assiging value to associative array
On Thursday, 28 January 2021 at 22:11:40 UTC, tsbockman wrote: Alternatively, you can design your APIs so that no pointer to GC memory is ever owned or mutated by any thread unknown to that GC. (This is the only option when working across language boundaries.) Yes, thank you for your input - I was already thinking about that as it shows the better design and also wouldn't require that the DLL itself build-in much redundant code. However, if I run GC.collect() after every DLL function was done then it clearly shows which data goes away and bite me. Basically most objects are allocated in the main EXE/DLL anyway - it only comes in trouble where a new separate object is returned by the sub DLL - GC.addRoot() really solves that problem.
