Re: Designated initializers to function argument
On Friday, 28 July 2023 at 21:07:47 UTC, bachmeier wrote: On Friday, 28 July 2023 at 17:07:37 UTC, IchorDev wrote: No shit, it felt like an eternity. But it's still not in the spec...? I'd expect it to appear in the spec after there's a real release. This is the first I've heard of it being supported, and it sounds like it's incomplete. https://github.com/orgs/dlang/projects/19 Dennis has been working on this in pieces rather than all at once, as it required modification to multiple parts of the compiler.
Re: AA vs __gshared
On 7/28/23 11:15 AM, IchorDev wrote: On Friday, 28 July 2023 at 11:15:31 UTC, Steven Schveighoffer wrote: All `__gshared` does is give you storage that is accessible from all threads, "All __gshared does is give you [a live bomb, ready to go off at any moment]" !! It seems like it's not __gshared at all, but misusing malloc with GC pointers. On Friday, 28 July 2023 at 14:10:16 UTC, Kagamin wrote: Your error is using allocating the object with malloc. Since gc doesn't see your AA, the AA is freed and you get UAF. Friend, I think you nailed it. After adding this I haven't been able to reproduce the segfault again: ```d this(long n){ (){ cache = new typeof(cache); assert(cache); import core.memory; core.memory.GC.addRoot(cast(void*)cache); }(); // ... ``` It did always happen at random, so perhaps I haven't spent enough time testing it yet, but I've gone far longer without it segfaulting than ever before. This is the wrong approach, it's the allocating call that should add the root (actually a range). For instance, the mutex is not added, that might be collected. Or if you add more GC-pointing things into the class, that could be a problem. What I'd do is: ```d T alloc(T, A...)(auto ref A args){ enum classSize = __traits(classInstanceSize, T); void* mem = core.stdc.stdlib.malloc(classSize); assert(mem !is null, "Out of memory"); core.memory.GC.addRange(mem[0 .. classSize]); scope(failure) { core.memory.GC.removeRange(mem[0 .. classSize]); core.stdc.stdlib.free(mem); } T inst = cast(T)mem; inst.emplace(__traits(parameters)); return inst; } ``` And of course, a `dealloc` that removes the range should also be added. -Steve
Re: Designated initializers to function argument
On Friday, 28 July 2023 at 17:07:37 UTC, IchorDev wrote: On Friday, 28 July 2023 at 17:04:33 UTC, bachmeier wrote: [The DIP](https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1030.md) was approved long ago. It was waiting for an implementation. No shit, it felt like an eternity. But it's still not in the spec...? I'd expect it to appear in the spec after there's a real release. This is the first I've heard of it being supported, and it sounds like it's incomplete.
Re: Thread/Task cancellation
On Friday, 28 July 2023 at 18:52:59 UTC, Ruby The Roobster wrote: On Friday, 28 July 2023 at 18:17:18 UTC, Gjiergji wrote: I am coming from a C# background. I understood that there is no async/await equivalent in D (except fibers which are not suitable for multi threading), but if I am using threads, what is the D idiom to implement cancellation? Usually a long running thread looks like this (in C#): ```csharp try { while (!cancellationToken.IsCancellationRequested) { //do some work await SomethingAsync(cancellationToken); //do some other work await Task.Delay(TimeSpan.FromSeconds(5000), cancellationToken); } } catch (OperationCancelledException e) when (e.Token == cancellationToken) { //someone cancelled any of the await calls above, we can swallow it or log it } ``` The question is how do I pass a `cancellationToken` to the calls from the loop in order to terminate them before completion. For example, I learnt about `Thread.sleep` in phobos, but I cannot pass a cancellation token in order to cancel it before the intended sleep duration. Thx. [SNIP] You could use a thread to check if the token has been sent via message passing, and when it is sent, throw an exception, like this: This does however, terminate with signal 11 upon sending the terminate signal, and I'm not sure why. Ignore my above code, Here is something that should work: ```d import std.concurrency; void foo() { try { auto tid = spawnLinked(); ownerTid.send(tid); // do stuff // ... auto c = receiveTimeout(0.msecs, (ubyte a) {}); // Since bar is linked, this will throw an exception when bar terminates } catch(Exception e) { // Do whatever with the exception message, but it terminates the function execution. } } void bar() { bool terminate = false; terminate = receiveOnly!bool(); } void main() { spawn(); Tid tid = receiveOnly!Tid(); // ... if(needsToTerminateFooForSomeReason) tid.send(true); // ... } ``` This is the only way I could think of doing this, since exceptions don't travel up the threads.
Re: Thread/Task cancellation
On Friday, 28 July 2023 at 18:17:18 UTC, Gjiergji wrote: I am coming from a C# background. I understood that there is no async/await equivalent in D (except fibers which are not suitable for multi threading), but if I am using threads, what is the D idiom to implement cancellation? Usually a long running thread looks like this (in C#): ```csharp try { while (!cancellationToken.IsCancellationRequested) { //do some work await SomethingAsync(cancellationToken); //do some other work await Task.Delay(TimeSpan.FromSeconds(5000), cancellationToken); } } catch (OperationCancelledException e) when (e.Token == cancellationToken) { //someone cancelled any of the await calls above, we can swallow it or log it } ``` The question is how do I pass a `cancellationToken` to the calls from the loop in order to terminate them before completion. For example, I learnt about `Thread.sleep` in phobos, but I cannot pass a cancellation token in order to cancel it before the intended sleep duration. Thx. You could use a thread to check if the token has been sent via message passing, and when it is sent, throw an exception, like this: ```d import std.concurrency; Tid tid; void foo() { try { tid = spawn(); // do stuff } catch(Exception e) { // ... } } void bar() { bool terminate = false; terminate = receiveOnly!bool(); if(terminate) { throw new Exception("Thread terminated"); } } void main() { spawn(); // ... if(needsToTerminateFooForSomeReason) tid.send(true); // ... } ``` This does however, terminate with signal 11 upon sending the terminate signal, and I'm not sure why.
Re: Perspective Projection
On Friday, 28 July 2023 at 16:20:26 UTC, Dennis wrote: On Friday, 28 July 2023 at 16:08:43 UTC, Ruby The Roobster wrote: Everything displays fine (with orthographic projection, of course) if you leave the projection as the identity matrix, but setting it as I have done results in a blank screen. How do you pass the matrix to OpenGL? Be careful that gl3n uses row major matrices, but OpenGL uses column major matrices, so you either need to transpose it yourself, or pass `true` to the `transpose` argument in `glUniformMatrix4fv`. Thank you very much! Changing GL_FALSE to GL_TRUE in all of the `glUniformMatrix4fv` calls resulted in something being displayed, though very incorrectly. As it turns out, there is an inconsistency in the gl3n API, that you write a rotation matrix in radians, but you must write the FOV in degrees. After that, it worked as it was supposed to.
Thread/Task cancellation
I am coming from a C# background. I understood that there is no async/await equivalent in D (except fibers which are not suitable for multi threading), but if I am using threads, what is the D idiom to implement cancellation? Usually a long running thread looks like this (in C#): ```csharp try { while (!cancellationToken.IsCancellationRequested) { //do some work await SomethingAsync(cancellationToken); //do some other work await Task.Delay(TimeSpan.FromSeconds(5000), cancellationToken); } } catch (OperationCancelledException e) when (e.Token == cancellationToken) { //someone cancelled any of the await calls above, we can swallow it or log it } ``` The question is how do I pass a `cancellationToken` to the calls from the loop in order to terminate them before completion. For example, I learnt about `Thread.sleep` in phobos, but I cannot pass a cancellation token in order to cancel it before the intended sleep duration. Thx.
Re: Designated initializers to function argument
On Friday, 28 July 2023 at 17:04:33 UTC, bachmeier wrote: [The DIP](https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1030.md) was approved long ago. It was waiting for an implementation. No shit, it felt like an eternity. But it's still not in the spec...?
Re: Designated initializers to function argument
On Friday, 28 July 2023 at 07:35:00 UTC, IchorDev wrote: On Tuesday, 11 July 2023 at 17:43:43 UTC, Steven Schveighoffer wrote: On 7/11/23 11:22 AM, Ki Rill wrote: On Tuesday, 11 July 2023 at 15:16:54 UTC, Ki Rill wrote: apply(Appearance(color: BLACK, strokeWidth: 4)); // other fields are default initialized: strokeOpacity, fillOpacity, Yes, I was going to reply that v 2.103 has added (stealthily) named parameters *as a partial implementation*. Included in this is struct initializers, and constructors and functions that are *not* templates. If you are willing to use DMD 2.103 and above, you should be good. -Steve N-no way?! The spec makes no mention of them, is it really safe to use them yet? [The DIP](https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1030.md) was approved long ago. It was waiting for an implementation.
Re: Perspective Projection
On Friday, 28 July 2023 at 16:08:43 UTC, Ruby The Roobster wrote: Everything displays fine (with orthographic projection, of course) if you leave the projection as the identity matrix, but setting it as I have done results in a blank screen. How do you pass the matrix to OpenGL? Be careful that gl3n uses row major matrices, but OpenGL uses column major matrices, so you either need to transpose it yourself, or pass `true` to the `transpose` argument in `glUniformMatrix4fv`.
Perspective Projection
I again am having issues with OpenGL, this time with the projection matrix. Using gl3n, I have the following code: ```d // model matrix mat4 trans = mat4(0f); trans.make_identity(); trans = trans.rotatex(radians(-55)); // view matrix: mat4 view = mat4(0f); view.make_identity(); view = view.translation(0f ,0f, -3f); // projection matrix mat4 projection; projection.make_identity(); projection = projection.perspective(800f, 600f, radians(45f), .1f, 100f); ``` I am binding all of these matrices to the correct locations, and this seems to be the gl3n equivalent to what is given in the OpenGL tutorial: ```c // create transformations glm::mat4 model = glm::mat4(1.0f); // make sure to initialize matrix to identity matrix first glm::mat4 view = glm::mat4(1.0f); glm::mat4 projection= glm::mat4(1.0f); model = glm::rotate(model, glm::radians(-55.0f), glm::vec3(1.0f, 0.0f, 0.0f)); view = glm::translate(view, glm::vec3(0.0f, 0.0f, -3.0f)); projection = glm::perspective(glm::radians(45.0f), 800f / 600f, 0.1f, 100.0f); ``` Everything displays fine (with orthographic projection, of course) if you leave the projection as the identity matrix, but setting it as I have done results in a blank screen. I assume it has to do with the values of `(projection * view * trans * vec4(vertex, 1)).w` not being 1. Should I just use a different library, and if so, how to use it to generate a perspective matrix?
Re: AA vs __gshared
On Friday, 28 July 2023 at 11:15:31 UTC, Steven Schveighoffer wrote: All `__gshared` does is give you storage that is accessible from all threads, "All __gshared does is give you [a live bomb, ready to go off at any moment]" !! On Friday, 28 July 2023 at 14:10:16 UTC, Kagamin wrote: Your error is using allocating the object with malloc. Since gc doesn't see your AA, the AA is freed and you get UAF. Friend, I think you nailed it. After adding this I haven't been able to reproduce the segfault again: ```d this(long n){ (){ cache = new typeof(cache); assert(cache); import core.memory; core.memory.GC.addRoot(cast(void*)cache); }(); // ... ``` It did always happen at random, so perhaps I haven't spent enough time testing it yet, but I've gone far longer without it segfaulting than ever before.
Re: Syntax for Static Import of User Define Attributes
Whenever there might be symbol clash, or when i want to make sure i can identify where something from from i do: ```d import me = my.awesome.module; void main() { me.hi(); } ```
Re: AA vs __gshared
Your error is using allocating the object with malloc. Since gc doesn't see your AA, the AA is freed and you get UAF.
Re: Syntax for Static Import of User Define Attributes
On Friday, 28 July 2023 at 12:20:05 UTC, Steven Schveighoffer wrote: On 7/28/23 8:10 AM, Vijay Nayar wrote: It might be possible to expand the grammar. It seems very specific to UDAs, as it doesn't just throw out `Expression` or whatnot. It probably has to do with the spot that it's in (declaration). Yes, parsing arbitrary expressions after an `@` would result in this: ```D void f(int x) @att in (x > 0) { } ``` Being parsed as: ```D void f(int x) @(att in (x > 0)) { } ``` And things like `@3 + 3` don't look like they would be parsed as `@(3 + 3)`, it looks like `(@3) + 3`. So the syntax as `@(expression)` to make it clear where the expression ends. Then there's `@identifier` and `@identifier(args)` as shorthand for common cases that do look clear. I recently added `@TemplateSingleArgument` so you can do `@"abc"` or `@3` as well. Perhaps the syntax can be expanded to allow `@a.b.c(d)` as well, as well as `@a.b.c!d`, though there's a risk of the rules getting convoluted.
Re: Syntax for Static Import of User Define Attributes
On 7/28/23 8:10 AM, Vijay Nayar wrote: However, this makes me wonder. Is there any reason why the `@` shouldn't recognize the dots in a fully-qualified-name on its own, without the need for parentheses? It might be possible to expand the grammar. It seems very specific to UDAs, as it doesn't just throw out `Expression` or whatnot. It probably has to do with the spot that it's in (declaration). I'll defer to the compiler experts to decide what makes the most sense. -Steve
Re: Syntax for Static Import of User Define Attributes
On Friday, 28 July 2023 at 11:54:12 UTC, Steven Schveighoffer wrote: On 7/28/23 4:15 AM, Vijay Nayar wrote: I tried it and it worked for me. The template-argument syntax is normal. It's just a list of types or expressions (i.e. not a function argument list) What is the error? -Steve You're right, I must have tried something slightly different without realizing it. The syntax that works seems to be to enclose the entire fully-qualified-name of the user-defined-attribute along with all its arguments within parentheses. However, this makes me wonder. Is there any reason why the `@` shouldn't recognize the dots in a fully-qualified-name on its own, without the need for parentheses?
Re: Syntax for Static Import of User Define Attributes
On 7/28/23 4:15 AM, Vijay Nayar wrote: On Thursday, 27 July 2023 at 21:24:44 UTC, Dennis wrote: On Thursday, 27 July 2023 at 21:19:08 UTC, Vijay Nayar wrote: Attempted Fix 2: Enclose the entire attribute name in parenthesis. ``` static import vibe.data.serialization; class ChatCompletionFunctions { @(vibe.data.serialization.name)("name") ... } ``` Try: ```D @(vibe.data.serialization.name("name")) ``` This one causes a different error, because it invokes the template-argument syntax: https://dlang.org/spec/attribute.html#uda I tried it and it worked for me. The template-argument syntax is normal. It's just a list of types or expressions (i.e. not a function argument list) What is the error? -Steve
Re: Designated initializers to function argument
On 7/28/23 3:35 AM, IchorDev wrote: On Tuesday, 11 July 2023 at 17:43:43 UTC, Steven Schveighoffer wrote: On 7/11/23 11:22 AM, Ki Rill wrote: On Tuesday, 11 July 2023 at 15:16:54 UTC, Ki Rill wrote: apply(Appearance(color: BLACK, strokeWidth: 4)); // other fields are default initialized: strokeOpacity, fillOpacity, Yes, I was going to reply that v 2.103 has added (stealthily) named parameters *as a partial implementation*. Included in this is struct initializers, and constructors and functions that are *not* templates. If you are willing to use DMD 2.103 and above, you should be good. N-no way?! The spec makes no mention of them, is it really safe to use them yet? It isn't going away. I would wait a bit for libraries, because you don't want to force your users to require such a recent version of the compiler. Using it in an executable is fine. -Steve
Re: AA vs __gshared
On 7/28/23 4:39 AM, IchorDev wrote: Issue is, this code I posted actually runs just fine, unlike the real code. My actual code does this HIGHLY SUSPICIOUS thing when printing their length each time before using them: ``` 766 766 765 766 767 768 768 768 768 768 768 768 768 768 768 768 768 768 768 128000 Error Program exited with code -11 (backtrace:) #0 0x55670c46 in rt.aaA.Impl.findSlotLookup(ulong, scope const(void*), scope const(TypeInfo)) inout () #1 0x55661592 in _aaInX () ``` My suspicion would be a race condition in your real code, and no race condition in this toy code. Second suspicion would be memory corruption (possibly caused by a race condition). Therefore I must logically conclude that DRuntime's AAs are cursed! Unless this is a well-known issue or something... AAs have worked forever. I've never had problems with them. Not saying there's not a bug here, maybe there is. But I would be surprised. Thinking back, I've actually had them cause segfaults in non-threaded code, maybe `__gshared` was never the cause at all. All `__gshared` does is give you storage that is accessible from all threads, but is not typed as `shared`. It doesn't change how the data is used. -Steve
openssl 1.1 vs. 3.0 for vibe.d:tls on Ubuntu 22.04
Hello, some context first: I recently updated a server to Ubuntu 22.04 which appears to have only openssl 3.0.2 installed. Dub could compile my project, but could not link it anymore, as the D code seemed to be expecting openssl 1.1 whereas only 3.0.2 was installed. That type of errors: ``` /usr/bin/ld: ./vibe-d-0.9.6/vibe-d/tls/vibe/stream/openssl.d:370: undefined reference to `SSL_get_peer_certificate' /usr/bin/ld: ./vibe-d-0.9.6/vibe-d/tls/vibe/stream/openssl.d:1485: undefined reference to `ERR_put_error' collect2: error: ld returned 1 exit status ``` I tried various things, including all possible `subConfiguration` values for `vibe-d:tls`, and ended up: (1) selecting `"vibe-d:tls": "openssl-1.1"` (2) compiling openssl-1.1 from source following https://askubuntu.com/a/1458747 (3) symlinking the resulting files `libssl.a` and `libcrypto.a` into LDC's `lib` directory, since libs under that dir seem to have precedence over the system libs, as visible in the `/usr/bin/cc` call triggered by DUB. Then the project compiled again. . Now to the actual question: I am a bit confused since the source code of `vibe-d:tls` seem to support openssl-3.0, as visible e.g. in [1] but then in the config [2] I don't see anything like `"openssl-3.0"`. Maybe I missed something obvious! [1] https://github.com/vibe-d/vibe.d/blob/master/tls/vibe/stream/openssl.d#L198 [2] https://github.com/vibe-d/vibe.d/blob/master/tls/dub.sdl Could anyone please shed some light on a cleaner solution to get `vibe.d:tls` running on Ubuntu 22.04, esp. to get it running with openssl-3.0+? Thanks in advance, Guillaume . For info, the project has the following `dub.json` config: ``` ... "dependencies": { "dub": "~>1.33.1", "openssl": "~>3.3", "vibe-d": "~>0.9.6", "vibe-d:tls": "~>0.9.6" }, ... "libs": [ "curl" ], ... "subConfigurations": { "vibe-d:tls": "openssl-1.1" }, "versions": [ "VibeDefaultMain" ] ... ```
Re: AA vs __gshared
On Thursday, 27 July 2023 at 21:31:02 UTC, Jonathan M Davis wrote: What should normally be happening is that you use shared, and then when you've protected the object so that you know that it can only be accessed on the current thread by the section of code that you're in (e.g. by locking a mutex), you temporarily cast away shared to operate on the object via a thread-local reference. Then, before exiting that section of code and removing the protections that are preventing other threads from accessing the object (e.g. by unlocking the mutex), you make sure that you've gotten rid of all of the thread-local references to the object so that only the shared reference exists. That way, you don't accidentally mutate the object while it's not protected from access by other threads. Doing this doesn't help, unfortunately. This is an abstract version what the problematic code looks like now: ```d import std.stdio: writeln; import core.sync.mutex; import core.thread; import core.time; static import core.stdc.stdlib; struct Pos{ long x,y; } shared Obj obj; final class Obj{ CacheData[Pos] cache; CacheData* getCache(Pos key){ if(auto item = key in cache){ if(++item.useCount >= 8){ cache.remove(key); //I thought this ^ might cause a use-after-free issue, but apparently not. } return item; } return null; } struct CacheData{ double[1000] data; uint useCount = 1; } double[1000] doStuff(Pos pos){ immutable data = (){ if(auto data = getCache(pos)){ return *data; }else{ double[1000] data; //initialisses it with something, this is arbirray though: foreach(ref item; data){ import std.random; item = uniform01(); } cache[pos] = CacheData(data); return CacheData(data); } }(); //do stuff with data return data.data; } } __gshared OtherObj otherObj; final class OtherObj{ shared Mutex mutex; __gshared ubyte[2^^18] data; this(long n){ obj = cast(shared Obj)alloc!Obj(n); mutex = new shared Mutex; } void callObj(Pos pos){ double[1000] data; { auto objRef = cast(Obj)obj; data = objRef.doStuff(pos); } //do things with returned value... } } void thread(){ bool run = true; Pos pos; while(run){ otherObj.mutex.lock(); foreach(i; 0..100){ otherObj.callObj(pos); //this is pretty arbirary: import std.random; if(uniform01() > 0.9){ auto v = uniform01(); if(v < 0.25) pos.x--; else if(v < 0.5) pos.y++; else if(v < 0.75) pos.y--; else pos.x++; } } otherObj.mutex.unlock(); Thread.sleep(1.seconds / 20); } } void main(){ otherObj = alloc!OtherObj(-7); assert(otherObj !is null); auto otherThread = new Thread(); otherThread.start(); bool run = true; while(run){ if(!otherThread.isRunning()) otherThread.join(); otherObj.mutex.lock(); foreach(val; otherObj.data){ //do stuff } otherObj.mutex.unlock(); Thread.sleep(1.seconds / 80); } } T alloc(T, A...)(auto ref A args){ enum classSize = __traits(classInstanceSize, T); void* mem = core.stdc.stdlib.malloc(classSize); scope(failure) core.stdc.stdlib.free(mem); assert(mem !is null, "Out of memory"); mem[0..classSize] = __traits(initSymbol, T); T inst = cast(T)mem; static if(__traits(hasMember, T, "__ctor")){ inst.__ctor(__traits(parameters)); } return inst; } ``` Issue is, this code I posted actually runs just fine, unlike the real code. My actual code does this HIGHLY SUSPICIOUS thing when printing their length each time
Re: Syntax for Static Import of User Define Attributes
On Thursday, 27 July 2023 at 21:24:44 UTC, Dennis wrote: On Thursday, 27 July 2023 at 21:19:08 UTC, Vijay Nayar wrote: Attempted Fix 2: Enclose the entire attribute name in parenthesis. ``` static import vibe.data.serialization; class ChatCompletionFunctions { @(vibe.data.serialization.name)("name") ... } ``` Try: ```D @(vibe.data.serialization.name("name")) ``` This one causes a different error, because it invokes the template-argument syntax: https://dlang.org/spec/attribute.html#uda
Re: Designated initializers to function argument
On Tuesday, 11 July 2023 at 17:43:43 UTC, Steven Schveighoffer wrote: On 7/11/23 11:22 AM, Ki Rill wrote: On Tuesday, 11 July 2023 at 15:16:54 UTC, Ki Rill wrote: apply(Appearance(color: BLACK, strokeWidth: 4)); // other fields are default initialized: strokeOpacity, fillOpacity, Yes, I was going to reply that v 2.103 has added (stealthily) named parameters *as a partial implementation*. Included in this is struct initializers, and constructors and functions that are *not* templates. If you are willing to use DMD 2.103 and above, you should be good. -Steve N-no way?! The spec makes no mention of them, is it really safe to use them yet?
Re: AA vs __gshared
On Friday, 28 July 2023 at 04:13:13 UTC, Kagamin wrote: The difference between them is purely formal if you're not on an old gdc, where shared was synchronized like C# volatile. I'm not sure that's correct. Also I always use the latest compiler versions where possible. start many threads and access the locked AA concurrently. It's only being accessed from one thread, so such a test wouldn't be an accurate reproduction of the issue I'm having. Also I'm only using 2 threads total.