Re: F*cked by memory corruption after assiging value to associative array
On Wednesday, 27 January 2021 at 22:57:11 UTC, tsbockman wrote: There is supposed to only be one instance of the D GC running per process. If you have more than one running then either you aren't linking and loading the DLLs correctly, or you have run into a serious bug in the D tooling. What could I do wrong by just using SimpleDllMain and then put my exports? build line for DLL is: rdmd -shared --build-only -gf -m64 Under Linux everything is shared. Under Windows each DLL seems to run in its own thread, has its own rt_options and do not see any __gshared variable value. Its completely isolated and so I assume that also GC is. Also https://wiki.dlang.org/Win32_DLLs_in_D says: Each EXE and DLL will have their own gc instance. I also wonder why the static linked DLL should use a GC proxy while as SimpleDllMain does nothing with a proxy - should loadLibrary() take care off here automatically? It seems, it does not.
Re: Why filling AA in shared library freezes execution?
On Wednesday, 27 January 2021 at 16:38:07 UTC, Adam D. Ruppe wrote: ... Yikes! Ok, I thought DLLs were just "sort of" unusable due to the RTTI issue, but now I'm convinced that they're almost completely useless in their current state unless you want to live in a world of hurt and pain. It's almost comical. At least the underlying issue is complicated, instead of it rather being a case of "no one's worked on it at all". Unfortunately, the underlying issue is complicated, so I don't have the greatest faith in DLLs becoming useable anytime soon. Thanks for the explanation.
Re: Initializing D runtime and executing module and TLS ctors for D libraries
On Thursday, 28 January 2021 at 00:58:17 UTC, rikki cattermole wrote: On 28/01/2021 1:16 PM, tsbockman wrote: The documentation build on dlang.org is broken. Check the source code or Adam D. Ruppe's dpldocs.info for the complete documentation: http://dpldocs.info/experimental-docs/core.thread.osthread.html Fixed: https://issues.dlang.org/show_bug.cgi?id=21309 I still don't see thread_setThis and thread_detachThis anywhere on the dlang.org copy.
Re: Assigning to class struct member leading to unexpected behavior.
On Thursday, 28 January 2021 at 02:03:40 UTC, Paul Backus wrote: The braced-initializer syntax only works in declarations, not assignments. Oh, I see, I'm guessing that explains the (sadly unfinished) In-place struct initialization DIP of wilzbach. (https://github.com/dlang/DIPs/pull/71) Kind of sad so many DIP's seem to end up stranded :/
Re: Assigning to class struct member leading to unexpected behavior.
On Thursday, 28 January 2021 at 01:43:13 UTC, Paul wrote:
I'm experiencing a compile error, but for the life of me, I
cannot figure out what is wrong.
I'll try to keep it short but the code is roughly as follows:
class Window{
Screen screen;
alias screen this;
this() {
Screen s = {bottom_f: {[0, 1]}};
this.screen = s; // Works
this.screen = {bottom_f: {[0, 1]}}; // Leads to the
list of errors below
}
The braced-initializer syntax only works in declarations, not
assignments.
Screen s = {bottom_f: {[0, 1]}}; // Ok - this is a declaration
this.screen = {bottom_f: {[0, 1]}}; // No good - this is an
assignment
Assigning to class struct member leading to unexpected behavior.
I'm experiencing a compile error, but for the life of me, I
cannot figure out what is wrong.
I'll try to keep it short but the code is roughly as follows:
class Window{
Screen screen;
alias screen this;
this() {
Screen s = {bottom_f: {[0, 1]}};
this.screen = s; // Works
this.screen = {bottom_f: {[0, 1]}}; // Leads to the
list of errors below
}
...
}
struct Screen{
Vec!(2, int) bottom_f;
...
}
alias Vec(uint size, Type) = Mat!(size, 1, Type);
struct Mat(uint rows, uint columns = rows, Type = float) {
private enum uint size = rows * columns;
union{
Type[columns][rows] mat;
Type[size] vec;
}
static if (columns == 1) alias vec this;
else alias mat this;
. . .
}
List of errors:
1. "Label "bottom_f" is not used."
2. "expected ; instead of }"
3. "found `}` when expecting `;` following statement"
4. "use `{ }` for an empty statement, not `;`"
5. "found `void` when expecting `;` following statement" (later
at function below constructor)
Why would assigning to a member variable lead to errors when
using an intermediate variable of the same type does not?
I tried tweaking a lot but I just can't figure it out. Am I doing
something wrong?
Re: Conversion error.
On Thursday, 28 January 2021 at 01:09:52 UTC, Adam D. Ruppe wrote:
On Thursday, 28 January 2021 at 01:01:36 UTC, Ruby The Roobster
wrote:
readf("%d",x);
This is why I hate readf, it is sensitive to litte things.
If you put a space in that string I think it will fix it. What
happens here is it reads the float, then leaves the buffer at
the \n from when the user pressed enter. So when it comes
around it complains about that unexpected character.
So try readf(" %d") or readf("%d\n") or something like that.
OR just use `readln().strip().to!int` that kind fo thing.
Thanks!
Re: Conversion error.
On Thursday, 28 January 2021 at 01:01:36 UTC, Ruby The Roobster
wrote:
readf("%d",x);
This is why I hate readf, it is sensitive to litte things.
If you put a space in that string I think it will fix it. What
happens here is it reads the float, then leaves the buffer at the
\n from when the user pressed enter. So when it comes around it
complains about that unexpected character.
So try readf(" %d") or readf("%d\n") or something like that.
OR just use `readln().strip().to!int` that kind fo thing.
Conversion error.
I don't know any explanation for the following error:
std.conv.ConvException@D:\Programs\D\dmd2\windows\bin\..\..\src\phobos\std\conv.d(2437):
Unexpected '\n' when converting from type LockingTextReader to type int
Here is my code for reference:
module main;
import std.stdio;
import std.concurrency;
import core.thread;
import core.stdc.stdlib;
import std.string;
int main(string[] args)
{
Tid worker;
int x;
do
{
writeln("Enter in 1 for arabic, 2 for turkish, 0 to quit.");
readf("%d",x);
worker = spawn(&Translate_Hello);
worker.send(x);
}
while(x != 0);
return 0;
}
void Translate_Hello()
{
int num;
num = receiveOnly!int();
switch(num)
{
case 1:
stdout.writeln("مرحبا");
break;
case 2:
stdout.writeln("Merhaba");
break;
default:
stdout.writeln("Invalid Language");
}
}
Re: Conversion error.
On Thursday, 28 January 2021 at 01:01:36 UTC, Ruby The Roobster wrote: I don't know any explanation for the following error: std.conv.ConvException@D:\Programs\D\dmd2\windows\bin\..\..\src\phobos\std\conv.d(2437): Unexpected '\n' when converting from type LockingTextReader to type int In addition, this only gives that exception when you enter 1 or 2, so I don't know what is happening.
Re: Initializing D runtime and executing module and TLS ctors for D libraries
On 28/01/2021 1:16 PM, tsbockman wrote: The documentation build on dlang.org is broken. Check the source code or Adam D. Ruppe's dpldocs.info for the complete documentation: http://dpldocs.info/experimental-docs/core.thread.osthread.html Fixed: https://issues.dlang.org/show_bug.cgi?id=21309
Re: Initializing D runtime and executing module and TLS ctors for D libraries
On Sunday, 24 January 2021 at 03:59:26 UTC, Ali Çehreli wrote: I am surprised how much I had learned at that time and how much I've already forgotten. :/ For example, my PR involves thread_setThis, which seems to be history now: https://docarchives.dlang.io/v2.068.0/phobos/core_thread.html#.thread_setThis And thread_detachThis seems to be missing now: https://dlang.org/phobos/core_thread.html https://dlang.org/phobos/core_thread_osthread.html Documentation issue or is it not needed anymore? The documentation build on dlang.org is broken. Check the source code or Adam D. Ruppe's dpldocs.info for the complete documentation: http://dpldocs.info/experimental-docs/core.thread.osthread.html You'll find thread_setThis and thread_detachThis are still there.
Re: Compile time check for GC?
On Wednesday, 27 January 2021 at 20:21:02 UTC, Kagamin wrote: You can define a symbol that will conflict with GC and prevent linking with it. That was an interesting thought! Maybe that could be a first step. It didn't occur to me that I could sabotage the usage of GC. :-D
Re: F*cked by memory corruption after assiging value to associative array
On Wednesday, 27 January 2021 at 18:09:39 UTC, frame wrote: I have no idea if there are multiple runtimes. I just use the mixin SimpleDllMain. But there must be multiple instances of GCs running because Another thread is running right now which I think is touching upon these same issues. Adam D. Ruppe explains some of what's going on: https://forum.dlang.org/post/veeksndchoppftluj...@forum.dlang.org Sadly, it looks like shared D DLLs are just kind of broken on Windows, unless you want to go the betterC route...
Re: F*cked by memory corruption after assiging value to associative array
On Wednesday, 27 January 2021 at 18:09:39 UTC, frame wrote: there must be multiple instances of GCs running because Sharing data between multiple threads that each use a different instance of the D GC will definitely not work right, because each GC will only know to pause the threads and scan the roots that it has been directly informed of. There is supposed to only be one instance of the D GC running per process. If you have more than one running then either you aren't linking and loading the DLLs correctly, or you have run into a serious bug in the D tooling. Or my debugger lied to me. I have found the gdb debugger on Linux often breaks horribly on my D code, especially when it is multi-threaded, and the debugger is only semi-usable. Maybe the Windows debugger is better now? (I wouldn't know, since I haven't used it in a while.) I think skepticism is warranted here.
Re: 200-600x slower Dlang performance with nested foreach loop
On 1/27/21 10:14 AM, Paul Backus wrote: On Wednesday, 27 January 2021 at 15:12:32 UTC, Paul Backus wrote: Maybe it's to avoid invalidating the result of `key in aa` when adding or removing entries? The spec doesn't say anything about it either way [1], but allowing invalidation would make AAs much more difficult to use in @safe code. Yes, that's the reason. Correction: the GC would take care of the safety issue, of course. I haven't had my morning tea yet, and clearly I'm not thinking straight. :) No, it wouldn't. The problem is not a GC issue, but an issue with aliasing expectations (if you rehash, you completely rearrange the buckets). so if you have: auto x = key in aa; aa[someOtherKey] = value; // rehashes Code at this point currently can count on x pointing at the item being used in the AA. If we change it now, lots of code will break. This is not truly a horrible issue, you can use a custom implementation (see any number of container libraries on code.dlang.org). What would be nice though, is to provide an actual template, that builtin AAs are an alias for (like string), and then if you want slightly different behavior, you provide different parameters. But at the end of the day, the builtin AAs will ALWAYS do this. We can't change it now. -Steve
Re: How do I get the output of the time bash command?
On Wednesday, 27 January 2021 at 09:58:18 UTC, Arafel wrote:
On 27/1/21 10:35, Anthony wrote:
I'm trying to read the timed output of a pipeShell command but
it only results in empty output.
Does anyone know why this is?
```
auto p = pipeShell("time ls");
foreach(str; p.stdout.byLine) {
writefln("%s",str);
}
```
I'm not sure why you get an empty output, you should get at
least the `ls` output unless it's an empty directory (or one
with only "dot" files).
However, in any case `time` returns the timing information
through `stderr`, not `stdout`[1]. You can try [2,3] (untested):
```
auto p = pipeShell("time ls", Redirect.stderrToStdout);
```
Best,
A.
[1]: https://linux.die.net/man/1/time
[2]: https://dlang.org/library/std/process/pipe_shell.html
[3]: https://dlang.org/library/std/process/redirect.html
Thanks! I think this was the issue.
Turns out that pipeShell uses sh not bash so the "time" program
isn't available.
When I printed stderr, it showed me the error.
Re: Compile time check for GC?
You can define a symbol that will conflict with GC and prevent linking with it.
Re: F*cked by memory corruption after assiging value to associative array
On Wednesday, 27 January 2021 at 17:41:05 UTC, Ali Çehreli wrote: On 1/26/21 6:31 AM, frame wrote: > all GCs Multiple D runtimes? That might work I guess but I've never heard of anybody talking about having multiple runtimes. Does rt_init() initialize *a* D runtime or *the* D runtime? If it indeed works we definitely need much better documentation. I load my libraries with loadLibrary[1] so that "[if] the library contains a D runtime it will be integrated with the current runtime." Ali [1] https://dlang.org/library/core/runtime/runtime.load_library.html I have no idea if there are multiple runtimes. I just use the mixin SimpleDllMain. But there must be multiple instances of GCs running because 1) command line argument --DRT-gcopt=gc:manual was seen by the EXE but ignored by the DLL and still crashed 2) after "burning in" gc:manual in the DLL, observing GC.profileStats.numCollections shows in one DLL thread 0 and the other DLL thread > 0 and thus crashed. Or my debugger lied to me. I also use loadLibrary.
Re: F*cked by memory corruption after assiging value to associative array
On 1/26/21 6:31 AM, frame wrote: > all GCs Multiple D runtimes? That might work I guess but I've never heard of anybody talking about having multiple runtimes. Does rt_init() initialize *a* D runtime or *the* D runtime? If it indeed works we definitely need much better documentation. I load my libraries with loadLibrary[1] so that "[if] the library contains a D runtime it will be integrated with the current runtime." Ali [1] https://dlang.org/library/core/runtime/runtime.load_library.html
Re: How to dinamically create Tuples?
On Wed, Jan 27, 2021 at 05:17:18PM +, Paul Backus via Digitalmars-d-learn wrote: > On Wednesday, 27 January 2021 at 17:11:52 UTC, Leonardo wrote: > > Hi, I want to know if are some way to dinamically create Tuples, > > with variable size and types defined at runtime. Thanks. > > No. D is a statically-typed language, so all types have to be defined > at compile time. But you can probably achieve equivalent semantics with an array of Variant (see std.variant). T -- Written on the window of a clothing store: No shirt, no shoes, no service.
Re: How to dinamically create Tuples?
On Wednesday, 27 January 2021 at 17:11:52 UTC, Leonardo wrote: Hi, I want to know if are some way to dinamically create Tuples, with variable size and types defined at runtime. Thanks. No. D is a statically-typed language, so all types have to be defined at compile time.
Re: F*cked by memory corruption after assiging value to associative array
On Tuesday, 26 January 2021 at 14:31:58 UTC, frame wrote: but why does not see DLL 1 then that sub objects of B are still alive? I may fool myself but could it be caused by an already gone slice data? It very looks like that only a specific string property is corrupted which got the same slice data as an input parameter. I thought that the slice data should stay referenced in the persistent object anyway but the GC seems not so smart to detect this. The error can be prevented with .dup so far.
How to dinamically create Tuples?
Hi, I want to know if are some way to dinamically create Tuples, with variable size and types defined at runtime. Thanks.
Re: Why filling AA in shared library freezes execution?
On Wednesday, 27 January 2021 at 15:25:17 UTC, H. S. Teoh wrote: I'm surprised this stuff hasn't been fixed yet, considering Walter (mostly?) works on Windows. Has he never run into these issues before? It had a dconf talk spell it all out. But it can be difficult to reproduce the nasty cases in a reduced test. Well, the exception catching ones are trivial to reproduce, it just plain doesn't work. But the GC and EH tables not lining up are harder to find. The EH table one rears its head once you fix the typeinfo problem that prevents the catch in the first place. But then the GC one is just hard to trigger. Unless you're using dll plugins in my day job project. (but even there it only happens after like 100 iterations of stuff, just the day job scripts run hundreds of times a day so you see it regularly... just can't fix it...)
Re: Why filling AA in shared library freezes execution?
On Wed, Jan 27, 2021 at 02:39:08PM +, Adam D. Ruppe via Digitalmars-d-learn wrote: > On Wednesday, 27 January 2021 at 14:36:16 UTC, Adam D. Ruppe wrote: > > (btw as for me fixing it myself > > oh edit, I should point out it also requires some degree of language > change to match what Microsoft's C++ does. They do dllimport and > dllexport annotations to help the compiler generate the stuff. So that > prolly is another complication to actually merge it. "Breaking" > changes lololololol as if it can get more broken than it is now. I'm surprised this stuff hasn't been fixed yet, considering Walter (mostly?) works on Windows. Has he never run into these issues before? T -- One reason that few people are aware there are programs running the internet is that they never crash in any significant way: the free software underlying the internet is reliable to the point of invisibility. -- Glyn Moody, from the article "Giving it all away"
Re: 200-600x slower Dlang performance with nested foreach loop
On Wednesday, 27 January 2021 at 14:15:26 UTC, FeepingCreature wrote: Associative arrays allocate per entry added?! https://github.com/dlang/druntime/blob/master/src/rt/aaA.d#L205 Oh God, associative arrays allocate per entry added! Maybe it's to avoid invalidating the result of `key in aa` when adding or removing entries? The spec doesn't say anything about it either way [1], but allowing invalidation would make AAs much more difficult to use in @safe code. [1] https://dlang.org/spec/hash-map.html
Re: 200-600x slower Dlang performance with nested foreach loop
On Wednesday, 27 January 2021 at 15:12:32 UTC, Paul Backus wrote: Maybe it's to avoid invalidating the result of `key in aa` when adding or removing entries? The spec doesn't say anything about it either way [1], but allowing invalidation would make AAs much more difficult to use in @safe code. [1] https://dlang.org/spec/hash-map.html Correction: the GC would take care of the safety issue, of course. I haven't had my morning tea yet, and clearly I'm not thinking straight. :)
Re: Why filling AA in shared library freezes execution?
On Wednesday, 27 January 2021 at 14:36:16 UTC, Adam D. Ruppe wrote: (btw as for me fixing it myself oh edit, I should point out it also requires some degree of language change to match what Microsoft's C++ does. They do dllimport and dllexport annotations to help the compiler generate the stuff. So that prolly is another complication to actually merge it. "Breaking" changes lololololol as if it can get more broken than it is now.
Re: Why filling AA in shared library freezes execution?
On Wednesday, 27 January 2021 at 13:39:32 UTC, SealabJaster wrote: The biggest one for me, is that RTTI isn't "shared" across boundaries. Yeah, that's a big one. Exception handling tables are also not properly merged leading to random behavior even if you do manage to catch the exception (I wrote a PR for that but it isn't theoretically sound https://github.com/dlang/druntime/pull/2874# ), the GC proxy is done wrong - but there might be a chance to fix that - and there's a bug with static data not being scanned. Possibly related to the GC proxy, not sure. It is all related to the same root problem - on Windows, unlike Linux, symbols are not automatically merged. More info http://dconf.org/2016/talks/thaut.html Yes, from 2016. And a branch aiming to fix it: https://github.com/ingrater/dmd/tree/DllSupportD So what happens is if you have a global variable in the dll and exe, each one has their own copy of it. So typeid are not merged, GC meta is not merged. druntime tries to hack around this but it doesn't do a good job. But then in user libs the same thing happens and they almost never make any attempt to handle it, so if you using other stuff, there be dragons. It might be perfectly fine, separate copies of data is often not fatal at all. But if they expect sharing - you can get separate singletons and such - you're in some trouble. Technically what dmd does for dlls isn't wrong. That is the way the underlying operating system works. But since it is more useful to handle these things and the Microsoft C++ compilers paper over this with auto-generated indirection, even Windows programmers might not realize it and it just seems completely broken. (btw as for me fixing it myself, there's a number of subtleties that I don't even know. Maybe i could figure them out but the fact is it is prolly gonna be a buggy transition. But the most discouraging part is seeing all that talk and code from 2016 just sitting there, ignored. What's the point of even trying if it is just going to rot again.)
Re: 200-600x slower Dlang performance with nested foreach loop
On Wednesday, 27 January 2021 at 02:14:39 UTC, H. S. Teoh wrote: Yes, definitely try this. This will completely eliminate the overhead of using an AA, which has to allocate memory (at least) once per entry added. Especially since the data has to be sorted eventually anyway, you might as well sort first then use the sortedness as a convenient property for fast de-duplication. Since .uniq traverses the range linearly, this will be cache-friendly, and along with eliminating GC load should give you a speed boost. T Associative arrays allocate per entry added?! https://github.com/dlang/druntime/blob/master/src/rt/aaA.d#L205 Oh God, associative arrays allocate per entry added!
Re: How do I get the output of the time bash command?
On Wednesday, 27 January 2021 at 09:58:25 UTC, Marcone wrote:
On Wednesday, 27 January 2021 at 09:35:21 UTC, Anthony wrote:
I'm trying to read the timed output of a pipeShell command but
it only results in empty output.
Does anyone know why this is?
```
auto p = pipeShell("time ls");
foreach(str; p.stdout.byLine) {
writefln("%s",str);
}
```
writeln(executeShell("time ls")[1]);
This is the correct answer. The documentation for pipeShell says
Like the functions they wrap, these functions return
immediately, leaving the child process to execute in parallel
with the invoking process. It is recommended to always call
wait on the returned ProcessPipes.pid, as detailed in the
documentation for wait.
Re: Why filling AA in shared library freezes execution?
On Wednesday, 27 January 2021 at 13:39:32 UTC, SealabJaster wrote: I've crossed them out of my mind entirely at this point though, since Windows in general doesn't seem to get much love in certain parts of D (e.g. the makefile for Phobos didn't support things that the posix makefile did). Should just quickly clarify that these "certain parts of D" are usually minor/niche parts and things.
Re: Why filling AA in shared library freezes execution?
On Tuesday, 26 January 2021 at 17:39:50 UTC, H. S. Teoh wrote: On Tue, Jan 26, 2021 at 02:12:17PM +, Adam D. Ruppe via Digitalmars-d-learn wrote: On Monday, 25 January 2021 at 21:48:10 UTC, Vitalii wrote: > Q: Why filling assoc.array in shared library freeze > execution? D exes loading D dlls are very broken on Windows. You can kinda make it work but there's a lot of bad design and showstopper bugs. [...] Just out of curiosity, what are some of the showstoppers? I'd have expected D exe's loading D dll's should be the first priority in making Windows dll's work in D. I'm surprised there are still obvious problems. T The biggest one for me, is that RTTI isn't "shared" across boundaries. So typeid(int) in your .exe isn't compatible with typeid(int) from a .dll I found out the hard way, since sdlang-d was giving me a very strange type mismatch error which was caused due to this issue in typeid. Am I wrong in saying that it was fixed at some point though, or at least, someone was attempting a fix? It's kind of discouraging to hear from Adam that there's actually even *more* issues regarding DLLs. I've crossed them out of my mind entirely at this point though, since Windows in general doesn't seem to get much love in certain parts of D (e.g. the makefile for Phobos didn't support things that the posix makefile did).
Re: How do I get the output of the time bash command?
On Wednesday, 27 January 2021 at 09:35:21 UTC, Anthony wrote:
I'm trying to read the timed output of a pipeShell command but
it only results in empty output.
Does anyone know why this is?
```
auto p = pipeShell("time ls");
foreach(str; p.stdout.byLine) {
writefln("%s",str);
}
```
Probably, just getting a string output is not enough. It looks a
little outdated, but there is this library:
https://code.dlang.org/packages/dexpect
Re: How do I get the output of the time bash command?
On 27/1/21 10:35, Anthony wrote:
I'm trying to read the timed output of a pipeShell command but it only
results in empty output.
Does anyone know why this is?
```
auto p = pipeShell("time ls");
foreach(str; p.stdout.byLine) {
writefln("%s",str);
}
```
I'm not sure why you get an empty output, you should get at least the
`ls` output unless it's an empty directory (or one with only "dot" files).
However, in any case `time` returns the timing information through
`stderr`, not `stdout`[1]. You can try [2,3] (untested):
```
auto p = pipeShell("time ls", Redirect.stderrToStdout);
```
Best,
A.
[1]: https://linux.die.net/man/1/time
[2]: https://dlang.org/library/std/process/pipe_shell.html
[3]: https://dlang.org/library/std/process/redirect.html
Re: How do I get the output of the time bash command?
On Wednesday, 27 January 2021 at 09:35:21 UTC, Anthony wrote:
I'm trying to read the timed output of a pipeShell command but
it only results in empty output.
Does anyone know why this is?
```
auto p = pipeShell("time ls");
foreach(str; p.stdout.byLine) {
writefln("%s",str);
}
```
writeln(executeShell("time ls")[1]);
How do I get the output of the time bash command?
I'm trying to read the timed output of a pipeShell command but it
only results in empty output.
Does anyone know why this is?
```
auto p = pipeShell("time ls");
foreach(str; p.stdout.byLine) {
writefln("%s",str);
}
```
