Paper comparing languages with egs?
I _think_ I remember seeing somewhere on the D site a reference to a paper on a comparison of programming languages that used a few small programs written in each compared language. I can't recall if D was one of the languages in the paper or whether someone had done D versions separately. Can anyone give me the link to the paper (and D versions) -- assuming I haven't completely misremembered!
Re: Is garbage detection a thing?
Recovering from memory errors at run time is unreliable. I should add that I have more like a romantic view of software release cycles where testing is done until the software is in a very, very sophisticated and stable state. More than usual. Not that I want to solely rely on such an approach. It should still try to recover from failures, and recovering should be trained like fire fighting is trained. But there should be no smokers in the building, so to speak.
Re: Is garbage detection a thing?
Elimination of memory problems is much more valuable than detection. Recovering from memory errors at run time is unreliable. I'm not sure but I have a gut feeling that I am just in a position that is not good to defend. I want small software that fails hard on weak causes, and the industry wants software that fails soft on strong causes. I cannot win any argument and will, as hobbyist who likes elegant code, face frustration after frustration, admittedly in all digital products, not only compilers. I'm sure you're right. Elimination of memory problems is what must be provided. I agree. The advantage of D is that all options are open. This allows the following approach: ... ... This makes starting a project in D a safe choice, in multiple meanings of the word. — Bastiaan. Thanks! :) Well, in the end, there should be a way like this.
Re: Is garbage detection a thing?
The reason this distinction is important, and the reason I bring up graph theory, is that liveness is impossible to prove. Seriously: it's impossible, in the general case, for the GC to prove that an object is still alive. Whereas it's trivial to prove reachability. My motivation was actually just that I wanted a very small compiler with no libraries, because I got a bit tired of big things with little defects. But the liveness is a thing I would like to say something about: I don't want a compiler that tries to prove it for me. The reason is that if I did manual memory management and created a use-after-free bug, then my personal world would be still very good. It's just that the industry isn't happy when releasing banana software is a serious problem? In the case of the use-after-free I would say that my software needs correct memory state and correct logic. The logic so to speak is what I actually wanted to implement in the first place. If the program is perfect and works in all situations, how could it do this with bugs in the memory management? It can't. So, I didn't really think often about it, but when Rust came out, there was some trend into this direction, and my feeling was: I can follow it, it looks good, but can I still just use C? The way it works would just be that I have correct out-of-bounds and use-after-free/double-free detection and race condition detection if my software is supposed to be chaotic server or browser software (?), both things at runtime. Given that it is true (?) that software cannot do a task correctly when it does part of it (memory management) incorrectly. More or less it was an idea to get what Rust and Swift try to do without all the language features. C with less language constructs would be nice. It's just that the processors are bad for me, for what I'm trying to do. Intel offers MPX. It's good I guess. But why does Visual C++ implement it like an easter egg. Why have they now added ASAN as experimental feature that immediately fails. They do things I don't really like, because the industry needs it like this, but not me. And at the high level, there's bloat and fancy colors. I want just a toolkit to create exactly what can be created, and if I do it wrong it should fail hard. And I'm trying to make it so that I don't write assembly, if that is possible and good in my situation. I'd say, I have just not understood the whole thing and maybe should try a different hobby, or finally create the thing I'm looking for, which turns out to be an endless story. It's just that a few hours ago I had the hope that the holy grail would exist. I had found the solution to almost every problem in Golang. And then after one year my hobby just broke down, because Go outputs wrong line numbers. It's no good compiler when it does that. I'd rather quit my hobby than accept it. Thanks a lot for your explanation! Really kind regards, you helped me to understand it.
Re: Is garbage detection a thing?
Looking at Ada now. I found: Ada is not good for me. It has no augmented assignment. It's just that I want DRY because I use very verbose variable names, and in the past I had a real world case (game in Lua) where I became frustrated when I had to repeat the names. I understand that NASA or so will repeat their variable names. They get paid. ;) Kind regards
Re: Is garbage detection a thing?
I could use AddressSanitizer indirectly by using Go. But their Oh wait, it was ThreadSanitizer that Go uses, right? I failed at talking. I would probably use ASAN under Linux, because that is the right thing to do? Looking at Ada now.
Re: Is garbage detection a thing?
On Sunday, 29 November 2020 at 16:21:59 UTC, Daniel N wrote: On Sunday, 29 November 2020 at 16:05:04 UTC, Mark wrote: Thanks a lot for reading, and sorry for a lot of text that is off-topic and is not related to D. Sounds like what you want is ASAN? You can use it with plain C or D(LDC). https://clang.llvm.org/docs/AddressSanitizer.html I could use AddressSanitizer indirectly by using Go. But their compiler gave me wrong line numbers for errors and I have not yet overcome this psycholocicaly, to be honest. They have a fixed version, which is a WIP which is already using generics. So I went on and saw that Visual C++ now features AdressSanitizer. It showed faulty behavior very soon, a false positive AFAIR. It's in experimental stage. /d2MPX is out of the early development stage but it's no prominent feature. I went on with Nim, which then gave me wrong line numbers for error messages. I'm not counting wrong, they really do, both Golang and Nim gave me errors that triggered me. ;) I began a little C compiler project based on c4, knowing that I would be very old when this should ever be finished. Actually I am looking for a good compiler for Windows or maybe macOS, and looking at JAI, too. Maybe I should just install Linux. But ... the drivers... My Thinkpad just doesn't like any Linux. I run out of ideas. In the first place all I wanted to do is make some music. Kind regards
Is garbage detection a thing?
Hi, can I ask you something in general? I don't know anyone whom I could ask. I'm a hobbyist with no science degree or job in computing, and also know no other programmers. I have no good understanding why "garbage collection" is a big thing and why "garbage detection" is no thing (I think so). I want to get rid of undefined behavior. So I tell myself, what is this actually? It's most of the time corrupted heap memory and the C++ compiler giving me errors that I thought were kind of impossible. Now I could follow all C++ guidelines and almost everything would be okay. But many people went into different directions, e.g. 1995 they released Java and you would use garbage collection. What I don't understand is, when today there exist tools for C++ (allocator APIs for debugging purposes, or Address-Sanitizer or maybe also MPX) to just detect that your program tried to use a memory address that was actually freed and invalidated, why did Java and other languages not stop there but also made a system that keeps every address alive as long as it is used? One very minor criticism that I have is: With GC there can be "semantically old data" (a problematic term, sorry) which is still alive and valid, and the language gives me the feeling that it is a nice system that way. But the overall behavior isn't necessarily very correct, it's just that it is much better than a corrupted heap which could lead to everything possibly crashing soon. My bigger criticism is just, that compilers with garbage collection are big software (with big libraries) and tend to have defects in other parts. E.g. such compilers (two different ones) lately gave me wrong line numbers in error messages. And other people's (not mine, not really much) criticism is that they say garbage collection increases the use of memory and it can create a blocking of threads when accessing shared memory, or something like this. So... I wonder where the languages are that only try to give this type of error: Your faulty program has (at runtime) used memory which has already been freed. Not garbage collection. The compiled program just stops all execution and tells me this, so that I would go on with my manual memory management. Now, from today's perspective I could use Rust to create a very formal representation of my requirements and create a program that is very deterministic and at the same time uses very few resources. But I'd like to pretend there is no Rust (because the lifetimes and some other things make it a domain-specific language to some extent), and I would like to ask about the "runtime-solution". Why shouldn't it be a good thing? Has it been tried? All I would *need* to do additionally is dividing the project into two sub-projects as it is done with C++: Debug build an release build. Then the debug build would use a virtual machine that uses type information from compilation for garbage detection, but not garbage collection. And when I have tested all runtime cases of my compiled software, which runs slow, but quite deterministically, I will go on and build the release build. And if the release build (which is faster) does not behave deterministically, I would fix the "VM/Non-VM compiler" I'm talking about until the release build shows the same behavior. I guess there is a way this approach could fail: Timing may have influence and make the VM behave differently from the Non-VM (e.g. x64). And it's surely not easy to write a compiler that creates code which traces pointers and still leaves you much freedom to cast and alter pointers. In some way it is doomed to fail, but there are language constructs that work. There have been C interpreters, iterators as pointer replacements, or just any replacement. BTW I know of CINT and safe-c, but I'm not happy how these projects look from the outside. If I had the education and persistence I would like to try to build my own "safe-c", yet another one. But I think it's better to ask you why garbage detection isn't a popular thing. Does it exist at all as core idea in a language (probably a C improvement)? Where are the flaws in my thinking? I currently think, if I were serious about it (I'm not 100% sure), I should just find a C interpreter. CINT? Or this one academic compiler from five years ago? (I believe this compiler needs a special CPU) To be honest, I have no clue. Just one "interpreter" that tries to mimic pointers as much as it can, and later I would be free to port the code to Microsoft's C. Or maybe I could use the safe-c subset in D? But I believe it uses garbage collection. I know nothing about it, sorry. What I tried in the past few days was porting working Go code to C. I wanted the C code to be Go-idiomatic, and I was looking there for the common subset from Golang combined with C. Well, I used macros, had a few ideas, but then this C style quickly failed. Really frustrating. But..
C++ or D?
Hi all, my question would be about using D or not using D. Is the newest C++ iteration any good compared to D? The reason I haven't used C++ anymore for years is that I was too naive sometimes. I tried to use new features in Visual C++, found myself being like a beta-tester for some things. And the way C++ was always able to frustrate me is mainly, when undefined behavior takes too long to take effect. Sometimes I must have had bad luck, or I'm too silly. But for some subtle bugs in my hobbyist C++ past the solution would have been to rewrite a lot of code. Sometimes it was just too much work to get anything done at all. When I used shared pointers, it got better. But it got a little bit ugly, too. And in general, being encouraged or forced to write boiler plate code was also frustrating. I haven't looked into the newest C++. In theory, they might have added something helpful in the past years. Anyone have any thoughts how C++ and D compare?
Re: GtkD - how to list 0..100K strings
Continuing this in the GtkD mailing list: https://forum.gtkd.org/groups/GtkD/thread/1370/
Re: GtkD - how to list 0..100K strings
On Tuesday, 28 April 2020 at 18:46:18 UTC, Kagamin wrote:
Try this:
void populate(NameAndDescription[] namesAndDescriptions) {
if(namesAndDescriptions.length>100)namesAndDescriptions=namesAndDescriptions[0..100];
innerView.viewData.populate(namesAndDescriptions);
}
I tried that and it worked fine. So I then used a binary chop and
discovered that 0..n where n <= 1170 works fine; n > 1170 crashes.
I'm not sure where that takes me but seems suggestive that the
problem is Gtk or GtkD rather than my code?
Re: It won't run in gdb...
On Monday, 27 April 2020 at 12:26:23 UTC, Adam D. Ruppe wrote: On Mon, Apr 27, 2020 at 10:56:09AM +, mark via Digitalmars-d-learn wrote: Thread 1 "DebFind" received signal SIGUSR1, User defined signal 1. The GC sends that signal to pause other threads when it is about to collect. You can tell gdb to just ignore it. handle SIGUSR1 noprint handle SIGUSR2 noprint I added those to my .gdbinit personally. Thanks Adam, I took your advice and now have a bt. I have put it in a new message thread: "GtkD crash: 'BadAlloc (insufficient resources for operation)'"
Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'
I took Adam's advice about .gdbinit and now it runs in gdb. When I ran the program I did Find 'memoize' which worked. Then 'memoize python' which also worked. Then said to find 'any word' (which produces 1000s of rows) at which point it crashed. Below is the bt. Does it look like my bug or a Gtk or GtkD bug? $ gdb DebFind GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git [snip] Reading symbols from DebFind...done. (gdb) run Starting program: /home/mark/app/d/debfind/DebFind [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7fffeddfc700 (LWP 2441)] [New Thread 0x7fffed5fb700 (LWP 2442)] [New Thread 0x7fffe7fff700 (LWP 2443)] [New Thread 0x77fe2700 (LWP 2444)] [New Thread 0x77fdc700 (LWP 2445)] [New Thread 0x77fd6700 (LWP 2446)] [New Thread 0x77e64700 (LWP 2447)] [New Thread 0x77e5e700 (LWP 2448)] [New Thread 0x77e58700 (LWP 2449)] [New Thread 0x77e52700 (LWP 2450)] warning: Corrupted shared library list: 0x560a5e60 != 0x55dabe00 [New Thread 0x7fffc20ff700 (LWP 2451)] [New Thread 0x7fffc18fe700 (LWP 2452)] [Thread 0x7fffe7fff700 (LWP 2443) exited] [Thread 0x7fffc18fe700 (LWP 2452) exited] populateNames populateNames populateNames (DebFind:2436): Gdk-ERROR **: 15:48:40.080: The program 'DebFind' received an X Window System error. This probably reflects a bug in the program. The error was 'BadAlloc (insufficient resources for operation)'. (Details: serial 12790 error_code 11 request_code 130 (MIT-SHM) minor_code 5) (Note to programmers: normally, X errors are reported asynchronously; that is, you will receive the error a while after causing it. To debug your program, run it with the GDK_SYNCHRONIZE environment variable to change this behavior. You can then get a meaningful backtrace from your debugger if you break on the gdk_x_error() function.) Thread 1 "DebFind" received signal SIGTRAP, Trace/breakpoint trap. 0x766c3ea1 in ?? () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 (gdb) bt #0 0x766c3ea1 in () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #1 0x766c6819 in g_log_writer_default () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #2 0x766c4a8e in g_log_structured_array () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #3 0x766c54ce in g_log_structured_standard () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #4 0x75f62c41 in () at /usr/lib/x86_64-linux-gnu/libgdk-3.so.0 #5 0x75f6fac3 in () at /usr/lib/x86_64-linux-gnu/libgdk-3.so.0 warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) warning: (Internal error: pc 0x5579a280 in read in psymtab, but not in symtab.) warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) #6 0x737138fa in _XError () at /usr/lib/x86_64-linux-gnu/libX11.so.6 warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) #7 0x7371082b in () at /usr/lib/x86_64-linux-gnu/libX11.so.6 #8 0x737108d5 in () at /usr/lib/x86_64-linux-gnu/libX11.so.6 #9 0x73711205 in _XEventsQueued () at /usr/lib/x86_64-linux-gnu/libX11.so.6 #10 0x73702d3d in XPending () at /usr/lib/x86_64-linux-gnu/libX11.so.6 #11 0x75f6a09e in () at /usr/lib/x86_64-linux-gnu/libgdk-3.so.0 #12 0x766bdb28 in g_main_context_prepare () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #13 0x766be4fb in () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) #14 0x766be6dc in g_main_context_iteration () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) #15 0x7537eefd in g_application_run () at /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0 warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) #16 0x5579a2d5 in warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) _D3gio11ApplicationQn3runMFAAyaZiwarning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) (warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) this=0x77ed0360, warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) argv=...)warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) at Application.dwarning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) warning: (Internal error: pc 0x5579a2d4 in read in psymtab, but not in symtab.) :931 #17 0x55784380 in D main (args=...) at app.d:18
GtkD crash: 'BadAlloc (insufficient resources for operation)'
I'm getting a crash when I add 1000s of rows to a tree (up to 100s seems to work ok). The source code is here: https://github.com/mark-summerfield/debfind Note that this will only build and run on a Debian or Debian-derived system (e.g., Ubuntu). I am pretty well reaching the point of giving up with GtkD -- and therefore with GUI programming in D since there isn't really a decent reliable D GUI library that I can find. (I don't want to use Qt or QML because of the licensing issues.) (DebFind:3285): Gdk-ERROR **: 12:59:59.299: The program 'DebFind' received an X Window System error. This probably reflects a bug in the program. The error was 'BadAlloc (insufficient resources for operation)'. (Details: serial 25818 error_code 11 request_code 130 (MIT-SHM) minor_code 5) (Note to programmers: normally, X errors are reported asynchronously; that is, you will receive the error a while after causing it. To debug your program, run it with the GDK_SYNCHRONIZE environment variable to change this behavior. You can then get a meaningful backtrace from your debugger if you break on the gdk_x_error() function.) Program exited with code -5
It won't run in gdb...
: dub build Performing "debug" build using /home/mark/opt/ldc2-1.21.0-linux-x86_64/bin/ldc2 for x86_64. aaset 0.2.5: target for configuration "library" is up to date. gtk-d:gtkd 3.9.0: target for configuration "library" is up to date. debfind ~master: target for configuration "application" is up to date. To force a rebuild of up-to-date targets, run again with --force. : gdb DebFind GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git [snip] Reading symbols from DebFind...done. (gdb) run Starting program: /home/mark/app/d/debfind/DebFind [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7fffeddfc700 (LWP 8186)] [New Thread 0x7fffed5fb700 (LWP 8187)] [New Thread 0x7fffe7fff700 (LWP 8188)] [New Thread 0x77fe2700 (LWP 8189)] [New Thread 0x77fdc700 (LWP 8190)] [New Thread 0x77fd6700 (LWP 8191)] [New Thread 0x77e64700 (LWP 8192)] [New Thread 0x77e5e700 (LWP 8193)] [New Thread 0x77e58700 (LWP 8194)] [New Thread 0x77e52700 (LWP 8195)] Thread 1 "DebFind" received signal SIGUSR1, User defined signal 1. 0x750ba2ea in ?? () from /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (gdb) bt #0 0x750ba2ea in () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #1 0x750bbae5 in () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #2 0x71af0e28 in () at /lib/x86_64-linux-gnu/libexpat.so.1 #3 0x71af1bfc in () at /lib/x86_64-linux-gnu/libexpat.so.1 #4 0x71aef823 in () at /lib/x86_64-linux-gnu/libexpat.so.1 #5 0x71af050b in () at /lib/x86_64-linux-gnu/libexpat.so.1 #6 0x71af40ed in XML_ParseBuffer () at /lib/x86_64-linux-gnu/libexpat.so.1 #7 0x750bab43 in () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #8 0x750baf76 in FcConfigParseAndLoad () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #9 0x750bafe8 in FcConfigParseAndLoad () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #10 0x750bb15e in () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #11 0x71af0e28 in () at /lib/x86_64-linux-gnu/libexpat.so.1 #12 0x71af1bfc in () at /lib/x86_64-linux-gnu/libexpat.so.1 #13 0x71aef823 in () at /lib/x86_64-linux-gnu/libexpat.so.1 #14 0x71af050b in () at /lib/x86_64-linux-gnu/libexpat.so.1 #15 0x71af40ed in XML_ParseBuffer () at /lib/x86_64-linux-gnu/libexpat.so.1 #16 0x750bab43 in () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #17 0x750baf76 in FcConfigParseAndLoad () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #18 0x750add24 in () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #19 0x750adf76 in () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #20 0x750a1a27 in () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #21 0x750a3ab5 in FcConfigSubstituteWithPat () at /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 #22 0x75cfe718 in () at /usr/lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 #23 0x72c97552 in () at /usr/lib/x86_64-linux-gnu/libpangoft2-1.0.so.0 #24 0x75abfa67 in () at /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0 warning: (Internal error: pc 0x5584c444 in read in psymtab, but not in symtab.) warning: (Internal error: pc 0x5584c430 in read in psymtab, but not in symtab.) warning: (Internal error: pc 0x5584c444 in read in psymtab, but not in symtab.) #25 0x75ac10c0 in pango_itemize_with_base_dir () at /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0 warning: (Internal error: pc 0x5584c444 in read in psymtab, but not in symtab.) #26 0x75ac974d in () at /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0 #27 0x75acab32 in pango_layout_get_unknown_glyphs_count () at /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0 #28 0x7fffef522f8a in () at /usr/lib/x86_64-linux-gnu/libgtk-3.so.0 #29 0x7fffef522ff8 in () at /usr/lib/x86_64-linux-gnu/libgtk-3.so.0 #30 0x7fffef523168 in () at /usr/lib/x86_64-linux-gnu/libgtk-3.so.0 #31 0x769bd9c5 in g_type_create_instance () at /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 #32 0x7699e748 in () at /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 #33 0x7699fee5 in g_object_new_with_properties () at /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 #34 0x769a0961 in g_object_new () at /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 #35 0x5584c445 in _D3gtk5EntryQg6__ctorMFZCQxQvQx (this=0x77f98800) at Entry.d:215 #36 0x55749c80 in qtrac.debfind.appwindow.AppWindow.makeWidgets() (this=0x77ed7a00) at appwindow.d:85 #37 0x55749919 in _D5qtrac7debfind9appwindow9AppWindow6__ctorMFC3gtk11ApplicationQnZCQCnQCkQCfQBy (this=0x77ed7a00, application=0x77ed0360) at appwindow.d:62 #38 0x55784449 in _D5qtrac7debfind3app4mainFAAyaZ__T12__dgliteral2TC3gio11ApplicationQnZQBkMFQBaZv
Re: GtkD - how to list 0..100K strings: new problem
With the new code if I have 1000s of rows I get this error: (DebFind:8087): Gdk-ERROR **: 11:50:46.787: The program 'DebFind' received an X Window System error. This probably reflects a bug in the program. The error was 'BadAlloc (insufficient resources for operation)'. (Details: serial 8810 error_code 11 request_code 130 (MIT-SHM) minor_code 5) (Note to programmers: normally, X errors are reported asynchronously; that is, you will receive the error a while after causing it. To debug your program, run it with the GDK_SYNCHRONIZE environment variable to change this behavior. You can then get a meaningful backtrace from your debugger if you break on the gdk_x_error() function.) Program exited with code -5 This means the program is unusable (again) because there are often queries that result in 1000s of rows of results.
Re: GtkD - how to list 0..100K strings [solved]
I renamed the class shown in my previous post from View to
InnerView, then created a new View class:
class View : ScrolledWindow {
import qtrac.debfind.modelutil: NameAndDescription;
InnerView innerView;
this() {
super();
innerView = new InnerView;
addWithViewport(innerView);
}
void clear() {
innerView.viewData.clear;
}
void populate(NameAndDescription[] namesAndDescriptions) {
innerView.viewData.populate(namesAndDescriptions);
}
}
Re: GtkD - how to list 0..100K strings
I've now got it to work but it is unusable!
It can show small numbers of rows with no problem.
However, if it has to show 100s of rows it expands the tree
vertically way beyond the bottom of the screen and is impossible
to navigate.
However, if it has to show 1000s of rows it goes into an infinite
loop producing endless error messages:
(DebFind:6615): Gtk-WARNING **: 10:57:04.275: infinite
surface size not supported
I suppose I was expecting scrollbars to appear, but maybe with
Gtk you have do add them separately?
// Here's the view I'm now using
class View : TreeView {
import gtk.CellRendererText: CellRendererText;
import gtk.TreeViewColumn: TreeViewColumn;
import qtrac.debfind.modelutil: NameAndDescription;
import qtrac.debfind.viewdata: ViewData;
ViewData viewData;
TreeViewColumn nameColumn;
TreeViewColumn descriptionColumn;
this() {
super();
setActivateOnSingleClick(true);
viewData = new ViewData;
setModel(viewData);
auto renderer = new CellRendererText;
nameColumn = new TreeViewColumn("Name", renderer, "text",
0);
nameColumn.setResizable(true);
appendColumn(nameColumn);
renderer = new CellRendererText;
descriptionColumn = new TreeViewColumn("Description",
renderer,
"text", 1);
descriptionColumn.setResizable(true);
appendColumn(descriptionColumn);
}
void clear() {
viewData.clear;
}
void populate(NameAndDescription[] namesAndDescriptions) {
viewData.populate(namesAndDescriptions);
}
}
// Here's the ListStore I'm using:
class ViewData : ListStore {
import qtrac.debfind.modelutil: NameAndDescription;
this() {
import gobject.c.types: GType;
super([GType.STRING, GType.STRING]);
}
void populate(NameAndDescription[] namesAndDescriptions) {
import gtk.TreeIter: TreeIter;
clear;
TreeIter iter;
foreach (i, nameAndDescription; namesAndDescriptions) {
append(iter);
setValue(iter, 0, nameAndDescription.name);
setValue(iter, 1,
nameAndDescription.description);
}
}
}
// I have the TreeView side-by-side with a TextView: here're
snippets:
Paned splitter;
View debsView;
TextView debTextView;
...
splitter = new Paned(GtkOrientation.HORIZONTAL);
splitter.setWideHandle(true);
debsView = new View;
debsView.setHexpand(true);
debsView.setVexpand(true);
debTextView = new TextView;
...
auto grid = new Grid;
...
splitter.pack1(debsView, false, true);
splitter.pack2(debTextView, true, true);
grid.attach(splitter, 0, 3, 6, 1);
GtkD - how to list 0..100K strings
I'm trying to develop an application in GtkD.
I need a widget to display a list of strings: there could be
anything from 0 to 100K strings, but typically a few hundred or
thousand.
Using the DemoCustomList as a model I have created this code:
// Note: DebNames is an AAset!string (AAset is a wrapper around a
D AA, so in this case a set of strings)
// namestore.d
import gtk.ListStore: ListStore;
class NameStore : ListStore {
this(DebNames names) {
import gobject.c.types: GType;
import gtk.TreeIter: TreeIter;
super([GType.STRING]);
TreeIter iter;
foreach (name; names) {
append(iter);
setValue(iter, 0, name);
}
}
}
// appwindow.d
final class AppWindow: ApplicationWindow {
TreeView debsTreeView;
NameStore nameStore;
// ...
private void populateNames(DebNames names) {
import gtk.CellRendererText: CellRendererText;
import gtk.TreeViewColumn: TreeViewColumn;
nameStore = new NameStore(names);
auto column = new TreeViewColumn;
auto renderer = new CellRendererText;
column.packStart(renderer, true);
column.addAttribute(renderer, "text", 0);
column.setTitle("Names");
debsTreeView.appendColumn(column);
}
}
When populateNames() is called the treeview expands horizontally
but shows nothing, so I'm stuck.
Can anyone help?
Note that I don't have to use a tree widget if there's a better
one for this use case. I did see ListBox but that seemed to be a
list of widgets which would be a bit heavy for 100K strings?
Re: can a unittest read main()'s args?
On Sunday, 22 March 2020 at 07:59:01 UTC, rikki cattermole wrote:
On 22/03/2020 8:57 PM, mark wrote:
I have a module with a unittest { ... } block. However, when I
run dub test sometimes I want to output some extra data when
the test runs. At the moment I control this by using an
environment variable, but I wondered if it was possible to
pass a command line argument 'dub test myarg' and if so how to
access it?
dub run -- args
https://dlang.org/phobos/core_runtime.html#.Runtime.args
Thanks, that works great!
can a unittest read main()'s args?
I have a module with a unittest { ... } block. However, when I
run dub test sometimes I want to output some extra data when the
test runs. At the moment I control this by using an environment
variable, but I wondered if it was possible to pass a command
line argument 'dub test myarg' and if so how to access it?
Re: A set type implemented as an AA wrapper
On Thursday, 12 March 2020 at 16:02:14 UTC, H. S. Teoh wrote:
On Thu, Mar 12, 2020 at 08:51:24AM +, mark via
Digitalmars-d-learn wrote: [...]
YYY: The range() method is clearly not good D style but I
don't know
how to support foreach (item; aaset) ...
The usual idiom is to overload .opSlice, then you can do:
foreach (item; aaset[]) { ... }
IOW rename .range to .opSlice.
ZZZ: I can't figure out how to support the in operator.
Note that 'x in aa' returns a pointer, not a bool. You could
try:
bool opBinaryRight(string op : "in")(T lhs) {
return (lhs in set) !is null;
}
I renamed .range to .opSlice (and deleted the alias range this)
and foreach(item; aaset) works fine.
As for the != null, that was a mistake (due to Java) which I've
now fixed.
I'm hoping to add union & intersection soon too.
Thanks.
Re: A set type implemented as an AA wrapper
Just in case anyone would like to use it, I've put it on github and also added it as a dub package. https://code.dlang.org/packages/aaset
Re: A set type implemented as an AA wrapper
On Thursday, 12 March 2020 at 11:33:25 UTC, Simen Kjærås wrote:
[snip]
I'd suggest simply testing if an AA with that key type is valid:
struct AAset(T) if (is(int[T]))
That's very subtle, but it works.
As Ferhat points out, you could use opApply for this. There's
also the option of implenting the range primitives front,
popFront() and empty. However, the easiest solution is the one
you've already chosen, combined with alias this:
struct AAset(T) if (is(int[T])) {
// stuffs...
auto range() { return set.byKey; }
alias range this;
}
Again, subtle, and again it works!
I would also suggest using a template specialization instead of
static if and static assert:
bool opBinaryRight(string op : "in")(T lhs) {
return (lhs in set) != null;
}
Thanks, you've solved all the issued I had! Here's the complete
revised code:
struct AAset(T) if (is(int[T])) {
private {
alias Unit = void[0];
enum unit = Unit.init;
Unit[T] set;
}
size_t length() const { return set.length; }
void add(T item) { set[item] = unit; }
bool remove(T item) { return set.remove(item); }
auto range() { return set.byKey; }
alias range this;
bool opBinaryRight(string op: "in")(T lhs) {
return (lhs in set) != null;
}
}
unittest {
import std.algorithm: sort;
import std.array: array;
import std.range: enumerate;
import std.stdio: writeln;
import std.typecons: Tuple;
writeln("unittest for the aaset library.");
alias Pair = Tuple!(int, "count", string, "word");
immutable inputs = [Pair(1, "one"), Pair(2, "two"), Pair(3,
"three"),
Pair(4, "four"), Pair(4, "two"), Pair(5,
"five"),
Pair(6, "six")];
AAset!string words;
assert(words.length == 0);
foreach (pair; inputs) {
words.add(pair.word);
assert(words.length == pair.count);
}
immutable len = words.length;
assert(!words.remove("missing"));
assert(words.remove("one"));
assert(words.length == len - 1);
immutable expected = ["five", "four", "six", "three", "two"];
foreach (i, word; words.array.sort.enumerate)
assert(word == expected[i]);
assert("Z" !in words);
assert("three" in words);
}
A set type implemented as an AA wrapper
I use sets a lot and since I believe that D's rbtree is O(lg n)
for add/remove/in and that D's AA is O(1) for these, I want to
implement a set in terms of an AA.
Below is the code I've got so far. It allows for add and remove.
However, it has three problems (that I know of):
XXX: I need to use an if on the struct to restrict T to be a type
that supports toHash and opEquals (i.e., to be a valid AA key)
YYY: The range() method is clearly not good D style but I don't
know how to support foreach (item; aaset) ...
ZZZ: I can't figure out how to support the in operator.
// XXX how do I write the if to ensure T is a valid AA key that
suppors
// toHash & opEquals?
struct AAset(T) {
private {
alias Unit = void[0];
enum unit = Unit.init;
Unit[T] set;
}
size_t length() const { return set.length; }
void add(T item) { set[item] = unit; }
bool remove(T item) { return set.remove(item); }
// YYY is there a better way so that people can just use
// foreach (var; aaset)
auto range() { return set.byKey; }
bool opBinaryRight(string op)(T lhs) { // ZZZ doesn't work
static if (op == "in") return lhs in set;
else static assert(0, "operator " ~ op ~ " not
supported");
}
// TODO union(), intersection(), difference(),
symmetric_difference()
}
unittest {
import std.algorithm: sort;
import std.array: array;
import std.range: enumerate;
import std.stdio: writeln;
import std.typecons: Tuple;
alias Pair = Tuple!(int, "count", string, "word");
auto inputs = [Pair(1, "one"), Pair(2, "two"), Pair(3,
"three"),
Pair(4, "four"), Pair(4, "two"), Pair(5,
"five"),
Pair(6, "six")];
AAset!string words;
assert(words.length == 0);
foreach (pair; inputs) {
words.add(pair.word);
assert(words.length == pair.count);
}
auto len = words.length;
assert(!words.remove("missing"));
assert(words.remove("one"));
assert(words.length == len - 1);
auto expected = ["five", "four", "six", "three", "two"];
foreach (i, word; words.range.array.sort.enumerate)
assert(word == expected[i]);
/*
assert("Z" !in words);
assert("three" !in words);
*/
}
Re: Aliases to mutable thread-local data not allowed [testable source code]
On Wednesday, 11 March 2020 at 14:01:13 UTC, Simen Kjærås wrote:
[snip]
Yeah, I forgot we cast to immutable to be able to send, so
receive has to receive immutable(Deb)*, after which you can
call deb.dup to get a mutable copy:
receive(
(immutable(Deb)* deb) { debForName[deb.name] = deb.dup;
},
(DoneMessage m) { jobs--; }
);
Thanks, that fixed it.
However, timing-wise the single threaded version (st) is fastest,
then the task multi-threaded version (mt), and finally this
version (mto).
Re: Aliases to mutable thread-local data not allowed [testable source code]
On Wednesday, 11 March 2020 at 12:22:21 UTC, Simen Kjærås wrote: On Wednesday, 11 March 2020 at 09:29:54 UTC, mark wrote: [snip] Fascinating. It works just fine when compiling for 32-bit targets with DMD on Windows, but not for 64-bit targets, nor when compiling with LDC. Apparently, this difference is due to DMD supporting 80-bit reals, and thus giving a different size to Variant (VariantN!20 on DMD on Windows, VariantN!16 or VariantN!32 elsewhere). There's a bug in VariantN that then causes the compilation to fail (https://issues.dlang.org/show_bug.cgi?id=20666). The issue at hand then, is that Deb is too big until that issue if fixed. The simple solution to this is to allocate Deb on the heap with new and pass pointers instead of instances directly. Since you are already calling .dup whenever you pass a Deb somewhere, you can simply modify .dup to return a Deb* and the receive function to receive a Deb*, and I think you should be good to go. I did that and it compiles & runs, but no Debs get added to the collection. See https://github.com/mark-summerfield/d-debtest-experiment -- the 'mto' version is the one with your fixes.
Re: Aliases to mutable thread-local data not allowed [solved-ish]
I finally got a threaded version that works, and a lot more
cleanly than using send/receive. (But performance is dismal, see
the end.)
Here's the heart of the solution:
void readPackages() {
import std.algorithm: max;
import std.array: array;
import std.parallelism: taskPool, totalCPUs;
import std.file: dirEntries, FileException, SpanMode;
try {
auto filenames = dirEntries(PACKAGE_DIR,
PACKAGE_PATTERN,
SpanMode.shallow).array;
foreach (debs;
taskPool.map!readPackageFile(filenames))
foreach (deb; debs)
debForName[deb.name] = deb.dup;
} catch (FileException err) {
import std.stdio: stderr;
stderr.writeln("failed to read packages: ", err);
}
}
I had to change readPackageFile (and the functions it calls),
e.g.,:
Deb[] readPackageFile(string filename) {
import std.file: FileException;
import std.range: enumerate;
import std.stdio: File, stderr;
Deb[] debs;
Deb deb;
try {
bool inDescription = false; // Descriptions can by
multi-line
bool inContinuation = false; // Other things can be
multi-line
auto file = File(filename);
foreach(lino, line; file.byLine.enumerate(1))
readPackageLine(debs, deb, filename, lino, line,
inDescription,
inContinuation);
if (deb.valid)
debs ~= deb.dup;
} catch (FileException err) {
stderr.writefln("error: %s: failed to read packages: %s",
filename, err);
}
return debs;
}
I also changed main() to do some timings & to allow me to compare
outputs:
void main(const string[] args) {
import std.datetime.stopwatch: AutoStart, StopWatch;
import std.stdio: stderr, writeln;
auto model = Model();
auto timer = StopWatch(AutoStart.yes);
model.readPackages();
stderr.writefln("read %,d packages in %s", model.length,
timer.peek);
if (args.length > 1)
foreach (deb; model.debForName)
writeln(deb);
}
This produces the same output as the single-threaded version.
Here's the output of a typical single-threaded version's run:
read 65,480 packages in 1 sec, 314 ms, 798 μs, and 7 hnsecs
And here's the output of a typical task-based multi-threaded
version's run:
read 65,480 packages in 1 sec, 377 ms, 605 μs, and 3 hnsecs
In fact, the multi-threaded has never yet been as fast as the
single-threaded version!
I've put both versions on github in case anyone's interested:
https://github.com/mark-summerfield/d-debtest-experiment
Re: Aliases to mutable thread-local data not allowed [testable source code]
Hi Simen,
I think you must have done something else but didn't mention to
get it to compile. I did the exact changes you said and it
wouldn't compile. Here's what I get with changes mentioned below
(with new full source):
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/variant.d(701,26):
Error: cannot implicitly convert expression rhs of type immutable(Deb) to Deb
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/variant.d(603,17):
Error: template instance
std.variant.VariantN!32LU.VariantN.opAssign!(immutable(Deb)) error instantiating
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(126,22):
instantiated from here: __ctor!(immutable(Deb))
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(656,23):
instantiated from here: __ctor!(immutable(Deb))
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(647,10):
instantiated from here: _send!(immutable(Deb))
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(626,10):
instantiated from here: _send!(immutable(Deb))
src/app.d(89,17):instantiated from here:
send!(immutable(Deb))
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/ldc2 failed with exit
code 1.
Here is a modified version that names the DoneMessage variable
and makes readPackageFile and readPackageLine into functions.
Instead of making them static, I just took them outside the
struct: after all, they never access struct member data. I also
now use ownerTid.
// app.d
import std.typecons: Tuple;
void main() {
import std.stdio: writeln, writefln;
auto model = Model();
model.readPackages();
writefln("read %,d packages", model.length);
}
enum PACKAGE_DIR = "/var/lib/apt/lists";
enum PACKAGE_PATTERN = "*Packages";
alias Unit = void[0]; // These two lines allow me to use AAs as
sets
enum unit = Unit.init;
alias MaybeKeyValue = Tuple!(string, "key", string, "value",
bool, "ok");
struct DoneMessage {}
struct Deb {
string name;
string description;
Unit[string] tags; // set of tags
Deb dup() const {
Deb deb;
deb.name = name;
deb.description = description;
foreach (key; tags.byKey)
deb.tags[key] = unit;
return deb;
}
bool valid() {
import std.string: empty;
return !name.empty && !description.empty;
}
void clear() {
name = "";
description = "";
tags.clear;
}
}
struct Model {
import std.concurrency: Tid;
private Deb[string] debForName; // Only read once populated
size_t length() const { return debForName.length; }
void readPackages() {
import std.concurrency: receive, spawn;
import std.file: dirEntries, FileException, SpanMode;
Tid[] tids;
try {
foreach (string filename; dirEntries(PACKAGE_DIR,
PACKAGE_PATTERN,
SpanMode.shallow))
tids ~= spawn(, filename);
auto jobs = tids.length;
while (jobs) {
receive(
(Deb deb) { debForName[deb.name] = deb; },
(DoneMessage m) { jobs--; }
);
}
} catch (FileException err) {
import std.stdio: stderr;
stderr.writeln("failed to read packages: ", err);
}
}
}
void readPackageFile(string filename) {
import std.concurrency: ownerTid, send;
import std.file: FileException;
import std.range: enumerate;
import std.stdio: File, stderr;
try {
bool inDescription = false; // Descriptions can by
multi-line
bool inContinuation = false; // Other things can be
multi-line
Deb deb;
auto file = File(filename);
foreach(lino, line; file.byLine.enumerate(1))
readPackageLine(filename, lino, line, deb,
inDescription,
inContinuation);
if (deb.valid)
send(ownerTid, cast(immutable)deb.dup);
} catch (FileException err) {
stderr.writefln("error: %s: failed to read packages: %s",
filename, err);
}
send(ownerTid, DoneMessage());
}
void readPackageLine(const string filename, const int lino,
const(char[]) line, ref Deb deb,
ref bool inDescription, ref bool
inContinuation) {
import std.concurrency: ownerTid, send;
import std.path: baseName;
import std.stdio: stderr;
import std.string: empty, startsWith, strip;
if (strip(line).empty) {
if (deb.valid)
send(ownerTid, cast(immutable)deb.dup);
else if (!deb.name.empty || !deb.description.empty ||
!deb.tags.empty)
stderr.writefln("error: %s:%,d: incomplete package:
%s",
baseName(filename), lino, deb);
deb.clear;
Re: Aliases to mutable thread-local data not allowed [testable source code]
I've managed to make a cut-down version that's < 170 LOC.
It needs to be run on Debian or a Debian-based Linux (e.g.,
Ubuntu).
Below is the full source followed by the error output.
// app.d
import std.typecons: Tuple;
void main() {
import std.stdio: writeln, writefln;
auto model = Model();
model.readPackages();
writefln("read %,d packages", model.length);
}
enum PACKAGE_DIR = "/var/lib/apt/lists";
enum PACKAGE_PATTERN = "*Packages";
alias Unit = void[0];
enum unit = Unit.init;
alias MaybeKeyValue = Tuple!(string, "key", string, "value",
bool, "ok");
struct DoneMessage {}
struct Deb {
string name;
string description;
Unit[string] tags; // set of tags
Deb dup() const {
Deb deb;
deb.name = name;
deb.description = description;
foreach (key; tags.byKey)
deb.tags[key] = unit;
return deb;
}
bool valid() {
import std.string: empty;
return !name.empty && !description.empty;
}
void clear() {
name = "";
description = "";
tags.clear;
}
}
struct Model {
import std.concurrency: Tid;
private Deb[string] debForName; // Only read once populated
size_t length() const { return debForName.length; }
void readPackages() {
import std.concurrency: receive, thisTid, spawn;
import std.file: dirEntries, FileException, SpanMode;
Tid[] tids;
try {
foreach (string filename; dirEntries(PACKAGE_DIR,
PACKAGE_PATTERN,
SpanMode.shallow))
tids ~= spawn(, thisTid,
filename);
auto jobs = tids.length;
while (jobs) {
receive(
(Deb deb) { debForName[deb.name] = deb; },
(DoneMessage) { jobs--; }
);
}
} catch (FileException err) {
import std.stdio: stderr;
stderr.writeln("failed to read packages: ", err);
}
}
private void readPackageFile(Tid parentTid, string filename) {
import std.concurrency: send;
import std.file: FileException;
import std.range: enumerate;
import std.stdio: File, stderr;
try {
bool inDescription = false; // Descriptions can by
multi-line
bool inContinuation = false; // Other things can be
multi-line
Deb deb;
auto file = File(filename);
foreach(lino, line; file.byLine.enumerate(1))
readPackageLine(parentTid, filename, lino, line,
deb,
inDescription, inContinuation);
if (deb.valid)
send(parentTid, cast(immutable)deb.dup);
} catch (FileException err) {
stderr.writefln("error: %s: failed to read packages:
%s",
filename, err);
}
send(parentTid, DoneMessage());
}
private void readPackageLine(
Tid parentTid, const string filename, const int lino,
const(char[]) line, ref Deb deb, ref bool
inDescription,
ref bool inContinuation) {
import std.concurrency: send;
import std.path: baseName;
import std.stdio: stderr;
import std.string: empty, startsWith, strip;
if (strip(line).empty) {
if (deb.valid)
send(parentTid, cast(immutable)deb.dup);
else if (!deb.name.empty || !deb.description.empty ||
!deb.tags.empty)
stderr.writefln("error: %s:%,d: incomplete
package: %s",
baseName(filename), lino, deb);
deb.clear;
return;
}
if (inDescription || inContinuation) {
if (line.startsWith(' ') || line.startsWith('\t')) {
if (inDescription)
deb.description ~= line;
return;
}
inDescription = inContinuation = false;
}
immutable keyValue = maybeKeyValue(line);
if (!keyValue.ok)
inContinuation = true;
else
inDescription = populateDeb(deb, keyValue.key,
keyValue.value);
}
}
MaybeKeyValue maybeKeyValue(const(char[]) line) {
import std.string: indexOf, strip;
immutable i = line.indexOf(':');
if (i == -1)
return MaybeKeyValue("", "", false);
immutable key = strip(line[0..i]).idup;
immutable value = strip(line[i + 1..$]).idup;
return MaybeKeyValue(key, value, true);
}
bool populateDeb(ref Deb deb, const string key, const string
value) {
import std.conv: to;
switch (key) {
case "Package":
deb.name = value;
return false;
case "Description", "Npp-Description": // XXX ignore Npp-?
deb.description ~= value;
Re: Aliases to mutable thread-local data not allowed
On Tuesday, 10 March 2020 at 15:27:04 UTC, Steven Schveighoffer wrote: On 3/10/20 7:09 AM, mark wrote: [snip] Still, the correct thing here is to handle immutable(Deb). However, I'd strongly caution you against casting away immutable, that can lead to undefined behavior in D. Better to just store it as immutable to begin with. I'm happy to store immutable Debs in Deb[string] debForName; but the syntax immutable(Deb)[string] debForName; just produced errors. [snip] Hard to see what the problem is here, it looks like you have a function that has unreachable statements, which is why the spawn isn't compiling (the function you are passing it can't compile). They are only unreachable (I think) because of the stuff further down the backtrace. [snip] This looks like an error in std.concurrency (or std.variant), and I'm not sure what is the problem here. It seems std.variant cannot accept an immutable, which makes no sense as immutable data is a huge usecase of std.concurrency. Can you narrow this down to a small example that can be submitted as a bug report? I'll try to put the project on github -- if I ever get it working it'll be GPL, just that I don't like putting up pre-alpha stuff. So then I'll add the link. Thanks.
Re: Aliases to mutable thread-local data not allowed
I just tried:
auto jobs = tids.length;
while (jobs) {
receive(
(Deb deb) { debForName[deb.name] =
cast(Deb)deb; },
(DoneMessage m) { jobs--; }
);
}
to cast away the immutable from the sender, but that doesn't work
either:
src/model.d(85,30): Error: template std.concurrency.spawn cannot
deduce function from argument types !()(void delegate(Tid
parentTid, string filename), Tid, string), candidates are:
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(460,5):
spawn(F, T...)(F fn, T args)
with F = void delegate(Tid, string),
T = (Tid, string)
must satisfy the following constraint:
isSpawnable!(F, T)
src/model.d(86,13): Warning: statement is not reachable
src/model.d(87,13): Warning: statement is not reachable
src/model.d(86,13): Warning: statement is not reachable
src/model.d(87,13): Warning: statement is not reachable
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/variant.d(701,26):
Error: cannot implicitly convert expression rhs of type immutable(Deb) to Deb
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/variant.d(603,17):
Error: template instance
std.variant.VariantN!32LU.VariantN.opAssign!(immutable(Deb)) error instantiating
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(126,22):
instantiated from here: __ctor!(immutable(Deb))
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(656,23):
instantiated from here: __ctor!(immutable(Deb))
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(647,10):
instantiated from here: _send!(immutable(Deb))
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(626,10):
instantiated from here: _send!(immutable(Deb))
src/model.d(115,21):instantiated from here:
send!(immutable(Deb))
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/ldc2 failed with exit
code 1.
:
Re: Aliases to mutable thread-local data not allowed
On Tuesday, 10 March 2020 at 10:02:16 UTC, Simen Kjærås wrote: [snip] As the error message hints at, the problem is Deb may hold references to data that is shared with other objects on the thread from which it originates. Since you know this is not the case, even if the compiler can't prove it, you can safely cast your Deb to immutable: if (deb.valid) send(parentTid, cast(immutable)deb.dup); In fact, even the .dup is unnecessary here, since no data is shared with other objects, so you can simply write send(parentTid, cast(immutable)deb);. (Note on this point: since you have not included all your code, it could be other parts create shared mutable state, in which case .dup is necessary, and if badly written may not be sufficient. Thanks, that's led to some progress. Once each Deb is populated its contents are never modified. Nonetheless I've kept with .dup since I reuse the same Deb each time to populate, then send a copy, then clear and reuse the original. Unfortunately, I'm now getting different errors: src/model.d(85,30): Error: template std.concurrency.spawn cannot deduce function from argument types !()(void delegate(Tid parentTid, string filename), Tid, string), candidates are: /home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(460,5): spawn(F, T...)(F fn, T args) with F = void delegate(Tid, string), T = (Tid, string) must satisfy the following constraint: isSpawnable!(F, T) src/model.d(86,13): Warning: statement is not reachable src/model.d(87,13): Warning: statement is not reachable src/model.d(86,13): Warning: statement is not reachable src/model.d(87,13): Warning: statement is not reachable /home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/variant.d(701,26): Error: cannot implicitly convert expression rhs of type immutable(Deb) to Deb /home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/variant.d(603,17): Error: template instance std.variant.VariantN!32LU.VariantN.opAssign!(immutable(Deb)) error instantiating /home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(126,22): instantiated from here: __ctor!(immutable(Deb)) /home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(656,23): instantiated from here: __ctor!(immutable(Deb)) /home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(647,10): instantiated from here: _send!(immutable(Deb)) /home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(626,10): instantiated from here: _send!(immutable(Deb)) src/model.d(115,21):instantiated from here: send!(immutable(Deb)) /home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/ldc2 failed with exit code 1. Should I be somehow casting away the immutable at the receive end? (Even though received Debs are never modified?)
Aliases to mutable thread-local data not allowed
I have this struct:
struct Deb {
string name;
...
Unit[string] tags; // set of tags
Deb dup() const {
Deb deb;
deb.name = name;
...
foreach (key; tags.byKey)
deb.tags[key] = unit;
return deb;
}
}
And I want to populate an AA of them:
Deb[string] debForName;
This takes 1.5 sec which is too slow.
So now I'm trying to read the files concurrently:
private struct DoneMessage {}
void initialize() { // main thread (some code elided)
Tid[] tids;
try {
foreach (string filename; dirEntries(PATH_PATTERN,
SpanMode.shallow))
tids ~= spawn(, thisTid, filename);
auto jobs = tids.length;
while (jobs) {
receive(
(Deb deb) { debForName[deb.name] = deb; },
(DoneMessage m) { jobs--; }
);
}
} catch (FileException err) {
stderr.writeln("failed to read packages: ", err);
}
}
// This is called once per child thread and calls send() 1000s of
times to
// return Deb structs, finally sending DoneMessage.
void readPackageFile(Tid parentTid, string filename) { // (some
code elided)
try {
Deb deb;
auto file = File(filename);
foreach(lino, line; file.byLine.enumerate(1))
readPackageLine(parentTid, filename, lino, line, deb);
// readPackageLine also calls send with same code as
AAA below
if (deb.valid)
send(parentTid, deb.dup); // AAA
} catch (FileException err) {
stderr.writeln(err);
}
send(parentTid, DoneMessage());
}
Unfortunately, I can't send Debs:
src/model.d(71,30): Error: template std.concurrency.spawn cannot
deduce function from argument types !()(void delegate(Tid
parentTid, string filename), Tid, string), candidates are:
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(460,5):
spawn(F, T...)(F fn, T args)
with F = void delegate(Tid, string),
T = (Tid, string)
must satisfy the following constraint:
isSpawnable!(F, T)
src/model.d(72,13): Warning: statement is not reachable
src/model.d(73,13): Warning: statement is not reachable
src/model.d(79,13): Warning: statement is not reachable
src/model.d(72,13): Warning: statement is not reachable
src/model.d(73,13): Warning: statement is not reachable
src/model.d(79,13): Warning: statement is not reachable
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/concurrency.d(625,5): Error:
static assert: "Aliases to mutable thread-local data not allowed."
src/model.d(102,21):instantiated from here: send!(Deb)
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/ldc2 failed with exit
code 1.
Is there any nice solution to this? Surely it is a common pattern
to read multiple files and create lots of data items to be merged
into a collection?
Re: An struct copy constructor that can cope with an AA?
On Monday, 9 March 2020 at 14:45:15 UTC, Steven Schveighoffer wrote: On 3/9/20 9:23 AM, mark wrote: I have this struct: [snip] I would name it dup instead of copy for consistency with D. A copy constructor is pretty heavy for a struct to do a complete duplication of the AA. You should have to opt-in to that. -Steve Thanks, I'd already realised I ought to rename it .dup. Also, I only actually need to create it from the keys since I'm using the tags AA as a set so every value is the same. I did try tags.dup but got this: src/deb.d(25,24): Error: cannot implicitly convert expression dup(this.tags) of type const(void[0])[string] to void[0][string]
An struct copy constructor that can cope with an AA?
I have this struct:
struct Deb {
string name;
...
Unit[string] tags; // set of tags
Deb copy() const {
Deb deb;
...
foreach (key, value; tags) // XXX
deb.tags[key] = value;
return deb;
}
void clear() {
name = "";
...
tags.clear;
}
...
}
I am populating an AA with these structs:
Deb[string] debForName;
I'm using this approach (pseudo-code):
Deb deb;
foreach (datum; data) {
populateDeb(datum, deb);
debForName[deb.name] = deb.copy; // YYY
deb.clear;
}
(1) XXX Is there a nicer way to copy an AA?
(2) YYY Is there a nicer way to copy a struct? Use a copy
constructor or implement a dup or idup?
Re: How to use sets in D?
On Sunday, 8 March 2020 at 17:58:16 UTC, Jesse Phillips wrote: On Friday, 7 February 2020 at 19:37:08 UTC, mark wrote: [snip] I think I've usually used the associative arrays, but I also think I tend to avoid using this approach but couldn't quite remember what I do instead. I believe I have started just using an array. arr ~= addMyData; arr.sort.uniq Then I make use of the algorithms here. https://dlang.org/phobos/std_algorithm_setops.html I can see the sense in that for use with union & intersection. However, AA's (assuming they are hashes under the hood) give O(1) for `in` whereas those algorithms like the rbtree are O(lg n) for `in`, and most of what I'm doing at the moment is `in`. I'll keep the idea in mind though, because I have use cases for intersection.
Re: How to use sets in D?
I use sets a lot and am now working on a program that will need
to hold sets of 65,000+ items, so I thought I do some timings for
the different approaches.
Here are some timings (uset uses the AA Unit approach, tset uses
an rbtree, and aset uses an AA with bool values):
$ ./sets.d
size 50,000
uset 1 ms, 340 μs, and 8 hnsecs
tset 4 ms, 637 μs, and 1 hnsec
aset 1 ms, 402 μs, and 6 hnsecs
$ ./sets.d
size 100,000
uset 2 ms, 338 μs, and 4 hnsecs
tset 12 ms, 262 μs, and 6 hnsecs
aset 2 ms and 991 μs
$ ./sets.d
size 200,000
uset 5 ms, 971 μs, and 5 hnsecs
tset 30 ms, 675 μs, and 5 hnsecs
aset 6 ms, 74 μs, and 6 hnsecs
$ ./sets.d
size 400,000
uset 11 ms, 823 μs, and 4 hnsecs
tset 74 ms, 146 μs, and 2 hnsecs
aset 12 ms, 560 μs, and 5 hnsecs
What seems pretty clear is that for my purposes (checking
presence or absence of membership in a set), AAs are much faster
than rbtrees. (This is to be expected since if an AA uses a hash
is should be O(1) vs O(lg n) for an rbtree).
Here is the test code I used:
#!/usr/bin/env rdmd
enum SIZE = 400_000;
enum SUB_SIZE = SIZE / 10;
enum MIN_LEN = 10;
enum MAX_LEN = 50;
enum AZ = "abcdefghijklmnopqrstuvwxyz";
void main() {
import std.stdio: writefln;
auto data = Data.populate();
auto sets = Sets(data.words);
writefln("size %,d", SIZE);
check(sets.uset, data.present, data.absent, "uset");
check(sets.tset, data.present, data.absent, "tset");
check(sets.aset, data.present, data.absent, "aset");
}
struct Sets {
import std.container.rbtree: RedBlackTree;
alias Unit = void[0];
enum unit = Unit.init;
Unit[string] uset;
RedBlackTree!string tset;
bool[string] aset;
this(string[] words) {
tset = new RedBlackTree!string;
foreach (word; words) {
uset[word] = unit;
tset.insert(word);
aset[word] = false;
}
}
}
struct Data {
string[] words;
string[] present;
string[] absent;
static Data populate() {
Data data;
for (int i = 0; i < SIZE; i++) {
auto word = makeWord;
data.words ~= word;
if (data.present.length < SUB_SIZE)
data.present ~= word;
if (data.absent.length < SUB_SIZE)
data.absent ~= word ~ "9";
}
return data;
}
}
string makeWord() {
import std.random: randomCover, uniform;
import std.range: take;
import std.string: join, split;
enum AZS = (AZ ~ AZ ~ AZ ~ AZ ~ AZ ~ AZ).split("");
return randomCover(AZS).take(uniform(MIN_LEN,
MAX_LEN)).join("");
}
void check(T)(T set, string[] present, string[] absent, string
name) {
import std.datetime.stopwatch: AutoStart, StopWatch;
import std.stdio: writeln;
auto timer = StopWatch(AutoStart.yes);
foreach (string p; present)
assert(p in set);
foreach (string a; absent)
assert(a !in set);
writeln(name, " ", timer.peek);
}
Re: use of struct vs class
Steve, thank you once again. Now it compiles & runs!
I now create my tree like this:
auto debs = new RedBlackTree!(Deb, (a, b) => a.name < b.name);
(I feel that the rbtree docs are inadequate regarding creating
new empty trees, so have submitted a bug report:
https://issues.dlang.org/show_bug.cgi?id=20646 )
The other problem I had was that the Deb I created on the stack
had its own internal rbtree that was always null.
To address this I changed tags in the Deb struct to this:
auto tags = new RedBlackTree!string;
so it always creates a new empty tree. And I renamed clear to
reset and now do this:
void reset() {
name = "";
...
tags = new RedBlackTree!string;
...
}
I assume that I can safely rely on the GC to clean up for me.
So now I think I can safely use the pattern (1., repeat 2.1, ...
) described earlier: certainly it builds and runs.
Next I'll have to try really populating the Debs including their
tag trees and populating the debs tree of Debs structs.
Thank you.
Re: use of struct vs class
I've now gone back to using structs direct without pointers but
I'm still doing something wrong.
struct Deb {
string name;
...
RedBlackTree!string tags;
bool valid() { return !(name.empty || description.empty); }
void clear() { name = ""; ...; tags.clear; }
}
RedBlackTree!(Deb, (a, b) => a.name < b.name) debs;
Deb deb;
auto file = File(filename);
foreach(line; file.byLine) {
line = strip(line);
if (line.empty) {
if (deb.valid)
debs.insert(deb); // XXX
else // report incomplete
deb.clear;
continue;
}
...
}
if (deb.valid)
debs.insert(deb);
This compiles & crashes with "Program exited with code -11".
What I'm trying to do but clearly don't understand is this:
1. create a struct
repeat:
2.1. populate the struct
2.2. copy the struct into an rbtree // XXX
2.3. clear the original struct
2.4. loop
At XXX does a copy take place? I thought it did with structs?
Anyway, here's the backtrace:
$ gdb DebFind
GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
...
Reading symbols from DebFind...done.
(gdb) run
Starting program: /home/mark/app/d/debfind/DebFind
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x76e02700 (LWP 2366)]
...
Thread 1 "DebFind" received signal SIGSEGV, Segmentation fault.
0x55701ef0 in
_D3std9container6rbtree__T12RedBlackTreeTAyaVQea5_61203c2062Vbi0ZQBn5emptyMFNaNbNdNiNfZb (this=0x0)
at
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/container/rbtree.d:967
967 return _end.left is null;
(gdb) bt
#0 0x55701ef0 in
_D3std9container6rbtree__T12RedBlackTreeTAyaVQea5_61203c2062Vbi0ZQBn5emptyMFNaNbNdNiNfZb (this=0x0)
at
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/container/rbtree.d:967
#1 0x55701a11 in
qtrac.debfind.model.Model.readPackageFile(immutable(char)[])
(this=0x0, filename=...) at model.d:66
warning: (Internal error: pc 0x5591b874 in read in psymtab,
but not in symtab.)
warning: (Internal error: pc 0x5591b820 in read in psymtab,
but not in symtab.)
warning: (Internal error: pc 0x5591b874 in read in psymtab,
but not in symtab.)
#2 0x55701671 in
qtrac.debfind.model.Model.initialize(int) (this=0x0,
maxDebNamesForWord=100) at model.d:31
warning: (Internal error: pc 0x5591b874 in read in psymtab,
but not in symtab.)
#3 0x557045b0 in
_D5qtrac7debfind9appwindow9AppWindow6__ctorMFC3gtk11ApplicationQnZCQCnQCkQCfQBy (this=0x77ece630, application=0x77ed1360) at appwindow.d:31
#4 0x55722819 in
_D5qtrac7debfind3app4mainFAAyaZ__T12__dgliteral2TC3gio11ApplicationQnZQBkMFQBaZv (GioApplication=0x77ed1360) at app.d:16
#5 0x5575be54 in
_D7gobject8DClosureQj__T17d_closure_marshalTDFC3gio11ApplicationQnZvZQBtUPSQCv1c5types8GClosurePSQDrQwQw6GValuekQrPvQcZv (closure=0x55c37690, return_value=0x0, n_param_values=1, param_values=0x77ef46e0, invocation_hint=0x7fffd710 "\b", marshal_data=0x0) at DClosure.d:122
#6 0x7419410d in g_closure_invoke () at
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#7 0x741a705e in () at
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#8 0x741af715 in g_signal_emit_valist () at
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#9 0x741b012f in g_signal_emit () at
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
warning: (Internal error: pc 0x5591b874 in read in psymtab,
but not in symtab.)
#10 0x7fffee148b95 in () at
/usr/lib/x86_64-linux-gnu/libgio-2.0.so.0
warning: (Internal error: pc 0x5591b874 in read in psymtab,
but not in symtab.)
#11 0x7fffee148da6 in g_application_run () at
/usr/lib/x86_64-linux-gnu/libgio-2.0.so.0
warning: (Internal error: pc 0x5591b874 in read in psymtab,
but not in symtab.)
#12 0x5591b875 in warning: (Internal error: pc
0x5591b874 in read in psymtab, but not in symtab.)
_D3gio11ApplicationQn3runMFAAyaZiwarning: (Internal error: pc
0x5591b874 in read in psymtab, but not in symtab.)
warning: (Internal error: pc 0x5591b874 in read in psymtab,
but not in symtab.)
(warning: (Internal error: pc 0x5591b874 in read in psymtab,
but not in symtab.)
this=0x77ed1360, warning: (Internal error: pc 0x5591b874
in read in psymtab, but not in symtab.)
argv=...)warning: (Internal error: pc 0x5591b874 in read in
psymtab, but not in symtab.)
at Application.dwarning: (Internal error: pc 0x5591b874 in
read in psymtab, but not in symtab.)
warning: (Internal error: pc 0x5591b874 in read in psymtab,
but not in symtab.)
:931
#13 0x55722760 in D main (args=...) at app.d:18
And thanks for helping!
Re: use of struct vs class
change #1:
if (line.empty) {
if (deb != null && deb.valid)
debs.insert(deb);
else // report incomplete
deb = null;
continue;
}
if (deb == null)
deb = new Deb;
change #2: gets rid of most errors:
bool opEquals(const Deb* other) const @safe pure nothrow {
int opCmp(ref const Deb* other) const {
Just changed to pointers & moved to a separate file.
So now I just have one error in line 12 of model.d:
RedBlackTree!Deb* debs; // name-ordered list of deb packages LINE
12
Performing "debug" build using
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/ldc2 for x86_64.
gtk-d:gtkd 3.9.0: target for configuration "library" is up to
date.
debfind ~master: building configuration "application"...
src/model.d(12,9): Error: template instance
std.container.rbtree.RedBlackTree!(Deb) does not match template
declaration RedBlackTree(T, alias less = "a < b", bool
allowDuplicates = false)
with T = Deb
must satisfy the following constraint:
is(typeof(binaryFun!less(T.init, T.init)))
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/ldc2 failed with exit
code 1.
Re: use of struct vs class
Instead of deb.clear I'm now doing deb = null;
Re: use of struct vs class
On Saturday, 7 March 2020 at 10:30:06 UTC, drug wrote:
07.03.2020 13:20, mark пишет:
I have this struct (with details omitted
[ snip ]
Should Deb be a class rather than a struct?
Do you consider using pointers in AA:
```
Deb*[string] debForName;
```
I've done some changes including using Deb* as you suggested:
struct Deb {
string name;
...
RedBlackTree!string tags;
bool valid() { return !(name.empty || description.empty); }
size_t toHash() const @safe nothrow {
return typeid(name).getHash(); // names are unique
}
bool opEquals(const Deb other) const @safe pure nothrow {
return name == other.name; // names are unique
}
int opCmp(ref const Deb other) const {
return cmp(name, other.name); // names are unique
}
}
Which I now want to store in:
RedBlackTree!Deb* debs; // name-ordered list of deb packages
And now I'm populating like this:
Deb* deb;
auto file = File(filename);
foreach(line; file.byLine) {
line = strip(line);
if (line.empty) {
if (deb != null && deb.valid) {
debs.insert(deb);
deb.clear;
}
// else report incomplete package
continue;
}
if (deb == null)
deb = new Deb;
...
}
if (deb != null && deb.valid)
debs.insert(deb);
But it crashes with:
Performing "debug" build using
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/ldc2 for x86_64.
gtk-d:gtkd 3.9.0: target for configuration "library" is up to
date.
debfind ~master: building configuration "application"...
src/model.d(96,36): Error: template
std.container.rbtree.RedBlackTree!(Deb, "a < b",
false).RedBlackTree.stableInsert cannot deduce function from
argument types !()(Deb*), candidates are:
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/../import/std/container/rbtree.d(1256,12):
stableInsert(Stuff)(Stuff stuff)
with Stuff = Deb*
must satisfy the following constraint:
isImplicitlyConvertible!(Stuff, Elem)
...
use of struct vs class
I have this struct (with details omitted ... for brevity):
struct Deb {
string name;
...
RedBlackTree!string tags;
void clear() { name = ""; ...; tags.clear; }
bool valid() { return !(name.empty || description.empty); }
}
I plan to store >65K of these (with potential for growth to
>250K) in an AA:
Deb[string] debForName;
I plan to populate debForName by reading data files (actually
Debian Packages files) like this:
Deb deb;
auto file = File(filename):
foreach(line; file.byLine) {
if (line.empty) {
if (deb.valid) // end of package
debForName[deb.name] = deb; // XXX
// else report incomplete package
deb.clear;
continue;
}
... // populate the deb
}
if (deb.valid)
debForName[deb.name] = deb;
I'm assuming that line XXX will copy the Deb including the tree
(which as usual I'm using as a set -- I really miss a set class
in D!). Will this work (I'll find out myself next week when I get
further, but D experts can likely tell from the above).
Should Deb be a class rather than a struct?
Re: converting to/from char[]/string
On Thursday, 5 March 2020 at 13:31:14 UTC, Adam D. Ruppe wrote: On Thursday, 5 March 2020 at 11:03:30 UTC, mark wrote: I want to use the Porter stemming algorithm. There's a D implementation here: https://tartarus.org/martin/PorterStemmer/d.txt I think I (or ketmar and I stole it from him) ported that very same file before: https://github.com/adamdruppe/adrdox/blob/master/stemmer.d By just adding `const` where appropriate it becomes compatible with string and you can slice to take care of the size thing. https://github.com/adamdruppe/adrdox/blob/master/stemmer.d#L512 is that stem function as a const slice I thought the problem was using char[] rather than dchar[], but evidently not. I downloaded yours and it "just works": I didn't have to change anything. (dscanner gives a couple of const/immutable hints which I'll fix, but still.) Might be good to ask to add yours to https://tartarus.org/martin/PorterStemmer/ since it works and the old one doesn't. Thank you!
Re: converting to/from char[]/string
I suspect the problem is using .length rather than some other size property.
Re: converting to/from char[]/string
I changed int to size_t and used const(char[]) etc. as suggested. It ran but crashed. Each crash was a range violation, so for each one I put in a guard so instead of if ( ... m_b[m_k]) I used if (m_k < m_b.length && ... m_b[m_k) I did this kind of fix in three places. The result is that it does some but not all the stemming! Anyway, I'll compare it with the Python version and see if I can spot the problem(s). Thanks.
Re: converting to/from char[]/string
On Thursday, 5 March 2020 at 11:12:24 UTC, drug wrote: On 3/5/20 2:03 PM, mark wrote: [snip] Your code and errors seem to be not related. OK, it is probably that the D stemmer is 19 years old! I've now got Martin Porter's own Java version, so I'll have a go at porting that to D myself.
converting to/from char[]/string
I want to use the Porter stemming algorithm.
There's a D implementation here:
https://tartarus.org/martin/PorterStemmer/d.txt
The main public function's signature is:
char[] stem(char[] p, int i, int j)
But I work entirely in terms of strings (containing individual
words), so I want to add another function with this signature:
string stem(string word)
I've tried this without success:
public string stem(string word) {
import std.conv: to;
char[] chars = word.to!char[];
int end = chars.length.to!int;
return stem(chars, 0, end).to!string;
}
Here are just a few of the errors:
src/porterstemmer.d(197,13): Error: cannot implicitly convert
expression s.length of type ulong to int
src/porterstemmer.d(222,9): Error: cannot implicitly convert
expression cast(ulong)this.m_j + s.length of type ulong to int
src/porterstemmer.d(259,12): Error: function
porterstemmer.PorterStemmer.ends(char[] s) is not callable using
argument types (string)
src/porterstemmer.d(259,12):cannot pass argument "sses"
of type string to parameter char[] s
Re: What does assigning void mean?
On Thursday, 5 March 2020 at 08:35:52 UTC, drug wrote: On 3/5/20 10:47 AM, mark wrote: In Adam Ruppe's D Cookbook there're these lines in a ref counting example: RefCountedObject o = void; // What does this mean/do? o.data = new Implementation(); o.data.refcount = 1; I don't understand the first line; could someone explain please? In D all vars are initialized by default. If you use assigning void then the var won't be initialized. Thanks, I had read it (in "Learning D" I think), but had already forgotten.
What does assigning void mean?
In Adam Ruppe's D Cookbook there're these lines in a ref counting example: RefCountedObject o = void; // What does this mean/do? o.data = new Implementation(); o.data.refcount = 1; I don't understand the first line; could someone explain please?
Where are the GSOC 2020 ideas?
On https://wiki.dlang.org I can find GSOC ideas 2011-2019, but not 2020. I know the 2020 one's haven't been accepted, but I'd like to know what they are in case I feel like having a go at one as part of learning D.
SQLite 3 support?
There seems to be some support for SQLite 3 in std. lib. etc when looking at the stable docs: https://dlang.org/phobos/etc_c_sqlite3.html But this isn't visible when looking at stable (ddox). Is this the best SQLite 3 library to use or is a third-party library best? For example https://github.com/biozic/d2sqlite3
A small D/GtkD example game
I've just completed a small D/GtkD game. It might be useful for others trying to learn GtkD since it is only just over 1000 lines, yet shows how to create a dialog-style app with a modal dialog and a modeless dialog, and a custom drawn widget, as well as keyboard and mouse handling. The source (and a 64-bit Windows binary) is here: https://github.com/mark-summerfield/gravitate-d I'm posting to the Learn forum because I'm just a D beginner, so felt that using the announce list would be a bit presumptuous.
Re: dscanner and ref parameters
On Sunday, 23 February 2020 at 09:35:30 UTC, Jacob Carlborg wrote: On 2020-02-23 10:03, mark wrote: Then this would not only help dscanner, but also make it clear to programmers that the argument could be modified. It's not necessary for dscanner. It should look at the signature of `getKeyval` to see that it takes an argument by `ref`. Just realised that the arg is 'out' not 'ref'; don't know if that makes a difference to dscanner. Anyway, I've made a bug report: https://github.com/dlang-community/D-Scanner/issues/793
dscanner and ref parameters
I use dscanner to lint my code and find it helpful. However, in GtkD there are several functions which take ref args, and these confuse dscanner. For example: uint kv; event.getKeyval(kv); // ref arg is updated here dscanner incorrectly (but understandably) reports: Variable kv is never modified and could have been declared const or immutable. If D allowed the _optional_ use of ref at the call site: uint kv; event.getKeyval(ref kv); Then this would not only help dscanner, but also make it clear to programmers that the argument could be modified. (This is done in Rust with f( arg), and I certainly find it helpful.)
Re: GtkD crash
Thanks for your question, it led me to focus on the Label and now I've solved the problem. I thought that onChangeState was never called before the Label was constructed, but it turns out it is called before. So now I use: if (statusLabel !is null) statusLabel.setText(message); Now it works. Thanks!
GtkD crash
I'm porting a simple game to GtkD to learn the library and more
about D.
Unfortunately, I've hit a show-stopping crash.
I have a subclass of ApplicationWindow which has this method:
private void onChangeState(int score, Board.State state) {
import std.format: format;
string message;
if (state == Board.State.GAME_OVER)
message = format("%,d Game Over", score);
else if (state == Board.State.USER_WON) {
if (score > highScore) {
message = format("%,d New High Score!", score);
highScore = score; // TODO save highScore
} else
message = format("%,d You Won!", score);
} else // still playing
message = format("%,d/%,d", score, highScore);
statusLabel.setText(message); // BUG
}
This method gets passed to another widget which calls it whenever
the game's state or score changes.
If the BUG line is commented out, the program runs fine (well,
except that the Label always shows "0/0").
But if the BUG line is uncommented (as above), it crashes:
$ dub
Performing "debug" build using
/home/mark/opt/ldc2-1.20.0-linux-x86_64/bin/ldc2 for x86_64.
gtk-d:gtkd 3.9.0: target for configuration "library" is up to
date.
gravitate ~master: target for configuration "application" is up
to date.
To force a rebuild of up-to-date targets, run again with --force.
Running ./gravitate
onChangeState 0 PLAYING
Program exited with code -11
Here's what it looks like in gdb (slightly edited):
$ gdb gravitate
GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
[snip]
This GDB was configured as "x86_64-linux-gnu".
[snip]
Reading symbols from gravitate...done.
(gdb) run
Starting program: /home/mark/app/gravitate/d/gravitate
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x76e02700 (LWP 7017)]
[New Thread 0x76601700 (LWP 7018)]
[New Thread 0x75e00700 (LWP 7019)]
[New Thread 0x755ff700 (LWP 7020)]
[New Thread 0x74dfe700 (LWP 7021)]
[New Thread 0x7fffd700 (LWP 7022)]
[New Thread 0x7fffdf7fe700 (LWP 7023)]
[New Thread 0x7fffce3a5700 (LWP 7024)]
[New Thread 0x7fffcdba4700 (LWP 7025)]
onChangeState 0 PLAYING
Thread 1 "gravitate" received signal SIGSEGV, Segmentation fault.
0x55704cc1 in gamewindow.GameWindow.onChangeState(int,
board.Board.State) (this=0x77ecf700, score=0,
state=board.Board.PLAYING) at gamewindow.d:182
182 statusLabel.setText(message);
(gdb) bt
warning: (Internal error: pc 0x55703fbf in read in psymtab,
but not in symtab.)
warning: (Internal error: pc 0x55703fbf in read in psymtab,
but not in symtab.)
[ + several more like these ]
#0 0x55704cc1 in
gamewindow.GameWindow.onChangeState(int, board.Board.State)
(this=0x77ecf700, score=0, state=board.Board.PLAYING) at
gamewindow.d:182
warning: (Internal error: pc 0x55703fbf in read in psymtab,
but not in symtab.)
#1 0x55728246 in board.Board.newGame()
(this=0x7fffcc92d000) at board.d:60
warning: (Internal error: pc 0x55703fbf in read in psymtab,
but not in symtab.)
warning: (Internal error: pc 0x55703a2b in read in psymtab,
but not in symtab.)
#2 0x55704bb8 in
_D5board5Board6__ctorMFDFiEQzQv5StateZvZCQBnQBk (warning:
(Internal error: pc 0x55703fbf in read in psymtab, but not in
symtab.)
this=0x7fffcc92d000, warning: (Internal error: pc 0x55703fbf
in read in psymtab, but not in symtab.)
onChangeState=...)
at board.d:42
warning: (Internal error: pc 0x55703fbf in read in psymtab,
but not in symtab.)
warning: (Internal error: pc 0x55703a2b in read in psymtab,
but not in symtab.)
#3 0x55703fc0 in warning: (Internal error: pc
0x55703fbf in read in psymtab, but not in symtab.)
gamewindow.GameWindow.makeWidgets()warning: (Internal error: pc
0x55703fbf in read in psymtab, but not in symtab.)
warning: (Internal error: pc 0x55703fbf in read in psymtab,
but not in symtab.)
(warning: (Internal error: pc 0x55703fbf in read in psymtab,
but not in symtab.)
warning: (Internal error: pc 0x55703a2b in read in psymtab,
but not in symtab.)
this=0x77ecf700)warning: (Internal error: pc 0x55703fbf
in read in psymtab, but not in symtab.)
at gamewindow.dwarning: (Internal error: pc 0x55703fbf in
read in psymtab, but not in symtab.)
warning: (Internal error: pc 0x55703fbf in read in psymtab,
but not in symtab.)
:58
warning: (Internal error: pc 0x55703a2b in read in psymtab,
but not in symtab.)
#4 0x55703a2c in warning: (Internal error: pc
0x55703a2b in read in psymtab, but not in symtab.)
_D10gamewindow10GameWindow6__ctorMFC3gtk11ApplicationQnZCQCdQBuwarning:
(Internal error: pc 0x55703a2b in read in psymtab, but not in symtab.)
warning: (Internal error: pc 0x55703a2b in read in psymtab,
but not in symtab.)
(warning: (Internal error: pc
Re: Dscanner: is it possible to switch off style checks case-by-case?
On Saturday, 15 February 2020 at 07:23:02 UTC, Basile B. wrote: On Thursday, 13 February 2020 at 17:15:50 UTC, mark wrote: I'm starting out with GtkD and have this function: [snip] Otherwise here is an example of how you can tune the different checks: https://raw.githubusercontent.com/dlang/phobos/master/.dscanner.ini See also the last section of https://raw.githubusercontent.com/dlang-community/D-Scanner/master/README.md Thank you, I made the code changes and am reading the docs you linked to.
Dscanner: is it possible to switch off style checks case-by-case?
I'm starting out with GtkD and have this function:
void main(string[] args) {
Main.init(args);
auto game = new GameWindow();
Main.run();
}
and this method:
void quit(Widget widget) {
Main.quit();
}
When I run dscanner --styleCheck it reports:
./src/app.d(10:10)[warn]: Variable game is never used.
./src/app.d(22:22)[warn]: Parameter widget is never used.
These are correct. However, is it possible to switch them off
individually?
(In Python you can switch off lint checks using a special text in
a comment at the end of the line.)
Re: Some impressions/notes from a new D programmer
On Wednesday, 12 February 2020 at 18:20:47 UTC, Adam D. Ruppe wrote: On Wednesday, 12 February 2020 at 15:52:35 UTC, mark wrote: Yours rolls the two examples into one and doesn't show the Standards or Usage sections. Weird, that's a legit bug in there. I'll fix them. I also think you split into more HTML files which I prefer. OTOH yours doesn't have the search box. Given how new I am to D, I really need to be able to search. The search is in the upper right unless you resize the window, then it disappears. lol another bug, how did I not notice that before? well I'll fix those in a little bit. Please mention when you've fixed the search on this list since then I can switch to using your version of the docs. In my browser I can only see your search box if I expand the window to full screen which is wider than I need (I have a 1920x1200 monitor, so only have the browser window 1200x1100.)
Re: Some impressions/notes from a new D programmer
On Wednesday, 12 February 2020 at 14:15:40 UTC, Adam D. Ruppe wrote: On Wednesday, 12 February 2020 at 10:39:06 UTC, mark wrote: Library Reference Documentation Have you seen my fork? http://dpldocs.info/experimental-docs/std.zip.ZipArchive.html Yours is *much* clearer. However, if you compare: http://dpldocs.info/experimental-docs/std.zip.html vs https://dlang.org/phobos/std_zip.html Yours rolls the two examples into one and doesn't show the Standards or Usage sections. But the official page doesn't have the Bugs section. I also think you split into more HTML files which I prefer. OTOH yours doesn't have the search box. Given how new I am to D, I really need to be able to search. for example The documentation doesn't seem that easy to use I generated docs for this too http://gtk-d.dpldocs.info/gtk.AboutDialog.AboutDialog.html though since it is generated from C source ultimately the samples there are still C! But you can navigate members somewhat well. I hadn't seen this and it does looks easier to navigate. I've bookmarked it. Thanks.
Re: Some impressions/notes from a new D programmer
On Wednesday, 12 February 2020 at 11:46:02 UTC, Dennis wrote: Thanks for your perspective. Just a few things are unclear to me: On Wednesday, 12 February 2020 at 10:39:06 UTC, mark wrote: I don't find the presentation of the member properties and methods very easy to read Can you elaborate a bit on this? Maybe I'm just used to the Python docs, but I find them a lot easier to read. The lack of set and B-tree types is disappointing (esp. considering that the much younger Rust has them). I'm using rbtree for sets but that imposes a requirement that my items support < (rather than the == or hash I'd expect for a set). This confuses me. So there is std.container.rbtree, but you don't like that the element type needs to have an order defined? How can Rust do binary search in a tree that has no order? If you are looking for a hashset, you can use an associative array for that. Naturally a tree needs <. But I want a set and since D doesn't have one I can either use an AA or an rbtree and I was advised that an rbtree is better for this purpose. However, dub doesn't seem to be competitive with Rust's cargo. Getting fast statically built (no dependency) executables is really nice. I've heard good things about cargo, but haven't used it myself yet. Do you have a specific thing dub can improve the most on? Some cargo packages are applications. If I do 'cargo install someapp' it will be installed in $HOME/.cargo/bin. So by simply adding that to my PATH, I can easily use all installed rust apps. But dub doesn't appear to have an equivalent of this.
Some impressions/notes from a new D programmer
I've been learning D for a few weeks now. I'm an experienced programmer in other languages (esp. Python, but also Rust and C++). Here're some *early* impressions and notes. D Tour I found the D Tour, esp. "D's Basics" to be very helpful. Each part is short and in most cases understandable. Being able to run and edit the code is a real help for learning. D Playground The D playground https://run.dlang.io/ is very useful for trying out snippets and generally learning, so I use it a lot. (I still haven't worked out how to save a URL to my code though.) Library Reference Documentation The Library Reference documentation seems to be a mixed bag. Often I've found a good overview at the start, but then few or no examples in the docs for classes and methods (see e.g., https://dlang.org/phobos/std_zip.html#.ZipArchive). I don't find the presentation of the member properties and methods very easy to read, but the worst aspect is the lack of examples. Standard Library The library itself "feels" a bit incomplete, which is surprising given how long D's been around. To give just two examples: The lack of set and B-tree types is disappointing (esp. considering that the much younger Rust has them). I'm using rbtree for sets but that imposes a requirement that my items support < (rather than the == or hash I'd expect for a set). The fact that the return value of std.file.getAttributes() means completely different things on POSIX and Windows. That's fair enough, but there ought to be a platform-neutral equivalent for those writing cross-platform applications that returned, say, a struct or tuple with the common subset of attributes normalised. (And if there is such a function, why isn't it cross-referenced.) There seems to be a curious mixture of functions which are POSIX- or Windows-specific and those which are platform neutral. The D Language The D language seems to be a "kitchen sink" (i.e., has everything) like C++, Rust, (and nowadays, Python). This makes it big and a *lot* to learn. However, I managed to create a little library that used template types (with some help from this forum), and I _understand_ the templates. This is a huge improvement over C++ or Rust. And to my surprise, so far my D programs have about the same line counts as the Python versions. Also, I've found building much easier than C++. However, dub doesn't seem to be competitive with Rust's cargo. Getting fast statically built (no dependency) executables is really nice. GUI Programming I've tried a number of D GUI libraries, and all bar one have been problematic. To my surprise GtkD was easy to install on both Linux and Windows and getting "hello world" to build and run was fairly easy. The documentation doesn't seem that easy to use, but I'll start with Ron Tarrant's https://gtkdcoding.com/ and see how I get on from there. D Books I find Ali Çehreli's book (http://ddili.org/ders/d.en/index.html) more suited to complete beginners, but I am skim reading it and finding it useful here and there. The main books I'm reading are Mike Parker's Learning D and Adam Ruppe's D Cookbook, both of which I think are pretty good. (However, I hope both will produce more up-to-date and improved second editions with a better publisher.) Learn D Forum People on this forum have always provided polite and helpful answers. This is a very important intangible benefit of the language. Conclusion My hope was that D would offer a sweet spot between Python's ease and speed of development and Rust's performance. And so far this looks like being the case.
A D implementation of the Python difflib module's sequence matcher.
I've just completed my first D package: http://code.dlang.org/packages/ddiff It is a straight port, so it isn't at all functional-style. I'd be happy and interested if anyone could show me how to replace some/all of the for[each] loops (without reducing performance), or for any other code improvements that would make it more idiomatic D.
Re: Is there a std.zip.ZipArchive isDir or isFile method?
On Wednesday, 12 February 2020 at 05:59:53 UTC, cc wrote:
On Monday, 3 February 2020 at 13:26:38 UTC, mark wrote:
I'm using std.zip.ZipArchive to read zip files, e.g.:
[snip]
I couldn't find one either, I had to do this:
version(Windows) {
enum uint FILE_ATTRIBUTE_DIRECTORY = 0x10;
}
auto zip = new ZipArchive(buffer);
foreach (fn, am; zip.directory) {
if (am.fileAttributes & FILE_ATTRIBUTE_DIRECTORY)
... is directory
else
... is file
}
I need to work on both Linux and Windows, and on Linux
am.fileAttributes seems to be 0 for both files and directories.
The lack of a cross-platform way of distinguishing whether an
archive member is a directory or file does seem to be a missing
piece of the API.
As I'm looking at my code for this I'm also reminded that
different zip files can internally store path separators as
either \ or / depending on the platform that created them so
you may need to be careful about that too. I have a bit for
this that simply does:
version(StandardizePathSeparators) {
string filename = fn.replace("\\", "/");
} else {
string filename = fn;
}
Yes, I was aware of that, but I use:
string filename = fn.tr("\\", "/"); // tr is from std.string
Re: GtkD on Windows: notes + question
On Tuesday, 11 February 2020 at 20:49:40 UTC, Ron Tarrant wrote: On Sunday, 9 February 2020 at 13:28:59 UTC, mark wrote: I found a much easier way to get GtkD working on windows than that described in https://gtkdcoding.com/2019/01/11/-introduction-to-gtkDcoding.html Just FYI... I don't use dub because I don't have time to understand its foibles well enough to steer new D coders through the maze. Besides that, using tools like dub separates one from the process. Up to you, though, of course. That's understandable. Anyway, once I get back to learning GtkD I'll be starting with your blog.
Re: GtkD on Windows: notes + question
On Sunday, 9 February 2020 at 14:08:02 UTC, Daniel Kozak wrote: "lflags-windows": ["/SUBSYSTEM:WINDOWS", "/ENTRY:mainCRTStartup"], [snip] Is there a way to avoid the console Window, at least for release builds? Thank you! Your solution I guess is for dub.json. For those using dub.sdl the necessary line is: lflags "/SUBSYSTEM:WINDOWS" "/ENTRY:mainCRTStartup" platform="windows"
Re: GtkD on Windows: notes + question
On Sunday, 9 February 2020 at 14:08:02 UTC, Daniel Kozak wrote: "lflags-windows": ["/SUBSYSTEM:WINDOWS", "/ENTRY:mainCRTStartup"], [snip] Is there a way to avoid the console Window, at least for release builds? Thank you! Your solution I guess is for dub.json. For those using dub.sdl the necessary line is: lflags "/SUBSYSTEM:WINDOWS" "/ENTRY:mainCRTStartup" platform="windows"
GtkD on Windows: notes + question
I found a much easier way to get GtkD working on windows than that described in https://gtkdcoding.com/2019/01/11/-introduction-to-gtkDcoding.html 1. I downloaded and installed the Gtk3 runtime (the link is on https://gtkdcoding.com/2019/01/11/-introduction-to-gtkDcoding.html) 2. I downloaded and unzipped the GtkD3 zip to C:\bin\GtkD3 3. Since I'd already installed LDC I just had to run: dub add-path C:\bin\GtkD3 Now I'm able to build and run on windows using dub. And again, I get static builds so have deployable .exes. However, when I double-click a GtkD .exe it pops up a console window, then the GUI window, and the console window stays until I close the GUI. Is there a way to avoid the console Window, at least for release builds?
Re: How to compile with GtkD [solved]
Turns out I didn't need to add lines to dub.sdl at all. So my dub.sdl is now just: name "gtktest" description "Gtk Test" authors "Mark" targetType "executable" dependency "gtk-d:gtkd" version=">=3.9.0" The solution was to do this: $ dub add-path ~/opt/GtkD3/ After that it does static builds, which is great, gives me self-contained executables.
Re: How to compile with GtkD
On Sunday, 9 February 2020 at 12:03:23 UTC, Ron Tarrant wrote: On Sunday, 9 February 2020 at 11:52:19 UTC, mark wrote: right now I want to start on Linux and I'm stuck. Maybe this will help... https://gtkdcoding.com/2019/03/31/x0002-gtkd-in-a-linux-environment.html Unfortunately it didn't because I have locally installed LDC and GtkD. Generally, for development I prefer to install (and on Linux build) languages and libs myself rather than using system versions. What I really need is to know how to change my dub.sdl to send the right arguments to ldc2.
How to compile with GtkD
According to the GtkD web site https://gtkd.org/ this GUI library
is cross-platform, so hopefully just what I need.
The introductory blog
https://gtkdcoding.com/2019/01/11/-introduction-to-gtkDcoding.html explains how to install and get started with GtkD on Windows (which I plan to use later on), but right now I want to start on Linux and I'm stuck.
I have LDC in $HOME/opt/ldc2-1.19.0-linux-x86_64 with the bin/
dir in my path, and GtkD 3.9.0 in $HOME/opt/GtkD3
I am trying to build a simple "hello world" app (from the
gtkDcoding blog) in $HOME/app/d/gtktest (which is a dub init
created directory) and have this dub.sdl:
name "gtktest"
description "Gtk Test"
authors "Mark"
targetType "executable"
dependency "gtk-d:gtkd" version=">=3.9.0"
importPaths "$HOME/opt/GtkD3"
libs "libgtkd-3"
The error I get is:
$ dub
Performing "debug" build using /home/mark/opt/bin/ldc2 for x86_64.
gtk-d:gtkd 3.9.0: target for configuration "library" is up to
date.
gtktest ~master: building configuration "application"...
Linking...
/usr/bin/ld.gold: error: cannot find -llibgtkd-3
collect2: error: ld returned 1 exit status
Error: /usr/bin/cc failed with status: 1
/home/mark/opt/bin/ldc2 failed with exit code 1.
$ cd ~/opt/GtkD3
$ ls *.a
libgstreamerd-3.a
libgtkd-3.a
libgtkdsv-3.a
libpeasd-3.a
libvted-3.a
I also tried specifying libs as "gtkd-3" and had the same problem.
I can run $HOME/opt/GtkD3/TestWindow and it works fine. And I can
also build it in $HOME/opt/GtkD3/demos/gtkD/TestWindow. However,
it uses a dub.json with "dependencies": { "gtk-d:gtkd": {"path":
"../../../" }, } and I don't know what the dub.sdl equivalent of
this is. (I prefer .sdl to .json.)
Re: How to use sets in D?
On Friday, 7 February 2020 at 22:03:00 UTC, H. S. Teoh wrote: On Fri, Feb 07, 2020 at 07:37:08PM +, mark via Digitalmars-d-learn wrote: [snip] bool[E] works just fine. [snip] Or you can wrap void[0][E] in a nice user-defined type that gives nice set-like syntax. But IMO, this is all overkill, and adds needless complexity. Just use bool[E] or std.container.rbtree. :-D I didn't think bool[E] would be a win because although it is only one byte per item, it won't align so wouldn't it end up taking 4 bytes of space anyway. The void[0][E] you showed is good, but, I'll try using the rbtree since I'd rather use an out-of-the-box collection. Thanks for the replies.
How to use sets in D?
I am porting code from other languages to D as part of learning D, and I find I've used sets quite a lot. AFAIK D doesn't have a built-in set type or one in the std. lib. However, I've been perfectly successfully using int[E] where E is my ElementType, and adding with set[element] = 0. I mostly only need add, remove, iteration, and in, with uniqueness what I care most about. I know I could use bool[E] and set[element] = false, or I suppose container.rbtree. Would either of these--or something else built-in or in the std. lib.--be better?
Re: Does D have an equvalent of: if (auto = expr; expr)
Thanks for the excellent replies.
Does D have an equvalent of: if (auto = expr; expr)
Some languages support this kind of thing: if ((var x = expression) > 50) print(x, " is > 50") Is there anything similar in D?
Re: Trying to use a template class with ranges
On Thursday, 6 February 2020 at 16:29:57 UTC, Steven
Schveighoffer wrote:
On 2/6/20 11:05 AM, mark wrote:
src/package.d(50,35): Error: no property opCall for type
diffrange.Diff!(dchar[]), did you mean new Diff!(dchar[])?
Hah, forgot that it's a class. Yes, I DID mean new Diff ;)
-Steve
Wow, that's all it needed to compile!
And I've added the extra check you suggested:
auto differ(T)(T a, T b) if (
isForwardRange!T && // T is a range
is(typeof(T.init.front == T.init.front)) // Elements
support ==
) {
return new Diff!T(a, b);
}
Thanks!
Re: Trying to use a template class with ranges
On Thursday, 6 February 2020 at 15:21:46 UTC, Steven
Schveighoffer wrote:
[snip]
3. You should declare constraints signifying what types are
valid. i.e.:
class Diff(T) if (
isForwardRange!T // it's a forward range
&& is(typeof(T.init.front == T.init.front)) // elements are
comparable
)
Might be good to put this constraint on the factory function
too.
I don't know how to do that syntactically. I tried the obvious
and it wouldn't compile.
But even without that it won't compile although it is much closer
now!
import std.range: ElementType, front, isForwardRange;
class Diff(T) if (
isForwardRange!T && // T is a range
is(typeof(T.init.front == T.init.front)) // Elements
support ==
) {
alias E = ElementType!T;
T a;
T b;
size_t[][E] b2j;
this(T a, T b) {
this.a = a;
this.b = b;
chainB();
}
private final void chainB() {
foreach (i, element; b)
b2j[element] ~= i;
// TODO
}
}
auto differ(T)(T a, T b) { return Diff!T(a, b); }
unittest {
import std.array;
import std.stdio: writeln;
writeln("unittest for the diffrange library.");
auto d1 = differ("one two three four".array, "one too tree
four".array);
auto a = ["Tulips are yellow,", "Violets are blue,", "Agar is
sweet,",
"As are you."];
auto b = ["Roses are red,", "Violets are blue,", "Sugar is
sweet,",
"And so are you."];
auto d2 = differ(a, b);
}
Here's the error output:
Excluding package.d file from test due to
https://issues.dlang.org/show_bug.cgi?id=11847
src/package.d(50,35): Error: no property opCall for type
diffrange.Diff!(dchar[]), did you mean new Diff!(dchar[])?
src/package.d(57,21): Error: template instance
diffrange.differ!(dchar[]) error instantiating
src/package.d(50,35): Error: no property opCall for type
diffrange.Diff!(string[]), did you mean new Diff!(string[])?
src/package.d(62,21): Error: template instance
diffrange.differ!(string[]) error instantiating
/home/mark/opt/bin/ldc2 failed with exit code 1.
Curiously the "Excluding package.d file" message isn't true.
Not that I mind, I just want to be able to get it going.
Thanks!
PS The line numbers don't match because I've deleted some structs
& enums that are above the class but which aren't used yet.
Re: Trying to use a template class with ranges
I forgot to mention: I want the class to work with: Diff(aForwardRange, bForwardRange) where T = ForwardRange, E = anything that supports == A common use case is for two sequences of strings (i.e., lines read from two files). Diff(aString, bString) where T = char[] or wchar[] or dchar[] E = char or wchar or dchar
Trying to use a template class with ranges
I am starting on porting Python's difflib's sequence matcher to D.
I want to have a class that will accept two ranges whose elements
are of the same type and whose elements can be compared for
equality.
How do I make a class declaration that specifies a (forward)
range type and an equality-supporting element type?
Here's what doesn't work:
class Diff(T, E) {
T a; // T should be a forward range of E elements
T b; // E elements must support == and !=
// This is a hash key=E element, value=slice of size_t
size_t[][E] b2j;
this(T a, T b) {
this.a = a;
this.b = b;
chainB();
}
void chainB() {
foreach (i, element; b)
b2j[element] ~= i;
// TODO
}
}
unittest {
import std.stdio: writeln;
writeln("unittest for the diffrange library.");
auto a = ["Tulips are yellow,", "Violets are blue,",
"Agar is sweet,", "As are you."];
auto b = ["Roses are red,", "Violets are blue,",
"Sugar is sweet,", "And so are you."];
auto diff = Diff(a, b);
}
Re: dub for lib & app?
On Tuesday, 4 February 2020 at 09:21:17 UTC, Andre Pany wrote: On Tuesday, 4 February 2020 at 09:13:48 UTC, mark wrote: Is it possible to create a dub project that has one library and one or more executables (that use the library)? If so, could someone point me to the docs for this since I couldn't find this in the dub docs? [snip] Yes, using sub packages. Set the targetType of the root package to None and introduce 2 sub packages of targetType Executable and Library. Ah, thank you. And if I'm doing *separate* individual projects, for an app, dub creates source/app.d (which I rename src/app.d). But what should it be for a library? Should it be src/package.d along with any supporting src/one.d src/two.do etc? And is this documented anywhere? (I thought I'd seen dub used with a --template argument, but can't find it; so maybe I'm mixing it up with something else?)
dub for lib & app?
Is it possible to create a dub project that has one library and one or more executables (that use the library)? If so, could someone point me to the docs for this since I couldn't find this in the dub docs? Aside: I'm learning D to give me something approaching the convenience of Python with the fast executable that I'd get with Rust. I love Rust's cargo tool, but find that Rust is very frustrating to do prototyping or experimentation whereas D seems easy for these purposes. But dub does seem a bit basic (or maybe I just haven't found the right docs).
Re: Empty string vs null
Thanks for that thorough and careful explanation. Since I'm trying to learn to write D in good style and want my code to be reliable and maintainable, I've now switched to using "" rather than null.
Re: Empty string vs null
Just found this post by Mark Parker that explains:
https://forum.dlang.org/post/gvveit$10i5$1...@digitalmars.com
// test.d
import std.stdio;
import std.string;
void main()
{
report(null, "null");
report("");
report("x");
}
void report(const string x, const string name=null) {
writeln("\nx = \"", name is null ? x : name, "\"");
writeln("null = ", x is null);
writeln("\"\"= ", x == "");
writeln("empty = ", x.empty);
}
Output:
x = "null"
null = true
""= true
empty = true
x = ""
null = false
""= true
empty = true
x = "x"
null = false
""= false
empty = false
Empty string vs null
I have just discovered that D seems to treat empty and null
strings as the same thing:
// test.d
import std.stdio;
import std.string;
void main()
{
string x = null;
writeln("x = \"", x, "\"");
writeln("null = ", x == null);
writeln("\"\"= ", x == "");
writeln("empty = ", x.empty);
x = "";
writeln("\nx = \"", x, "\"");
writeln("null = ", x == null);
writeln("\"\"= ", x == "");
writeln("empty = ", x.empty);
x = "x";
writeln("\nx = \"", x, "\"");
writeln("null = ", x == null);
writeln("\"\"= ", x == "");
writeln("empty = ", x.empty);
}
Output:
x = ""
null = true
""= true
empty = true
x = ""
null = true
""= true
empty = true
x = "x"
null = false
""= false
empty = false
1. Why is this?
2. Should I prefer null or ""? I was hoping to return null to
indicate "no string that match the criteria", and "some string"
otherwise.
Is there a std.zip.ZipArchive isDir or isFile method?
I'm using std.zip.ZipArchive to read zip files, e.g.:
auto zip = new ZipArchive(read(filename));
// ...
foreach (name, member; zip.directory) {
if (name.endsWith('/')) // skip dirs
continue;
mkdirRecurse(dirName(name));
zip.expand(member);
write(name, member.expandedData());
}
As you can see, I am detecting directories with a crude test.
I really wish there was a method for this: and if there is, could
you give me the link 'cos I can't see one in the docs?
(BTW The code above is slightly simplified: the real code won't
unzip if there's an absolute path or .. present and also ensures
that all members are unzipped into a subdir even if the zip has
top-level names.)
Re: How do I fix my failed PRs?
Thanks Petar... I'm in no hurry, just glad that they're in process:-)
Re: How do I fix my failed PRs?
On Sunday, 2 February 2020 at 12:49:31 UTC, MoonlightSentinel wrote: On Sunday, 2 February 2020 at 08:54:02 UTC, mark wrote: However, four have not been accepted, apparently for technical reasons. But I don't understand what's wrong or what I need to do to fix them. (I'm not very knowledgeable about github.) The Travis log suggest that your PRs contain some whitespace errors: The command "grep -nr --include \*.md '\s$' . ; test $? -eq 1" exited with 1 You could try to execute the ggrep command locally to identify lines containing possible errors. PS: Feel free to comment on your PR directly s.t. reviewers/bypassers can see that you could use some assistance. I edit online on github itself so can't check for this kind of thing. Anyway, I've added a comment to each PR so hopefully that will help.
How do I fix my failed PRs?
I've done quite a few small corrections/improvements to the D-tour's English. Almost all have been accepted. However, four have not been accepted, apparently for technical reasons. But I don't understand what's wrong or what I need to do to fix them. (I'm not very knowledgeable about github.) These are the ones that are held up: https://github.com/dlang-tour/english/pull/336 https://github.com/dlang-tour/english/pull/335 https://github.com/dlang-tour/english/pull/328 https://github.com/dlang-tour/english/pull/316
Re: books for learning D
On Wednesday, 29 January 2020 at 11:57:14 UTC, Jan Hönig wrote: On Monday, 13 January 2020 at 16:37:31 UTC, Ron Tarrant wrote: On Monday, 13 January 2020 at 10:28:48 UTC, mark wrote: I'm just starting out learning D. Andrei Alexandrescu's "The D Programming Language" is 10 years old, so is it still worth getting? (I don't know how much D has changed in 10 years.) Actually, Andrei's book has been updated a few times over the years since first being published. The latest version says this on the copyright page: D version: 2.081.1 Book revision: 2018-10-17 So, it's really only about 14 months old. I am also curious. Where can i find the revised book. I'd also like to know where the revised "The D Programming Language" by Andrei Alexandrescu is: and whether it can be bought in physical form?
Re: wordladder - code improvement
I forgot to mention: I know it isn't worth bothering with const/immutable for this tiny example. But I want to learn how to write large D programs, so I need to get into the right habits and know the right things.
Re: wordladder - code improvement
Thanks for your implementation. I can't use the levenshtien distance because although it is a better solution, I want to keep the implementation as compatible with those in the other languages as possible. Your main() is much shorter than mine, but doesn't match the behaviour (and again I want to keep the same as the other implementations). Nonetheless, I didn't know about generate!()(), so I've learnt from yours. Nor did I know about the StopWatch which I'm now using, plus I've added your static assert, and also copied your idea of making update() an inner function (and now only needing one parameter). I still need to study your genLadder() more carefully to understand it because it is so compact.
wordladder - code improvement
Below is a program that produces a wordladder.
The algorithm is probably suboptimal, but I don't care since I've
implemented the same one in Python, Rust, Go, Java, and Nim, so I
find it useful for language comparison purposes.
What I'd like some feedback on is how to improve the code
(keeping the algorithm the same) to make it into more idiomatic D
and to take the most advantage of D's features (while keeping it
as understandable as possible).
For example, in the last function, compatibleWords(), I can't
help thinking that I could avoid the foreach loop.
Also I'm still not really clear on the appropriateness of
const/immutable/in in function arguments. The docs seem to
discourage in, and other things I've read seem to favour const.
Similarly, the appropriateness of const/immutable inside
functions.
// wordladder.d
import core.time: MonoTime;
import std.algorithm: any, count, filter, map, sum, until;
import std.array: array, join;
import std.conv: dtext, to;
import std.functional: not;
import std.range: assocArray, repeat, zip;
import std.random: choice;
import std.stdio: File, write, writeln;
import std.uni: isAlpha, toUpper;
enum WORDFILE = "/usr/share/hunspell/en_GB.dic";
enum WORDSIZE = 4; // Should be even
enum STEPS = WORDSIZE;
alias WordList = string[];
alias WordSet = int[string]; // key = word; value = 0
void main() {
immutable start = MonoTime.currTime;
auto words = getWords(WORDFILE, WORDSIZE);
int count = 1;
WordList ladder;
write("Try ");
while (true) {
write('.');
ladder = generate(words, STEPS);
if (ladder.length != 0)
break;
++count;
}
writeln(' ', count);
writeln(join(ladder, '\n'));
writeln(MonoTime.currTime - start);
}
WordSet getWords(const string filename, const int wordsize) {
return File(filename).byLine
.map!(line => line.until!(not!isAlpha))
.filter!(word => word.count == wordsize)
.map!(word => word.to!string.toUpper)
.assocArray(0.repeat);
}
WordList generate(WordSet allWords, const int steps) {
WordList ladder;
auto words = allWords.dup;
auto compatibles = allWords.dup;
auto prev = update(ladder, words, compatibles);
for (int i = 0; i <= steps; ++i) {
compatibles = compatibleWords(prev, words);
if (compatibles.length == 0)
return [];
prev = update(ladder, words, compatibles);
}
immutable first = dtext(ladder[0]);
immutable last = dtext(ladder[$ - 1]);
if (any!(t => t[0] == t[1])(zip(first, last)))
return []; // Don't accept any vertical letters in common
return ladder;
}
string update(ref WordList ladder, ref WordSet words,
const WordSet compatibles) {
immutable word = compatibles.byKey.array.choice;
ladder ~= word;
words.remove(word);
return word;
}
// Add words that are 1 letter different to prev
WordSet compatibleWords(const string prev, const WordSet words) {
WordSet compatibles;
immutable prevChars = dtext(prev);
immutable size = prevChars.length - 1;
foreach (word; words.byKey)
if (sum(map!(t => int(t[0] == t[1]))
(zip(prevChars, dtext(word == size)
compatibles[word] = 0;
return compatibles;
}
D Cookbook range save question
In the D Cookbook it has as part of the FibonacciRange example:
@property FibonacciRange save() { return this; }
And in the description it says:
"...save, which returns a new range that is a copy of the current
range and can be advanced independently..."
Why is this a *copy*? (For a copy (in C++) I'd have expected
return *this.)
Re: D Cookbook range save question
On Thursday, 30 January 2020 at 10:31:08 UTC, mark wrote:
In the D Cookbook it has as part of the FibonacciRange example:
@property FibonacciRange save() { return this; }
And in the description it says:
"...save, which returns a new range that is a copy of the
current range and can be advanced independently..."
Why is this a *copy*? (For a copy (in C++) I'd have expected
return *this.)
Oh, I understand now...
Sorry for the noise but I don't know how to delete a premature
post!
Re: compiler error when trying to get random key from AA
In the end I used this line since I'm not fussy about the rnd for this: auto word = compatibles.byKey.array.choice; Thank you!
Re: compiler error when trying to get random key from AA
On Saturday, 25 January 2020 at 08:59:23 UTC, Basile B. wrote:
On Saturday, 25 January 2020 at 08:35:18 UTC, mark wrote:
I have this code:
import std.random;
import std.stdio;
void main()
{
auto aa = ["one": 1, "two": 2, "three": 3];
writeln(aa);
auto rnd = rndGen;
auto word = aa.byKey.choice(rnd);
writeln(word);
}
And in the D playground it gives this error:
[snip]
I am treating aa as a set and want to pick a random word from
it.
What am I doing wrong?
I'm sorry I can't give a link to this code in the D playground
but the URL in the web browser is just https://run.dlang.io/
and when I click Shorten to get a URL nothing seems to happen
(using Firefox on Linux).
rndGen is a range.
Use `auto word = aa.byKey.choice(rnd.front())` as index instead.
Then `rndGen.popFront()` to advance.
I tried that. It doesn't solve the problem but does reduce the
size of the error output to:
onlineapp.d(9): Error: template std.random.choice cannot deduce
function from argument types !()(Result, uint), candidates are:
/dlang/dmd/linux/bin64/../../src/phobos/std/random.d(2599):
choice(Range, RandomGen = Random)(auto ref Range range, ref
RandomGen urng)
/dlang/dmd/linux/bin64/../../src/phobos/std/random.d(2609):
choice(Range)(auto ref Range range)
compiler error when trying to get random key from AA
I have this code:
import std.random;
import std.stdio;
void main()
{
auto aa = ["one": 1, "two": 2, "three": 3];
writeln(aa);
auto rnd = rndGen;
auto word = aa.byKey.choice(rnd);
writeln(word);
}
And in the D playground it gives this error:
onlineapp.d(8): Error: template std.random.choice cannot deduce
function from argument types !()(Result,
MersenneTwisterEngine!(uint, 32LU, 624LU, 397LU, 31LU,
2567483615u, 11LU, 4294967295u, 7LU, 2636928640u, 15LU,
4022730752u, 18LU, 1812433253u)), candidates are:
/dlang/dmd/linux/bin64/../../src/phobos/std/random.d(2599):
choice(Range, RandomGen = Random)(auto ref Range range, ref
RandomGen urng)
with Range = Result,
RandomGen = MersenneTwisterEngine!(uint, 32LU, 624LU,
397LU, 31LU, 2567483615u, 11LU, 4294967295u, 7LU, 2636928640u,
15LU, 4022730752u, 18LU, 1812433253u)
must satisfy the following constraint:
isRandomAccessRange!Range
/dlang/dmd/linux/bin64/../../src/phobos/std/random.d(2609):
choice(Range)(auto ref Range range)
I am treating aa as a set and want to pick a random word from it.
What am I doing wrong?
I'm sorry I can't give a link to this code in the D playground
but the URL in the web browser is just https://run.dlang.io/ and
when I click Shorten to get a URL nothing seems to happen (using
Firefox on Linux).
weekly news?
Is there a "D weekly news" I could do an email subscription to? Or at least a way to get notified by email when a new item appears on https://dlang.org/blog/ ?
