Re: Fastest JSON parser in the world is a D project
Am Fri, 13 Jul 2018 18:14:35 + schrieb iris : > Any idea about the performance of this json parser? > https://jsonformatter.org/json-parser ? That one is implemented in client side JavaScript. I didn't measure it, but the closest match in Kostya's benchmark could be the Node JS entry that is an order of magnitude slower. -- Marco
Re: DConf 2018 Livestream
Am Thu, 03 May 2018 09:23:43 + schrieb Mike Parker : > On Thursday, 3 May 2018 at 07:14:51 UTC, Timoses wrote: > > > > > Will the videos of the talks etc. be available online later? > > That's the plan. > > > > > As Bastiaan Veelo wrote it appears that something went wrong > > with the youtube recordings. Hope this gets resolved and at > > least the upcoming talks, if not all, will be available. > > Yes, unfortunately some of the videos were lost. We should be > good from here on out. Did the equipment only support live streaming OR recording or what was going on? Because one gets these 128 GB SD cards for 40€ and they are reusable. -- Marco
Re: Tilix 1.7.9 Released
Am Mon, 30 Apr 2018 15:18:18 + schrieb steven kladitis : > On Sunday, 29 April 2018 at 14:49:09 UTC, Dgame wrote: > > On Sunday, 29 April 2018 at 14:01:10 UTC, Gerald wrote: > >> [...] > > > > Since I'm on Windows I sadly can't use it, but I suggested it > > to my colleagues - which are using Ubuntu - half a year ago and > > they use it since then and still love it. So keep up the good > > work! :) > > > I am on Windows also. If you end up with one that works with > Windows. :):):):):):):) > > Thanks No no, the deal is, you Windows people switch to Linux if you suddenly feel like a terminal is a basic need. ;) https://linuxmint.com/ https://manjaro.org/ https://www.ubuntu.com/ https://www.debian.org/ https://solus-project.com/ https://elementary.io/ https://antergos.com/ https://getfedora.org/ https://www.trueos.org/ (FreeBSD) https://mxlinux.org/ -- Marco
Re: Why think unit tests should be in their own source code hierarchy instead of side-by-side
I understand your opinion and I think it is all reasonable. You talk about longer compile times since every D module is like a C++ header. That touches one of my pet peeves with the language or eco system as it stands and I wonder if you would agree with me on the following: Libraries should be tied into applications using interface files (*.di) that are auto-generated by the compiler for the _library author_ with inferred function attributes. If after a code change, a declaration in the *.di file changes, the library's interface changed and a new minor version must be released. The language must allow to explicitly declare a function or method as @gc, impure, etc. so the auto-inferred attributes don't later become an issue when the implementation changes from e.g. a pure to an impure one. Opaque struct pointers as seen in most C APIs should also be considered for *.di files to reduce the number of imports for member fields. That means: * No more fuzzyness about whether a library function will remain @nogc, @safe, etc. in the next update. * Explicit library boundaries that don't recursively import the world. -- Marco
Re: Beta 2.076.1
Am Sat, 07 Oct 2017 12:45:30 + schrieb Martin Nowak : > On Sunday, 1 October 2017 at 17:36:12 UTC, Marco Leise wrote: > > Other than that I'm happy with the package, as it provides the > > man pages, pre-built HTML documentation and a binary to > > bootstrap dmd on systems that lack a D compiler. (The use case > > being compilation from source for Gentoo Linux.) > > I won't reorganize the folder structure in a point release. But > thanks for pointing out that this needs a fix. > Would getting the sources as a separate download (or just from > github) be a feasible alternative. Yes, it is absolutely feasible to download several files. Currently the URIs are computable from the release year and version, so maintaining 2 or 3 URIs is ok. Also now that the path issues are worked around on my side they have no running maintenance cost. It is different with missing files, because then you have to manually collect the version that matches the release from GitHub. If you thought of removing the sources from the archives entirely and point to GitHub that's fine with me. The concern Eugene brought up about checksum verification failures still holds though. It then depends on how quickly the package maintainers respond and fix their scripts. -- Marco
Re: Beta 2.076.1
P.S.: The directory layout could be improved as well. Currently there is: src\ +-dmd +-druntime +-phobos But druntime in posix.mak:10 expects a src directory inside the dmd directory: dmd\ +-src So effectively the directory names have to be swapped for that to work. At that point the superfluous directories for the other operating systems, containing only dmd.conf could be removed as well. Other than that I'm happy with the package, as it provides the man pages, pre-built HTML documentation and a binary to bootstrap dmd on systems that lack a D compiler. (The use case being compilation from source for Gentoo Linux.) -- Marco
Re: Beta 2.076.1
Could you include the "default_ddoc_theme.ddoc" and "config.sh" in the source releases? The sources cannot be compiled without them. -- Marco
Re: new cpuid is ready for comments
Am Mon, 11 Jul 2016 16:30:44 + schrieb Ilya Yaroshenko : > Please report your CPU (GitHub/Gist): Instead of yet another Haswell/Broadwell, here is an oldie but goldie: A 16 year old Pentium-III-M from an IBM T23 notebook: https://gist.github.com/mleise/4ed5637371ca5cea07bef8241adc7d80 I repost this in your bug tracker momentarily. -- Marco
Re: QtE5 - is a wrapping of Qt-5 for D
Very nice! -- Marco
Re: LZ4 decompression at CTFE
Am Tue, 26 Apr 2016 23:55:46 -0700 schrieb Walter Bright : > On 4/26/2016 3:05 PM, Stefan Koch wrote: > > Hello, > > > > originally I want to wait with this announcement until DConf. > > But since I working on another toy. I can release this info early. > > > > So as per title. you can decompress .lz4 flies created by the standard lz4hc > > commnadline tool at compile time. > > > > No github link yet as there is a little bit of cleanup todo :) > > > > Please comment. > > Sounds nice. I'm curious how it would compare to: > > https://www.digitalmars.com/sargon/lz77.html > > https://github.com/DigitalMars/sargon/blob/master/src/sargon/lz77.d There exist some comparisons for the C++ implementations (zlib's DEFLATE being a variation of lz77): http://catchchallenger.first-world.info//wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO https://pdfs.semanticscholar.org/9b69/86f2fff8db7e080ef8b02aa19f3941a61a91.pdf (pg.9) The high compression variant of lz4 basically like gzip with 9x faster decompression. That makes it well suited for use cases where you compress once, decompress often and I/O sequential reads are fast e.g. 200 MB/s or the program does other computations meanwhile and one doesn't want decompression to use a lot of CPU time. -- Marco
Re: Blog post: PGO: Optimizing D's virtual calls
Am Fri, 15 Apr 2016 09:04:21 + schrieb Johan Engelen : > On Wednesday, 13 April 2016 at 11:34:26 UTC, Johan Engelen wrote: > > Hi all, > > I've written an article about how I implemented > > profile-guided optimization (PGO) of virtual calls to direct > > calls (a micro-micro-optimization, expected performance gain of > > just a few percent if any!). > > I've updated the article with performance measurements of just > the D code of LDC: the improvement with PGO is about 7% ! Compilers are full of logic that makes some code 0.5% faster. The sum of all these little bits is what makes up an optimizing compiler and if today you still find an optimization that's worth 1 or 2% you have a jackpot! 7% is quite a bit more than any other option like -cpu=native could achieve. If feasible I'd enjoy if ldc2 shipped with a pgo configuration option. If you use the compiler in a lot of edit/compile cycles it will pay off. -- Marco
Re: Release D 2.071.0
Am Thu, 07 Apr 2016 10:13:35 + schrieb Dicebot : > On Tuesday, 5 April 2016 at 22:43:05 UTC, Martin Nowak wrote: > > Glad to announce D 2.071.0. > > > > http://dlang.org/download.html > > > > This release fixes many long-standing issues with imports and > > the module > > system. > > See the changelog for more details. > > > > http://dlang.org/changelog/2.071.0.html > > > > -Martin > > It is second time git tag for DMD release refers to commit with > VERSION file content which doesn't match the tag. May indicate > something is wrong in release procedure. Or maybe something is wrong with source based Linux distributions in this time and age. I don't know. But I'm glad that you fire proof the source bundles first, before I move my lazy ass to update DMD packages for Gentoo. I hope to start from a good, clean 2.071.0/2.071.1 tar ball. :D Nice work on the import bugs. There is so much less on the attic now. -- Marco
Re: D Profile Viewer
Am Thu, 24 Mar 2016 20:34:07 + schrieb Andrew : > Hi > > I wrote a program to turn the non-human-readable trace.log into > an interactive HTML file that can be used to help profile a D > program. > > Its here: https://bitbucket.org/andrewtrotman/d-profile-viewer > > There's also a readme that (hopefully) explains how to use it. > > Please let me know if you find any bugs. > > Andrew. Sexy pie charts! Although I'm using OProfile since it works without instrumenting the code. -- Marco
Re: Graillon 1.0, VST effect fully made with D
We can probably agree that we don't know about the impact on a large multimedia application written in D. What you can communicate is: Create a @nogc thread routine and don't register it with the GC to write real-time VSTs. Guillaume did a good job, taking the GC out of the real-time thread. It's D, it is a bit of a hack, it's the correct way to do it and works. But I don't see it debunking any myths about GC and real time... A) It doesn't mix them to begin with. B) "Realtime GCs" are a thing. D's GC is not optimized for such a use case. C) With a small heap it doesn't matter. (We need more complex multimedia projects.) What I've seen is a program, a non-linear video editor, called PowerDirector that pauses for seconds every now and then. These pauses reminded me a lot of GC pauses, but I can't be sure. Although memory use is less after the pause, it could also be a cleaning of caches. In any case quite a few of these applications try to make "good use" of available RAM, causing constant memory pressure. Now there has been so much talk about the GC that I don't even know what the filter does! -- Marco
Re: Tonight: Introduction to D at Codeaholics (HK)
Am Thu, 12 Nov 2015 01:30:06 +0800 schrieb Lionello Lunesu : > * Why doesn't D explicitly specify the exceptions that can be thrown? > (To which I answered that I never saw the point in Java and only found > it tedious. This did not convince the person.) Maybe that's your point of view or maybe you were just undecided. When you write a library it is sometimes better to be explicit about your interface and that includes any exceptions. This not only enables users of the library to selectively catch exceptions they can handle at layer X, but facilitates static checks: * Are any exceptions missing from DDoc/@throws that are thrown in the code? (Potential for auto-generating the DDoc.) * A function is nothrow, but the user only catches, e.g. UtfException explicitly. Is that the only exception type that could occur? There were some more nice points that I don't remember from when I failed at implementing this many months ago. The questioner has my sympathy in any case, but it's certainly not a priority. The way I wanted to implement it was by making attribute-less functions map to @throws(Exception), which implicitly makes the feature opt-in: It is always correct to state a super set of the actual thrown exceptions in an API to have room for extensions. Thrown exceptions would be collected much like nothrow is deduced right now, but as a list with respect to the hierarchical nature of Exceptions. -- Marco
Re: Fastest JSON parser in the world is a D project
Am Tue, 27 Oct 2015 14:00:06 + schrieb Martin Nowak : > On Tuesday, 27 October 2015 at 13:14:36 UTC, wobbles wrote: > >> How can `coordinates` member be known at compile-time when the > >> input argument is a run-time string? > > > > I suspect through the opDispatch operator overload. > > > > http://dlang.org/operatoroverloading.html#dispatch > > Yikes, this is such an anti-pattern. > https://github.com/rejectedsoftware/vibe.d/issues/634 For my defense I can say that the JSON parser is not a range and thus less likely to be used in UFCS chains. It can be replaced with .singleKey!"coordinates"() -- Marco
Re: Fastest JSON parser in the world is a D project
Am Thu, 22 Oct 2015 06:10:56 -0700 schrieb Walter Bright : > On 10/21/2015 3:40 PM, Laeeth Isharc wrote: > > Have you thought about writing up your experience with writing fast json? > > A bit > > like Walter's Dr Dobbs's article on wielding a profiler to speed up dmd. > > Yes, Marco, please. This would make an awesome article, and we need articles > like that! > > You've already got this: > > https://github.com/kostya/benchmarks/pull/46#issuecomment-147932489 > > so most of it is already written. There is at least one hurdle. I don't have a place to publish articles, no personal blog or site I contribute articles to and I don't feel like creating a one-shot one right now. :) -- Marco
Re: Fastest JSON parser in the world is a D project
Am Wed, 21 Oct 2015 04:17:16 + schrieb Laeeth Isharc : > Very impressive. > > Is this not quite interesting ? Such a basic web back end > operation, and yet it's a very different picture from those who > say that one is I/O or network bound. I already have JSON files > of a couple of gig, and they're only going to be bigger over > time, and this is a more generally interesting question. > > Seems like you now get 2.1 gigbytes/sec sequential read from a > cheap consumer SSD today... You have this huge amount of Reddit API JSON, right? I wonder if your processing could benefit from the fast skipping routines or even reading it as "trusted JSON". -- Marco
Re: Fastest JSON parser in the world is a D project
Am Wed, 21 Oct 2015 17:00:39 + schrieb Suliman : > >> > Nice! I see you are using bitmasking trickery in multiple > >> > places. stdx.data.json is mostly just the plain lexing > >> > algorithm, with the exception of whitespace skipping. It was > >> > already very encouraging to get those benchmark numbers that > >> > way. Good to see that it pays off to go further. > >> > >> Is there any chance that new json parser can be include in > >> next versions of vibed? And what need to including its to > >> Phobos? > > > > It's already available on code.dlang.org: > > http://code.dlang.org/packages/std_data_json > > > Jonatan, I mean https://github.com/mleise/fast :) That's nice, but it has a different license and I don't think Phobos devs would be happy to see all the inline assembly I used and duplicate functionality like the number parsing and UTF-8 validation and missing range support. -- Marco
Re: Beta D 2.069.0-b2
Am Tue, 20 Oct 2015 19:26:13 +0200 schrieb Martin Nowak : > On 10/17/2015 09:05 PM, Marco Leise wrote: > > Oh wait, false alert. That was a relic from older days. My > > build script placed a dummy dmd.conf there. > > > > I do seem to get problems with ldc2-0.16.0: > > Are you using something befor 0.16.0-beta2, b/c I thought the problem > was resolved. > https://github.com/D-Programming-Language/dmd/pull/5025#issuecomment-142143727 Indeed I should have checked that. I'm using 0.16.0_alpha4. Alright then. Everything works as designed now. :) -- Marco
Re: Fastest JSON parser in the world is a D project
Am Sun, 18 Oct 2015 03:40:52 + schrieb rsw0x : > On Wednesday, 14 October 2015 at 07:01:49 UTC, Marco Leise wrote: > > JSON parsing in D has come a long way, especially when you look > > at it from the efficiency angle as a popular benchmark does > > that has been forked by well known D contributers like Martin > > Nowak or Sönke Ludwig. > > > > [...] > > Slightly OT: > You have a std.simd file in your repo, was this written by you or > is there a current std.simd proposal that I'm unaware of? Manu wrote that back in the days with the idea that it would help writing portable SIMD code on many architectures: https://github.com/TurkeyMan/simd Working in the 3D visualization business and having held at least one talk about SIMD it was no coincidence that he was interested in better vector math support. Inclusion into Phobos was planned. DMD needs some upgrading of the somewhat ad hoc SIMD intrinsic implementation though: https://issues.dlang.org/buglist.cgi?keywords=SIMD&resolution=--- Many instructions cannot be expressed outside of inline assembly which doesn't inline. -- Marco
Re: Fastest JSON parser in the world is a D project
Am Sat, 17 Oct 2015 16:27:06 + schrieb Sean Kelly : > On Saturday, 17 October 2015 at 16:14:01 UTC, Andrei Alexandrescu > wrote: > > On 10/17/15 6:43 PM, Sean Kelly wrote: > >> If this is the benchmark I'm remembering, the bulk of the time > >> is spent > >> parsing the floating point numbers. So it isn't a test of JSON > >> parsing > >> in general so much as the speed of scanf. > > > > In many cases the use of scanf can be replaced with drastically > > faster methods, as I discuss in my talks on optimization > > (including Brasov recently). I hope they'll release the videos > > soon. -- Andrei > > Oh absolutely. My issue with the benchmark is just that it claims > to be a JSON parser benchmark but the bulk of CPU time is > actually spent parsing floats. I'm on my phone though so perhaps > this is a different benchmark--I can't easily check. The one I > recall came up a year or so ago and was discussed on D.general. 1/4 to 1/3 of the time is spent parsing numbers in highly optimized code. You see that in a profiler the number parsing shows up on top, but the benchmark also exercises the structural parsing a lot. It is not a very broad benchmark though, lacking serialization, UTF-8 decoding, validation of results etc. I believe the author didn't realize how over time it became the go-to performance test. The author of RapidJSON has a very in-depth benchmark suite, but it would be a bit of work to get something non-C++ integrated: https://github.com/miloyip/nativejson-benchmark It includes conformance tests as well. -- Marco
Re: Beta D 2.069.0-b2
Oh wait, false alert. That was a relic from older days. My build script placed a dummy dmd.conf there. I do seem to get problems with ldc2-0.16.0: make -C druntime -f posix.mak MODEL=32 ../dmd/src/dmd -conf= -c -o- -Isrc -Iimport -Hfimport/core/sync/barrier.di src/core/sync/barrier.d core.exception.AssertError@expression.d(4369): Assertion failure That is this line of code: https://github.com/D-Programming-Language/dmd/blob/v2.069.0-b2/src/expression.d#L4369 Stack trace (with file + line numbers now, hey!): #0 StringExp::compare(RootObject*) (this=0xb66e30, obj=0xb65c80) at expression.d:4341 #1 0x004fb6ed in StringExp::equals(RootObject*) (this=0xb66e30, o=0xb65c80) at expression.d:4175 #2 0x004c4fe9 in match(RootObject*, RootObject*) (o1=0xb66e30, o2=0xb65c80) at dtemplate.d:246 #3 0x004c51c6 in arrayObjectMatch(Array*, Array*) (oa1=0x764aca98, oa2=0x764ac898) at dtemplate.d:290 #4 0x004cddd7 in TemplateInstance::compare(RootObject*) (this=0x764aca00, o=0x764ac800) at dtemplate.d:6232 #5 0x004cdaf8 in TemplateDeclaration::findExistingInstance(TemplateInstance*, Array*) (this=0x764ac600, tithis=0x764aca00, fargs=0x0) at dtemplate.d:2039 #6 0x004d2f24 in TemplateInstance::semantic(Scope*, Array*) (this=0x764aca00, sc=0x765dfc00, fargs=0x0) at dtemplate.d:5583 #7 0x00406877 in TemplateInstance::semantic(Scope*) (this=0x764aca00, sc=0x765dfc00) at dtemplate.d:5967 #8 0x0057a03a in TypeInstance::resolve(Loc, Scope*, Expression**, Type**, Dsymbol**, bool) (this=0x764ae100, loc=..., sc=0x765dfc00, pe=0x7fffcec0, pt=0x7fffcec8, ps=0x7fffceb8, intypeid=false) at mtype.d:7412 #9 0x0057a327 in TypeInstance::toDsymbol(Scope*) (this=0x764ae100, sc=0x765dfc00) at mtype.d:7459 #10 0x0043c5d6 in AliasDeclaration::semantic(Scope*) (this=0x764ae200, sc=0x765dfc00) at .:598 #11 0x004897f9 in Module::semantic() (this=0x76378400) at dmodule.d:976 #12 0x00488e0f in Import::semantic(Scope*) (this=0x76a82800, sc=0x76aa9c00) at dimport.d:258 #13 0x0042759a in AttribDeclaration::semantic(Scope*) (this=0x76a82900, sc=0x76aa9600) at attrib.d:168 #14 0x004897f9 in Module::semantic() (this=0x76a7fe00) at dmodule.d:976 #15 0x00488e0f in Import::semantic(Scope*) (this=0x77eca100, sc=0x77ed2200) at dimport.d:258 #16 0x0042759a in AttribDeclaration::semantic(Scope*) (this=0x77eca200, sc=0x77ecee00) at attrib.d:168 #17 0x004897f9 in Module::semantic() (this=0x77ec9000) at dmodule.d:976 #18 0x00567de5 in tryMain(unsigned long, char const**) (argc=8, argv=0x7fffe978) at mars.d:1485 #19 0x0056a567 in D main () at mars.d:1695 `sz` is 0, `string` and `len` seem to be ok. -- Marco
Re: Beta D 2.069.0-b2
Am Wed, 14 Oct 2015 15:52:57 +0200 schrieb Martin Nowak : > Second beta for the 2.069.0 release. > > http://dlang.org/download.html#dmd_beta > http://dlang.org/changelog/2.069.0.html > > Please report any bugs at https://issues.dlang.org > > -Martin When I use a specific host compiler, it still picks up the dmd.conf provided in the package and doesn't find object.d. Should I manually delete dmd.conf before building? -- Marco
Re: Fastest JSON parser in the world is a D project
Am Sat, 17 Oct 2015 11:12:08 + schrieb Ola Fosheim Grøstad : > […] you could try to construct a simple benchmark where you > iterate over memory (M*cache 3 size) using a "realistic" pattern > like brownian motion in N threads and also repeatedly/concurrently > load JSON code for different file sizes so that the CPUs page table > mechanisms are stressed by mmap, cache misses and (possibly) page > faults. O.O Are you kidding me? Just give me the correct value already. -- Marco
Re: Fastest JSON parser in the world is a D project
Am Sat, 17 Oct 2015 08:29:24 + schrieb Ola Fosheim Grøstad : > On Saturday, 17 October 2015 at 08:20:33 UTC, Daniel N wrote: > > On Saturday, 17 October 2015 at 08:07:57 UTC, Martin Nowak > > wrote: > >> On Wednesday, 14 October 2015 at 07:35:49 UTC, Marco Leise > >> wrote: > >>> - Data size limited by available contiguous virtual memory > >> > >> Mmaping files for sequential reading is a very debatable > >> choice, b/c the common use case is to read a file once. You > >> should at least compare the numbers w/ drop_caches between > >> each run. The results are: * The memory usage is then fixed at slightly more than the file size. (While it often stays below when using the disk cache.) * It would still be faster than copying the whole thing to a separate memory block. * Depending on whether the benchmark system uses a HDD or SSD, the numbers may be rendered meaningless by a 2 seconds wait on I/O. * Common case yes, but it is possible that you read JSON that had just been saved. > > It's a sensible choice together with appropriate madvise(). Obviously agreed :). Just that in practice (on my HDD system) it never made a difference in I/O bound sequential reads. So I removed posix_madvise. > Mmap is very expensive, as it affects all cores, you need a > realistic multithreaded aync benchmark on smaller files to see > the effect. That's valuable information. It is trivial to read into an allocated block when the file size is below a threshold. I would just need a rough file size. Are you talking about 4K pages or mega-bytes? 64 KiB maybe? -- Marco
Re: Fastest JSON parser in the world is a D project
Am Sat, 17 Oct 2015 09:27:46 +0200 schrieb Sönke Ludwig : > Am 16.10.2015 um 17:11 schrieb Marco Leise: > > Am Thu, 15 Oct 2015 18:17:07 +0200 > > schrieb Sönke Ludwig : > > > >> (...) > >> Do you have a test case for your error? > > > > Well it is not an error. Rory originally wrote about > > conversions between "1" and 1 happening on the browser side. > > That would mean adding a quirks mode to any well-behaving JSON > > parser. In this case: "read numbers as strings". Hence I was > > asking if the data on the client could be fixed, e.g. the json > > number be turned into a string first before serialization. > > > > Okay, I obviously misread that as a once familiar issue. Maybe it indeed > makes sense to add a "JavaScript" quirks mode that behaves exactly like > a JavaScript interpreter would. Ok, but remember: https://www.youtube.com/watch?v=20BySC_6HyY And then think again. :D -- Marco
Re: Fastest JSON parser in the world is a D project
Am Fri, 16 Oct 2015 11:09:37 + schrieb Per Nordlöw : > On Wednesday, 14 October 2015 at 07:01:49 UTC, Marco Leise wrote: > > https://github.com/kostya/benchmarks#json > > Does fast.json use any non-standard memory allocation patterns or > plain simple GC-usage? Plain GC. I found no way in D to express something as "borrowed", but the GC removes the ownership question from the picture. That said the only thing that I allocate in this benchmark is the dynamic array of coordinates (1_000_000 * 3 * double.sizeof), which can be replaced by lazily iterating over the array to remove all allocations. Using these lazy techniques for objects too and calling .borrow() instead of .read!string() (which .idups) should allow GC free usage. (Well, except for the one in parseJSON, where I append 16 zero bytes to the JSON string to make it SSE safe.) -- Marco
Re: Fastest JSON parser in the world is a D project
Am Thu, 15 Oct 2015 18:46:12 +0200 schrieb Sönke Ludwig : > Am 14.10.2015 um 09:01 schrieb Marco Leise: > > […] > > stdx.data.json: 2.76s, 207.1Mb (LDC) > > > > Yep, that's right. stdx.data.json's pull parser finally beats > > the dynamic languages with native efficiency. (I used the > > default options here that provide you with an Exception and > > line number on errors.) > > From when are the numbers for stdx.data.json? The latest results for > the pull parser that I know of were faster than RapidJson: > http://forum.dlang.org/post/wlczkjcawyteowjbb...@forum.dlang.org You know, I'm not surprised at the "D new lazy Ldc" result, which is in the ball park figure of what I measured without exceptions & line-numbers, but the Rapid C++ result seems way off compared to kostya's listing. Or maybe that Core i7 doesn't work well with RapidJSON. I used your fork of the benchmark, made some modifications like adding taggedalgebraic and what else was needed to make it compile with vanilla ldc2 0.16.0. Then I removed the flags that disable exceptions and line numbers. Compilation options are the same as for the existing gdc and ldc2 entries. I did not add " -partial-inliner -boundscheck=off -singleobj ". > Judging by the relative numbers, your "fast" result is still a bit > faster, but if that doesn't validate, it's hard to make an objective > comparison. Every value that is read (as opposed to skipped) is validated according to RFC 7159. That includes UTF-8 validation. Full validation (i.e. readJSONFile!validateAll(…);) may add up to 14% overhead here. -- Marco
Re: Fastest JSON parser in the world is a D project
Am Thu, 15 Oct 2015 18:17:07 +0200 schrieb Sönke Ludwig : > Am 15.10.2015 um 13:06 schrieb Rory McGuire via Digitalmars-d-announce: > > In browser JSON.serialize is the usual way to serialize JSON values. > > The problem is that on D side if one does deserialization of an object > > or struct. If the types inside the JSON don't match exactly then vibe > > freaks out. > > For float and double fields, the serialization code should actually > accept both, floating point and integer numbers: > > https://github.com/rejectedsoftware/vibe.d/blob/2fffd94d8516cd6f81c75d45a54c655626d36c6b/source/vibe/data/json.d#L1603 > https://github.com/rejectedsoftware/vibe.d/blob/2fffd94d8516cd6f81c75d45a54c655626d36c6b/source/vibe/data/json.d#L1804 > > Do you have a test case for your error? Well it is not an error. Rory originally wrote about conversions between "1" and 1 happening on the browser side. That would mean adding a quirks mode to any well-behaving JSON parser. In this case: "read numbers as strings". Hence I was asking if the data on the client could be fixed, e.g. the json number be turned into a string first before serialization. -- Marco
Re: Fastest JSON parser in the world is a D project
Am Wed, 14 Oct 2015 08:19:52 + schrieb Per Nordlöw : > On Wednesday, 14 October 2015 at 07:01:49 UTC, Marco Leise wrote: > > https://github.com/kostya/benchmarks#json > > I can't find fast.json here. Where is it? »»» D Gdc Fast 0.34226.7 ««« C++ Rapid 0.79687.1 Granted if he wrote "D fast.json" it would have been easier to identify. -- Marco
Re: Fastest JSON parser in the world is a D project
Am Wed, 14 Oct 2015 10:22:37 +0200 schrieb Rory McGuire via Digitalmars-d-announce : > Does this version handle real world JSON? > > I've keep getting problems with vibe and JSON because web browsers will > automatically make a "1" into a 1 which then causes exceptions in vibe. > > Does yours do lossless conversions automatically? No I don't read numbers as strings. Could the client JavaScript be fixed? I fail to see why the conversion would happen automatically when the code could explicitly check for strings before doing math with the value "1". What do I miss? -- Marco
Re: Fastest JSON parser in the world is a D project
Am Wed, 14 Oct 2015 07:55:18 +
schrieb Idan Arye :
> On Wednesday, 14 October 2015 at 07:35:49 UTC, Marco Leise wrote:
> > auto json = parseTrustedJSON(`{ "coordinates": [ { "x": 1,
> > "y": 2, "z": 3 }, … ] }`);
>
> I assume parseTrustedJSON is not validating? Did you use it in
> the benchmark? And were the competitors non-validating as well?
That is correct. For the benchmark parseJSONFile was used
though, which validates UTF-8 and JSON in the used portions.
That probably renders your third question superfluous. I
wouldn't know anyways, but am inclined to think they all
validate the entire JSON and some may skip UTF-8 validation,
which is a low cost operation in this ASCII file anyways.
--
Marco
Fastest JSON parser in the world is a D project
fast.json usage:
UTF-8 and JSON validation of used portions by default:
auto json = parseJSONFile("data.json");
Known good file input:
auto json = parseTrustedJSONFile("data.json");
auto json = parseTrustedJSON(`{"x":123}`);
Work with a single key from an object:
json.singleKey!"someKey"
json.someKey
Iteration:
foreach (key; json.byKey) // object by key
foreach (idx; json)// array by index
Remap member names:
@JsonRemap(["clazz", "class"])
struct S { string clazz; }
@JsonRemap(["clazz", "class"])
enum E { clazz; }
Example:
double x = 0, y = 0, z = 0;
auto json = parseTrustedJSON(`{ "coordinates": [ { "x": 1, "y": 2, "z": 3
}, … ] }`);
foreach (idx; json.coordinates)
{
// Provide one function for each key you are interested in
json.keySwitch!("x", "y", "z")(
{ x += json.read!double; },
{ y += json.read!double; },
{ z += json.read!double; }
);
}
Features:
- Loads double values in compliance with IEEE round-to-nearest
(no precision loss in serialization->deserialization round trips)
- UTF-8 validation of non-string input (file, ubyte[])
- Currently fastest JSON parser money can buy
- Reads strings, enums, integral types, double, bool, POD
structs consisting of those and pointers to such structs
Shortcomings:
- Rejects numbers with exponents of huge magnitude (>=10^28)
- Only works on Posix x86/amd64 systems
- No write capabilities
- Data size limited by available contiguous virtual memory
--
Marco
Fastest JSON parser in the world is a D project
JSON parsing in D has come a long way, especially when you look at it from the efficiency angle as a popular benchmark does that has been forked by well known D contributers like Martin Nowak or Sönke Ludwig. The test is pretty simple: Parse a JSON object, containing an array of 1_000_000 3D coordinates in the range [0..1) and average them. The performance of std.json in parsing those was horrible still in the DMD 2.066 days*: DMD : 41.44s, 934.9Mb Gdc : 29.64s, 929.7Mb Python : 12.30s, 1410.2Mb Ruby: 13.80s, 2101.2Mb Then with 2.067 std.json got a major 3x speed improvement and rivaled the popular dynamic languages Ruby and Python: DMD : 13.02s, 1324.2Mb In the mean time several other D JSON libraries appeared with varying focus on performance or API: Medea : 56.75s, 1753.6Mb (GDC) libdjson : 24.47s, 1060.7Mb (GDC) stdx.data.json: 2.76s, 207.1Mb (LDC) Yep, that's right. stdx.data.json's pull parser finally beats the dynamic languages with native efficiency. (I used the default options here that provide you with an Exception and line number on errors.) A few days ago I decided to get some practical use out of my pet project 'fast' by implementing a JSON parser myself, that could rival even the by then fastest JSON parser, RapidJSON. The result can be seen in the benchmark results right now: https://github.com/kostya/benchmarks#json fast: 0.34s, 226.7Mb (GDC) RapidJSON: 0.79s, 687.1Mb (GCC) (* Timings from my computer, Haswell CPU, Linux amd64.) -- Marco
Re: Walter and I talk about D in Romania
Am Fri, 2 Oct 2015 07:25:44 -0400 schrieb Andrei Alexandrescu : > Walter and I will travel to Brasov, Romania to hold an evening-long > event on the D language. There's been strong interest in the event with > over 300 registrants so far. > > http://curiousminds.ro > > Scott Meyers will guest star in a panel following the talks. We're all > looking forward to it! > > > Andrei That's a lot of people. You must be some kind of programming national hero in Romania. Good luck and watch out for those C++ moroi in the audience! -- Marco
Re: Release D 2.068.1
Am Tue, 15 Sep 2015 08:39:41 + schrieb John Colvin : > On Monday, 14 September 2015 at 23:53:16 UTC, Martin Nowak wrote: > > On Monday, 14 September 2015 at 20:14:45 UTC, Jack Stouffer > > wrote: > >> On Monday, 14 September 2015 at 17:51:59 UTC, Martin Nowak > >> wrote: > >>> What platform are you on? > >> > >> I'm on OS X, using the homebrew version of DMD. And homebrew > >> is telling me that I have 2.068.1 installed > > > > Well I guess it's a bug in the homebrew script then. > > Nobody is setting the VERSION file and there is no git repo to > > query. > > https://github.com/Homebrew/homebrew/blob/f8b0ff3ef63e60a1da17ec8d8e68d949b1cebc27/Library/Formula/dmd.rb#L50 > > Where is the VERSION file documented? Why does it need manual > intervention only for patch releases and pre-releases? I noticed this as well. There are different VERSION files depending on where you download DMD source code from: The releases on GitHub have 2.068 while the releases on dlang.org have 2.068.1 in VERSION. That's easy to miss when you build from source. (Also the dlang.org downloads add to the DMD download statistics, which is why I use those even though they include extra binaries for all OSs.) -- Marco
Re: Beta D 2.068.2-b1
And I found another regression in 2.068.1: https://issues.dlang.org/show_bug.cgi?id=15056 -- Marco
Re: Beta D 2.068.2-b1
Am Thu, 10 Sep 2015 05:37:48 +0200 schrieb Martin Nowak : > Due to a regression in 2.068.1 we'll directly follow up with an > unplanned point release 2.068.2. > This is the beta for that point release. > > http://downloads.dlang.org/pre-releases/2.x/2.068.2/ > > Please test any of your code against this beta to help finding bugs. > > https://issues.dlang.org/ > > -Martin I tested some code with 2.068.x today and filed this one: https://issues.dlang.org/show_bug.cgi?id=15042 Kudos to CyberShadow at this point for the excellent "digger" tool. -- Marco
Re: Arch Linux D package update
Am Fri, 12 Jun 2015 06:26:10 + schrieb "Dicebot" : > I will look into making it user-local service > (https://wiki.archlinux.org/index.php/Systemd/User), that is > simply I currently don't know much about and didn't want to delay > update much longer. I switched back to system wide for now. If dcd-server evolves further and starts reading not only includes but also the port number from a user local file (or uses per-user named sockets like some other services), I'll look into it again. Anyways, I learned a lot about systemd during this exercise, that I should have learned earlier. The Arch Linux Wiki is a great source of information. :) -- Marco
Re: Arch Linux D package update
Am Thu, 04 Jun 2015 14:46:38 + schrieb "Dicebot" : > - provides systemd service : `sudo systemctl enable dcd.service` to > start automatically upon system startup Is it really a system-wide server? I can see how it might be technically necessary due to how systemd works and how you would need to pass a different IP port for every user. On the other hand, every user can add imports at runtime and shut the server down. They might even have to, if they want to switch projects and clean up the import path list. > - provides default /etc/dcd.conf with stdlib paths for Arch Linux Should work so long noone gives root a XDG_CONFIG_HOME or writes to /root/.config/dcd/dcd.conf. Just contemplating if when running as system wide service it should generally read from /etc/[dcd/]dcd.conf instead of user directories. Maybe via a --system option to dcd-server. https://github.com/Hackerpilot/DCD/blob/master/src/server.d#L248 -- Marco
Re: Arch Linux D package update
Am Thu, 04 Jun 2015 14:46:38 + schrieb "Dicebot" : > - provides systemd service : `sudo systemctl enable dcd.service` to > start automatically upon system startup > - provides default /etc/dcd.conf with stdlib paths for Arch Linux Would you mind if I copy this idea? -- Marco
Re: New D tool releases
Am Mon, 08 Jun 2015 19:18:02 +
schrieb "Brian Schott" :
> Dfix 0.2.2:
> https://github.com/Hackerpilot/dfix/releases/tag/v0.2.2
> * Fixed a bug that caused the string concatenation fix to be
> applied
>inside of "deprecated" attributes. I plan to revert this fix if
>dmd starts accepting `deprecated("string1" ~ "string2")`
>
> Dfmt 0.4.0-beta1 and 0.3.6
> https://github.com/Hackerpilot/dfmt/releases/tag/v0.3.6
> * Fix a spacing bug with function contracts
>
> https://github.com/Hackerpilot/dfmt/releases/tag/v0.4.0-beta1
> * Introduced support for `// dfmt off` and `// dfmt on` comments
> * Added the ability to configure spacing around the `:` in
> selective
>imports.
> * Added the ability to configure indentation of case statements.
> * Added the ability to configure indentation of attribute
> declarations.
> * Added the ability to configure formatting of labeled loops.
>
> D-Scanner 0.2.0-beta1
> https://github.com/Hackerpilot/Dscanner/releases/tag/0.2.0-beta1
> * Fixed several bugs in the lint checks.
> * Added support for generating Emacs etags files.
> * `--imports` option can now work on more than one file at a time
> * Added undocumented public declaration check.
> * Added unused label check.
> * Added check for variables with the same name as labels.
> * Added check for redundant parenthesis.
> * Added "line" field to ctags output to work with the Tagbar
> plugin better.
> * Unused variable declaration check is now disabled inside of
> __traits
>expressions.
> * Undocumented declaration check is disabled for things marked
>"deprecated" or "@disable".
> * Undocumented declaration check ignores @property functions.
Sounds like a big step forward! One question though, what's
the rationale behind @properties not needing documentation?
--
Marco
Re: DUB 0.9.23 released
Am Mon, 06 Apr 2015 10:21:45 +0200 schrieb Sönke Ludwig : > The new version contains some important bug fixes for sub modules and > overridden string imports, as well as some other major fixes. Apart from > that, the major additions are: > > - "copyFiles" now get hard linked instead of copied and whole > directories can be copied in addition to individual files > - The "dub init" command has been extended to take an optional list of > dependencies that is added to the generated dub.json > - A new "deimos" template has been been implemented > (dub init --type=deimos) > - The default compiler is now chosen based on what is found in PATH How does it resolve preferences when 3 comilers are on PATH ? =) dmd first? > - New project generators for Sublime Text and CMake > - In single file build mode, "--parallel" can now be used to compile > with multiple compiler instances at once > > Find the full list of changes in the change log [1] and download at: > > http://code.dlang.org/download > > [1]: https://github.com/D-Programming-Language/dub/blob/master/CHANGELOG.md And one idea that recently came to my mind since I'm a fan of shared libraries. Would it work in D to say compile a shared library and generate a number of .di files required to use it? Similar to druntime that provides only a hand full of its modules as .di or where necessary as .d files. I'm looking for a perspective out of the "compile the world" model in Dlang, since as far as I understand it, the D front-end will always parse all imported modules recursively. Even in single-file compilation mode, when only "app.d" changed it would parse the whole thing down to the last corner of all used D libraries like GtkD or vibe.d. With the minimum amount of required .di files for libraries, further private imports inside their modules are hidden away from the compiler, thereby short-circuiting the "import the world" recursion. -- Marco
Re: Standardpaths library
Am Tue, 07 Apr 2015 17:27:34 + schrieb "FreeSlave" : > On Monday, 6 April 2015 at 21:40:28 UTC, Marco Leise wrote: > > > > I believe modern desktops offer enough granularity to cover > > each of those. For example if I was playing a game on Linux > > files would go here: > > > > /usr/share/[games/] - read-only data files. > > ~/.cache/ - downloaded archives, precompiled scripts, browser > > caches and other files that can be recreated or > > fetched again if they were to be deleted > > ~/.config/ - the user's personal configuration; may be > > overriding something in a system directory if > > desired > > ~/.local/share/ - pretty much a catch all for save games, > > user created content that goes beyond the > > scope of config files, highscores, > > highlighting schemes in an IDE, Steam, ... > > > > Or the other way around: > > > > data => /usr/share > > save games => ~/.local/share > > downloaded content => ~/.local/share (or ~/.cache) > > configs=> ~/.config > > > > Windows has the Local and Roaming directories, which > > serve similar but different purposes. E.g. anything machine > > specific or big must not be in Roaming. > > So if I generally asked for the config dir, I'd probably expect > > AppData/Local on Windows and ~/.config on Linux (because I > > might write a configuration that only works for this machine.) > > Roaming is interesting for users that have their profiles on > > servers and might switch to another workstation. So if some > > configuration is "portable" and you want to create something > > really fine grained you could offer that directory as an > > alternative "roaming config dir" (returning null or > > "~/.config" on Linux). > > In any case there will be multiple results for some > > directories (/usr/share, /usr/local/share) and also > > several standard paths mapping to the same physical directory > > (user data and user cache both map to AppData/Local on > > Windows). The user needs to be made aware of this so (s)he > > doesn't overwrite files in one standard path with files in a > > supposedly different one. > > > > So much for my thoughts on standard paths extreme edition. ;) > > Just found out, there's also special Saved Games directory on > Windows. But only starting with Vista, therefore it's not CSIDL, > but KNOWNFOLDERID (same as for Downloads). Still I did not find > fairly new winapi headers for D. Seems like the most use > translation of MinGW headers which stuck at, likely, Windows XP > era. Although we could just read some paths from registry > avoiding calling SHGetKnownFolderPath, it's way too hacky. The druntime Windows headers are written on a "as needed" basis. I don't think anyone would object if you added something you need, except maybe if it stops druntime from working in WinXP in general. Don't know about enums that are only meaningful on Vista and later... > Currently the only Roaming directories returned by standardpaths > library on Windows are Templates and Applications, which is fine. > The whole roaming thing is specific to Windows, but it would be > useful to add separate function probably. What do you think about OS agnostic code? string configPath = standardPath(StandardPath.config, appName); string roamingConfigPath = standardPath(StandardPath.roamingConfig, appName); If this worked on Linux it would return the same path for both calls, but you would avoid introducing version(Windows) switches in user code. The appName is there for the common use-case to get the path in the context of the calling application. If I write a program an want to open some "images/splash.png" in the installation directory, I'd expect just the executable directory on Windows, but "/usr/share//" (or one of the overrides) on Linux. Then again, maybe opening files is outside the scope of your library. I could imagine it would become a full file system abstraction layer just to support correct file lookup with all these /usr/share, /usr/local/share, ~/.local/share and manual overrides. :p > The same directory for configs, data and cache locations is the > problem. I should mention it in documentation. Though for cache > directory it returns AppData/Local/cache the same way as Qt does. -- Marco
Re: Standardpaths library
Am Tue, 07 Apr 2015 11:58:58 + schrieb "Kagamin" : > On Monday, 6 April 2015 at 21:40:28 UTC, Marco Leise wrote: > > So if some configuration is "portable" and you want to create > > something > > really fine grained you could offer that directory as an > > alternative "roaming config dir" (returning null or > > "~/.config" on Linux). > > Though it's unusual for an average programmer to figure out the > correct usage of roaming profiles :) All they need to know is that Roaming must only contain small and machine independent files. (Why? Because in environments where users profiles are stored on servers, this is the part that gets synced with those servers. So e.g. a browser cache should not end up there, nor should configuration that makes sense only on the current workstation.) Frankly, I didn't know until FreeSlave's library brought it up. :D Anyways my idea of portable libraries is that they should not implement the least common denominator, but the full width of what is available or not across different systems and offer a common API for that, so you have the option to write well behaving applications while rewriting as little code as possible. Two examples: There is an "application menu" in OS X and in Gnome 3, but other desktops don't provide it. I would support it, so applications integrate nicely with these desktops and turn it into another main menu item or taskbar right-click menu on other desktops. POSIX doesn't close file handles for spawned child-processes by default, which is now considered a bad default by most. I think a portable library should try to provide the same functionality across systems and open files in "close on exec" mode by default on POSIX so it is similar to Windows. -- Marco
Re: Standardpaths library
Am Sun, 05 Apr 2015 12:39:04 + schrieb "FreeSlave" : > On Sunday, 5 April 2015 at 11:42:42 UTC, Marco Leise wrote: > >is this Windows? > > return executable_path > > That depends on what do you understand by data. The data I was referring to is the stuff that is installed with the program executable. Files that are usually the same across installations. > Are game's saves data too? Or content downloaded while playing > (server-specific assets or new levels). > In the past it was ok to write configs and > data to the same path where the game (or application) originally > installed. But starting with Vista or Windows 7 it's not the case > since Program Files folder become write-protected by default. I believe modern desktops offer enough granularity to cover each of those. For example if I was playing a game on Linux files would go here: /usr/share/[games/] - read-only data files. ~/.cache/ - downloaded archives, precompiled scripts, browser caches and other files that can be recreated or fetched again if they were to be deleted ~/.config/ - the user's personal configuration; may be overriding something in a system directory if desired ~/.local/share/ - pretty much a catch all for save games, user created content that goes beyond the scope of config files, highscores, highlighting schemes in an IDE, Steam, ... Or the other way around: data => /usr/share save games => ~/.local/share downloaded content => ~/.local/share (or ~/.cache) configs=> ~/.config Windows has the Local and Roaming directories, which serve similar but different purposes. E.g. anything machine specific or big must not be in Roaming. So if I generally asked for the config dir, I'd probably expect AppData/Local on Windows and ~/.config on Linux (because I might write a configuration that only works for this machine.) Roaming is interesting for users that have their profiles on servers and might switch to another workstation. So if some configuration is "portable" and you want to create something really fine grained you could offer that directory as an alternative "roaming config dir" (returning null or "~/.config" on Linux). In any case there will be multiple results for some directories (/usr/share, /usr/local/share) and also several standard paths mapping to the same physical directory (user data and user cache both map to AppData/Local on Windows). The user needs to be made aware of this so (s)he doesn't overwrite files in one standard path with files in a supposedly different one. So much for my thoughts on standard paths extreme edition. ;) -- Marco
Re: Standardpaths library
Am Sun, 05 Apr 2015 12:31:23 + schrieb "FreeSlave" : > On Sunday, 5 April 2015 at 11:42:42 UTC, Marco Leise wrote: > > On another note when I ran your 'printdirs' it didn't list a > > user Fonts or Applications directory. The Applications > > directory is ok, but I do have a ~/.fonts/ directory and > > /etc/fonts/fonts.conf says: > > > > ~/.fonts > > Fonts in ~/.fonts are listed in LibreOffice. It seems like you > > do parse /etc/fonts/fonts.conf. Maybe there is a bug in the > > parser? > > > > The whole Applications thing doesn't make much sense on Linux, > > right? Is that a directory where applications are installed to > > including their assets? > > Probably you don't have local > $XDG_CONFIG_DIR/fontconfig/fonts.conf file. > > I've opened issue > https://github.com/MyLittleRobo/standardpaths/issues/8 > > About Applications: on my Windows 7 it returns > C:/Users/Username/Application Data/Microsoft/Windows/Start > Menu/ProgramsC:/ProgramData/Microsoft/Windows/Start Menu/Programs > where .lnk files are stored (I believe these are used in the > start menu). Since freedesktop systems use .desktop files it > would be sane to return paths which contain them > (~/.local/share/applications, /usr/local/share/applications and > /usr/share/applications). I just have not implemented it yet. > Though not sure it the whole thing can be useful, since things > are not the same on Windows and freedesktop: Windows uses > directories to make menu hierarchy, while freedesktop for the > same purpose use Categories field in .desktop files. Also .lnk > and .desktop are different things themselves. You are right, the two are very different. One has to write OS specific code to use them. The funny thing is, D as a systems programming language could actually be used by someone to write a Linux package manager or Windows installer. :p Do as you see fit. Qt as an inspiration is a good thing I believe. Some classes I had a look at were intuitive and well thought out. -- Marco
Re: Standardpaths library
Am Sun, 05 Apr 2015 09:08:12 +
schrieb "FreeSlave" :
> I wrote small library for getting standard paths (like Pictures,
> Music)
> Here's dub package http://code.dlang.org/packages/standardpaths
> And github repo https://github.com/MyLittleRobo/standardpaths
>
> You can see open issues on github. Please, participate in
> discussions if you're interested. The biggest problem now is OS X
> support https://github.com/MyLittleRobo/standardpaths/issues/4
>
> You can generate documentation with dub build --build=docs.
This is indeed useful. I once had a specific use case where I
needed the default location for where the assets of my
application is stored. On Windows applications come with an
installer and install the assets right into the installation
directory (usually current directory). On Linux applications
store their data in /usr/share//. So with a
compile-time only template I accessed these as:
string asset1 = dirs!"myapp".staticData ~ "/image.png";
Now I realize this is a bit simplistic for several reasons:
* On Windows, one might still want to create a subdirectory to
separate assets from program code, e.g. "data".
* On Linux, as your library shows, data directories are
layered like this:
~/.local/share, /usr/local/share, /usr/share
* The _actual_ data directory is often passed into the
configure script on Linux and no general algorithm could
guess it.
* An application name would typically become lower-case on
Linux while keeping its casing on Windows in e.g.
AppData/.
Unsure if that's it, but that could make data file lookup for
application "SomeTool" go like this:
was the data directory explicitly overridden (e.g.
through ./configure or by the programmer)?
return overridden_dir; // note: relative directories expand
as based on executable directory
is this Linux?
for (dir in
["${XDG_DATA_HOME}"/sometool, /usr/local/share/sometool, /usr/share/sometool])
is the file in dir ?
return dir
is this Windows?
return executable_path
This goes a bit into heuristics and best practices, so it
probably doesn't fit with your library that provides
clearly defined standard paths from the desktop environment.
Aside from that I think it is a common enough use case to
lookup assets that ship with your program in the "typical"
installation directories.
On another note when I ran your 'printdirs' it didn't list a
user Fonts or Applications directory. The Applications
directory is ok, but I do have a ~/.fonts/ directory and
/etc/fonts/fonts.conf says:
~/.fonts
Fonts in ~/.fonts are listed in LibreOffice. It seems like you
do parse /etc/fonts/fonts.conf. Maybe there is a bug in the
parser?
The whole Applications thing doesn't make much sense on Linux,
right? Is that a directory where applications are installed to
including their assets?
--
Marco
Re: Devisualization.Image
Am Wed, 19 Nov 2014 00:35:09 +0100 schrieb Marco Leise : > > I have working PNG export in my image processing package, part of dlib: > > https://github.com/gecko0307/dlib > > There're also importers for BMP and TGA. > > I also just wrote a TGA importer and found RLE and 16-bit in > particular badly supported on Linux. E.g. eye-of-gnome > (Gnome's picture viewer) doesn't correctly restore the last > pixel's color in an RLE image and most software doesn't handle > 16-bit at all. Add to that, that the specs are not very > specific as to what allowed values are and you get all > sorts of funny implementations quirks. For example: - the original Targa paint software TIPS used to store a palette even for TrueColor images, meaning you should actually calculate the data offset as header + id + palette even for 24- and 32-bit images. - the number of channels for a TrueColor TGA is always 3, plus as many attribute bits as given in the descriptor. If you only support 8-bits per color channel, you should check if bitsPerPixel is >= 24. Otherwise, bitsPerPixel == 16 would be interpreted as a two color channel image instead of three channels à 5-bits. And bitsPerPixel == 15 would become a one channel image. :p The 15-/16-bit variants were used quite a bit in the early days of 3D games, like in the menu graphics for "Trespasser". They are the same except that 16-bit allows for an additional attribute bit that could be used for a transparency mask. -- Marco
Re: Devisualization.Image
Am Tue, 18 Nov 2014 12:37:54 +0300 schrieb Timur Gafarov : > 15.11.2014 07:48, Rikki Cattermole пишет: > > To further Devisualization, I have got the start of an image library. > > It should be fairly interface complete now. > > > > For this I really could use help from anyone with experience with PNG > > especially with Adam7 interlacing and color correction such as gamma. > > Currently missing an exporter. Only imports. Does not import grayscale > > or palleted. > > > > Goal: to act as the image representation within memory (including colors). > > Usage: > > Devisualization.Window uses it for window icon's (untested for x11 can > > somebody confirm this does indeed work?). > > Devisualization.Font heavily uses it for glyphs and the output of its > > rasterization. > > > > [0] https://github.com/Devisualization/image > > I have working PNG export in my image processing package, part of dlib: > https://github.com/gecko0307/dlib > There're also importers for BMP and TGA. I also just wrote a TGA importer and found RLE and 16-bit in particular badly supported on Linux. E.g. eye-of-gnome (Gnome's picture viewer) doesn't correctly restore the last pixel's color in an RLE image and most software doesn't handle 16-bit at all. Add to that, that the specs are not very specific as to what allowed values are and you get all sorts of funny implementations quirks. -- Marco
Re: Atmosphere GM - Statistical package
Am Fri, 24 Oct 2014 17:54:26 + schrieb "Ilya Yaroshenko" : > Hello! > > link http://9il.github.io/atmosphere_gm > > You are welcome to suggest required algorithms! > > Best Regards, > Ilya Your English is a bit confusing :) What can I use this package for? Let's say I measure some value over a period of time where the distance between the samples is between 100 to 500 ms and there is an error of +/- 2%. Does Atmosphere GM provide algorithms to filter the error so I can derive from the graph without huge error spikes? Or is it more for card games and dice rolls? -- Marco
Re: Mono-D v2.5 - dub buildTypes
Am Thu, 16 Oct 2014 23:32:20 + schrieb "Alexander Bothe" : > Hi everyone, > > just gave the second drop down box in Xamarin Studio a use: > Selection of build types for dub projects. > > > Furthermore, please don't rage silently somewhere - tell me about > issues with Mono-D on github or in #d.mono-d on freenode! > http://wiki.dlang.org/Mono-D > https://github.com/aBothe/Mono-D/issues > > > > Cheers & thanks to everyone, > Alex I see what you are doing there. Furthermore, Carthage must me destroyed! -- Marco
Re: Digger 1.0
Am Mon, 22 Sep 2014 18:59:12 + schrieb "Vladimir Panteleev" : > On Monday, 22 September 2014 at 17:28:50 UTC, ketmar via > Digitalmars-d-announce wrote: > > On Mon, 22 Sep 2014 15:24:55 +0200 > > simendsjo via Digitalmars-d-announce > > wrote: > > > >> My guess is the average for developers is ~8GB. 2GB RAM is > >> really not > >> enough for pretty much anything these days - the browser alone > >> easily > >> chews 3-4GB on moderate use. I recommend you just upgrade your > >> computer (or compile on a better iron). > > are your primary language is java? i'm asking just out of > > curiousity. > > That was uncalled for. > > Firefox requires 4GB of memory to build. > Chromium requires 8GB of memory to build. > Android requires 16GB of memory to build. I've been compiling Firefox and Chromium with 4GB of RAM + 4GB of swap on Linux 64-bit in the past. I don't know if I had to close any applications for that. > I think 4GB for a modern programming language's implementation is > not an unreasonable requirement, even if it could be brought down > in the future. Especially considering that you can't even buy a > new laptop today with less than 4GB of RAM, and 3GB is becoming > the norm for smartphones. So why would Apple be able to get away with 1GB on its just released iPhone 6? Maybe 1048576 kilobytes is enough for everyone? -- Marco
Re: New Beta: 2.066.1-rc1 is Now Available
Am Fri, 12 Sep 2014 03:16:52 +0900 schrieb Andrew Edwards : > Clear cache and try again. Thx, "try again now" was enough. -- Marco
Re: New Beta: 2.066.1-rc1 is Now Available
P.S. Is this supposed to contain all 2.066 regression fixes pulled into master up to this point? As far as I can tell at least https://github.com/D-Programming-Language/dmd/pull/3961 from 4 days ago didn't get cherry picked yet. Or are these RCs more of a preliminary version? -- Marco
Re: New Beta: 2.066.1-rc1 is Now Available
Am Fri, 12 Sep 2014 01:51:33 +0900 schrieb Andrew Edwards : > DMD 2.066.1-rc1 is now available for testing. Visit > http://wiki.dlang.org/Beta_Testing to download. Please submit all bugs > to bugzilla at https://issues.dlang.org/. > > Happy testing I followed the first LINUX link on http://wiki.dlang.org/DMD_v2.066.1-rc1 and got this: http://downloads.dlang.org/pre-releases/2014/dmd.2.066.1-rc1.linux.zip 404 Not Found Code: NoSuchKey Message: The specified key does not exist. Key: pre-releases/2014/dmd.2.066.1-rc1.linux.zip RequestId: 041CB7ED85E79D22 HostId: 77PqxexMbECV1cBnYfWlCZhxrKtBgH2VhFhP8v2tlG6kpALNX3JGPYXbzhmYIx7w -- Marco
Re: 438-byte "Hello, world" Win32 EXE in D
Am Tue, 09 Sep 2014 10:20:43 + schrieb "Don" : > On Monday, 8 September 2014 at 08:18:32 UTC, Ola Fosheim Grøstad > wrote: > > On Monday, 8 September 2014 at 08:08:23 UTC, Kagamin wrote: > >> But that downloaded file is bloatware, because it has to > >> implement functionality, which is not provided by the system. > >> That tiny pe file doesn't download anything, it's completely > >> done by the system. > > > > Yeah… > > > > http://stackoverflow.com/questions/284797/hello-world-in-less-than-20-bytes > > My personal best -- > > At my first job, a customer once made a request for a very simple > DOS utility. They did mention that they didn't have much disk > space on their machine, so they asked me to try to make the > program small. > That was a once-in-a-lifetime opportunity. Naturally, I wrote it > in asm. > The final executable size was 15 bytes. > The customer loved it. Vladimir: Good job! Don: Nice story. What did it do? During my time at a vocation school I wrote some stuff like a tiny Windows media player with some of the ASM in the DOS/PE header area. And an animated GIF player in ASM as a .com executable with the GIF included in it. (Easy since GIF algorithms are 16-bit and they use 8-bit color palettes) -- Marco
Re: Dlang on Gentoo (update)
Added ebuild: * dub 0.9.21 Updated ebuilds: * GDC based on GCC 4.8.3 and DMD FE 2.065 * DMD 2.066.0 and tools I wonder if I can switch to linking with libphobos2.so now by default for dmd? Any outstanding issues/objections? -- Marco
Dlang on Gentoo (update)
I moved Dlang related ebuild to a new repository under the Gentoo organization on GitHub a while ago. It is available through "layman" as "dlang". (Not to be confused with the outdated "d" overlay there.) Information in the Gentoo Wiki: https://wiki.gentoo.org/wiki/Dlang Link to the repository on GitHub: https://github.com/gentoo/dlang/ -- Marco
Re: legacy code retreat's triva game : the D version
Am Sun, 22 Dec 2013 09:19:48 + schrieb "Chris Cain" : > On Sunday, 22 December 2013 at 08:06:30 UTC, Marco Leise wrote: > > Can you elaborate a bit? How do you know that the Java LCG > > can produce every 32-bit integer once? If that's true then > > the problem with the Java code was something different and I > > was just biased, because I was already expecting the code to > > fail before the fact. (Expectations can do strange things to > > your perception.) > > If I may, > > http://en.wikipedia.org/wiki/Linear_congruential_generator > > Definition of an LCG: > ``` > Xnext = (a * Xprev + c) % m > ``` > > An LCG is said to have a "full period" if the length of the > period is m. If the period is m, we know the LCG must produce > every number between 0 and m because if there was even one > repeated number then the generator as defined above would repeat > the entire sequence up to that point and, thus, the period would > not be m, which is a contradiction. > > According to the Hull-Dobell Theorem, an LCG will have a full > period iff: > 1. `c` and `m` are relatively prime. > For Java, c = 11 and m = 2^48 > This condition applies. > 2. `(a - 1)` is divisible by all prime factors of m` > For Java, a = 25214903917 and thus a-1 is even which means the > prime factors of m (just 2) do divide it. > This condition applies. > 3. `a - 1` is a multiple of 4 if `m` is a multiple of 4. > For Java, m is a multiple of 4. > `(a - 1)/4` is 6303725979, so it's also a multiple of 4. > This condition applies as well. > > Since Java's LCG has a full period over 2^48, we know that taking > the top 32 bits (which is what Java does to get "better" > randomness) would also all be represented. Am Sun, 22 Dec 2013 13:09:51 +0100 schrieb Timon Gehr : > On 12/22/2013 09:06 AM, Marco Leise wrote: > > Am Sun, 22 Dec 2013 02:12:51 +0100 > > schrieb Timon Gehr : > > > >> On 12/22/2013 02:09 AM, Timon Gehr wrote: > >>>> > >>>> The morale is that "uniform" random numbers doesn't imply that > >>>> every value in the range will eventually be generated once! > >>>> > >>> > >>> Yes it does. (The probability that some value is never generated is 0.) > >>> The actual morale is that random number generators do not generate true > >>> randomness, and poor random number generators may generate sequences > >>> that do not look remotely random. > >> > >> 'pseudo random number generators' would be a more accurate term. > > > > Can you elaborate a bit? > > The probability that a certain number does not occur in one round is > (n-1)/n. > > ((n-1)/n)^k goes to 0 rather fast as k goes to infinity. > > In fact, the expected number of trials until all numbers are covered is > ~ n log n, and the probability that the process runs significantly > longer is very small. > > See also: http://en.wikipedia.org/wiki/Coupon_collector%27s_problem > > > > How do you know that the Java LCG can produce every 32-bit integer once? > > Typically constants are chosen such that this holds, but your code would > require something stronger to fail, namely, that a certain congruence > class does not occur. Typically pseudo random number generators are > chosen such that the generated sequences look close to true randomness. > If such a simple process can be used to reliably distinguish true > randomness and the pseudo random number generator, then the pseudo > random number generator is not very good. Thank you two for explaining LCGs to me. That's good information for reasoning about code. Every good (full period) LCG is a specific permutation of the numbers [0..m). The next time I wonder how I can iterate in random order over a list of length n^2, I know what I'll use ;) -- Marco
Re: legacy code retreat's triva game : the D version
Am Sun, 22 Dec 2013 02:12:51 +0100 schrieb Timon Gehr : > On 12/22/2013 02:09 AM, Timon Gehr wrote: > >> > >> The morale is that "uniform" random numbers doesn't imply that > >> every value in the range will eventually be generated once! > >> > > > > Yes it does. (The probability that some value is never generated is 0.) > > The actual morale is that random number generators do not generate true > > randomness, and poor random number generators may generate sequences > > that do not look remotely random. > > 'pseudo random number generators' would be a more accurate term. Can you elaborate a bit? How do you know that the Java LCG can produce every 32-bit integer once? If that's true then the problem with the Java code was something different and I was just biased, because I was already expecting the code to fail before the fact. (Expectations can do strange things to your perception.) -- Marco
Re: legacy code retreat's triva game : the D version
Am Fri, 20 Dec 2013 15:53:08 +0100
schrieb "marcpmichel" :
>
> I participated in the "global day of code retreat 2013", and we
> had to do refactoring on a very ugly piece of code which was
> available on many languages.
> But there was no D version, so I made one (based on the java
> version) and pull-requested it.
>
> Here is the ugly thing :
> https://github.com/jbrains/trivia/tree/master/d
>
> EOT
bool notAWinner;
do {
game.roll(rand.front() % 5 + 1);
rand.popFront();
if (rand.front() % 9 == 7) {// <-- WARNING! WARNING!
notAWinner = game.wrongAnswer();
} else {
notAWinner = game.wasCorrectlyAnswered();
}
rand.popFront();
} while (notAWinner);
This kind of code is a dangerous gamble.
This is a story about my student time: I once sat in a Java
class and one of the students had an issue with their code not
outputting anything and not quitting either. When the teacher
came around, we found only one obvious point for an infinite
loop could occur and it looked like this:
Random rng = new Random();
int count = 0;
// Visit all items once
while (count < list.size()) {
bool found = false;
while (!found) {
int idx = rng.nextInt() % list.size();
if (list[idx].visited == false) {
list[idx].visited = true;
found = true;
count++;
}
}
}
[I don't remember the exact lines, but this is the gist of it.]
The teacher himself wrote this code and presented it to the
class as a simple way to iterate over a list in random order
which was part of todays programming task.
It didn't cause issues for any of the other students, but on
this particular computer the random seed that the Random ctor
chose caused a degenerate case where it never hit any of the 3
remaining indexes of the list.
The morale is that "uniform" random numbers doesn't imply that
every value in the range will eventually be generated once!
--
Marco
Gentoo overlay online
I have completed what I wanted to achieve for the new overlay and have uploaded it here: https://github.com/gentoo-dlang Features include: * Installation of DMD, GDC and LDC in parallel * Customizable »CFLAGS« for each D compiler * GDC is integrated with GCC for the best compatibility * Slotted installation of previous D compiler versions * Shared library support when using DMD * Easily compile debug builds with DMD and release builds with LDC/GDC even when they depend on libraries like GtkD. (Note: This depends on availability of libraries in the repository.) Try e.g.: ACCEPT_KEYWORDS="~amd64" USE="ldc2-0.12.0" emerge GtkD -- Marco
Re: The D in Novosibirsk State University
Am Mon, 11 Nov 2013 00:35:09 +0100 schrieb "Dicebot" : > On Sunday, 10 November 2013 at 23:19:22 UTC, Froglegs wrote: > > Slides are in English, do most Russian programmers speak > > English? > > No. However, most really interested in programming technology do. > Pretty much as in any other part of the world. Reminds me how I ended up in a 100% Russian bug tracker once, when I wanted to report a bug for a big, world-wide online game with localizations into 8 or more languages. In the end I used Google translate. (They do offer technical support in English, but I didn't want to deal with 1st level support and have mails going forth and back just to report an obvious bug. :p) -- Marco
Re: dmd 2.064.2
Am Tue, 05 Nov 2013 23:24:02 +0100 schrieb "Orvid King" : > On Tuesday, 5 November 2013 at 22:08:48 UTC, Walter Bright wrote: > > Ok, this is it: > > > > http://ftp.digitalmars.com/dmd_2.064.2-0_amd64.deb > > http://ftp.digitalmars.com/dmd-2.064.2-0.fedora.i386.rpm > > http://ftp.digitalmars.com/dmd-2.064.2-0.fedora.x86_64.rpm > > http://ftp.digitalmars.com/dmd_2.064.2-0_i386.deb > > http://ftp.digitalmars.com/dmd-2.064.2-0.openSUSE.i386.rpm > > http://ftp.digitalmars.com/dmd-2.064.2-0.openSUSE.x86_64.rpm > > http://ftp.digitalmars.com/dmd-2.064.2.exe > > http://ftp.digitalmars.com/dmd.2.064.2.zip > > http://ftp.digitalmars.com/dmd.2.064.2.dmg > > http://ftp.digitalmars.com/libphobos2-64_2.064.2-0_amd64.deb > > http://ftp.digitalmars.com/libphobos2-64_2.064.2-0_i386.deb > > Dear world: DO NOT use the windows installer if you have ANYTHING > other than the default installed files in your previous install > folder, because it will delete EVERYTHING. (and this isn't an > understatement, my C:/D folder is now entirely empty but for a > single git index which was locked by my IDE), this means I have > now lost my local checkouts of the D repos, my git-head dmd > install location, as well as my auto-build-install-test scripts, > and my local copy of my JSON work. Thankfully my JSON work wasn't > even the latest copy anyways. Deleting everything IS NOT > uninstalling. Stunned silence... You seem to have gotten away with only few losses. Thanks for sharing. I can only imagine what that would have done to someone who has no backups or online repositories for their code. I have to say though that I'm sometimes annoyed by accurate uninstallers that keep a directory because of a log file or modified configuration. I doesn't hurt to ask for a recursive deletion of the install directory. -- Marco
Re: LDC 0.12.0 has been released
Am Sat, 26 Oct 2013 18:34:55 +0200 schrieb "Kai Nacke" : > On Saturday, 26 October 2013 at 02:21:37 UTC, deadalnix wrote: > > The explanation is quite simple. LLVM understand C and C++ > > runtime. It doesn't understand D runtime (LDC is doing some > > work in that regard, but it is still limited). So you see a > > difference between C and C++ as some optimization will be done > > in C/C++ (for instance heap to stack promotion) when it won't > > be done in D. > > While this is true in general I don't think that it explains the > difference in the mentioned benchmark. A PRNG does not use too > much of the C runtime. And at a closer look, the author replaced calls to the C PRNG with a simple XorShift engine included in the source code to level the field between the languages. 60% of the runtime is spent checking for collisions between rectangles. > I believe that the IR generated by LDC could be improved and that > the difference in speed is caused by this. If the author had applied correct rounding both Clang and LDC would have had 100%. I assume the 1 ms (< 0.5%) difference stems from the startup time of D or maybe for C he timed the code internally and for D he used "time"; but it is really insignificant in relative numbers. > Regards > Kai -- Marco
Re: LDC 0.12.0 has been released
Am Sat, 26 Oct 2013 04:21:26 +0200 schrieb "deadalnix" : > On Friday, 25 October 2013 at 07:50:36 UTC, ilya-stromberg wrote: > > I think it's answer for your question. For example, Clang (LLVM > > C) is fastest, LDC (LLVM D) has 2-nd place, but LLVM-GHC (LLVM > > Haskell) ~ 2 times slower. > > The explanation is quite simple. LLVM understand C and C++ > runtime. It doesn't understand D runtime (LDC is doing some work > in that regard, but it is still limited). So you see a difference > between C and C++ as some optimization will be done in C/C++ (for > instance heap to stack promotion) when it won't be done in D. Some observations: The posted level creation benchmark contains no dynamic memory allocation and no runtime calls in the critical loop. As such you won't lose performance due to missing compiler intrinsics or the GC here and the runtime difference between the D and the C version is minimal (< 0.5% in fact). Yet I've altered it a bit to gain another 10% for the D version (without changing the algorithm): https://github.com/mleise/levgen-benchmarks/blob/master/D.d Now D is the fastest again and we are all happy :) -- Marco
Re: dunit 0.7.0 released
Am Mon, 21 Oct 2013 21:47:22 +0200 schrieb "ilya-stromberg" : > On Monday, 21 October 2013 at 19:02:45 UTC, Gary Willoughby wrote: > > On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis > > wrote: > >> On Monday, October 21, 2013 13:48:16 Dicebot wrote: > >>> On Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg > >>> wrote: > >>> > Guys, we have at least 5 (!) different unit test projects! > >>> > Can you cooperate your efforts to create one, but wonderful? > >>> > >>> ...and add it to Phobos review queue ;) > >> > >> I confess that I don't understand why anyone is creating any > >> unit test > >> projects for D, and I'd likely vote against any attempt to add > >> such a thing to > >> Phobos. D has built in unit testing functionality, and it > >> works great. Maybe > >> some additional assert-like functions could be useful (similar > >> to assertThrown > >> or assertNotThrown), but we really don't need much beyond what > >> the language > >> provides. > >> > >> - Jonathan m Davis > > > > Seriously though, unit testing is a major tool that all > > languages should have access to. The built-in stuff is adequate > > for very simple testing on simple types such as asserting > > something is true, etc. But when you're writing tests for a > > class that need dependencies then the built-in stuff won't cut > > it. The framework i'm currently working allows for mocking of > > dependencies which in itself is a big deal. Replacing > > dependencies with 'fake' stubs is something which makes unit > > tests a lot clearer and less complicated. It also means i can > > override the return value of any method at runtime to pretend > > returned data is real there too. > > > > I've also extended the assert mechanism in the D runtime to > > create better errors. Instead of just getting: > > > > core.exception.AssertError@file(57): Assertion failure > > > > You now get: > > > > > > +-- > > | Failed asserting equal > > > > +-- > > | File: file.d > > | Line: 57 > > > > +-- > > | ✓ Expected value: (int[]) [1, 2, 3, 4, 5] > > | ✗ Actual value: (int[]) [1, 2, 4, 5] > > > > This is way more clear exactly what and where failed. This is > > essential when unit testing using structs, arrays, classes, > > etc.. as you need this information to truly narrow down exactly > > what failed. Extending stuff like this also helps with overall > > project reporting, writing to a file or drawing pretty graphs > > etc.. > > +1 We had a discussion about verbose contract assertion failures a while ago. I did something similar, with an API like that: ensure!"=="(x, [1, 2, 3, 4, 5], "x isn't the ordered list of integers from 1 to 5"); It has its shortcomings though, especially when you need x to be one of a set of values or a power of 2 etc... Btw: Is ✓ and ✗ supported on Windows Vista and XP ? -- Marco
Re: Article: D Exceptions and C Callbacks
Am Thu, 08 Aug 2013 11:33:28 -0700 schrieb Walter Bright : > On 8/8/2013 6:48 AM, Marco Leise wrote: > > I recently proposed using nothrow in GtkD as well. This > > article was kind of a coincidence now. Yes, it is a real issue > > with binding to C code unfortunately. Especially on amd64 with > > GCC compiled stuff since it omits the stack frame that DMD > > requires to unwind the stack resulting in immediate crashes > > (druntime HLTs the execution) as soon as you throw an > > exception from a callback. > > > I agree that C callbacks called from C code should be nothrow. > > At a minimum, the C code that calls the callback is not going to be expecting > an > exit via exception, and so may leave things in an indeterminate state. Just for completeness: In a recent discussion on IRC I learned that vendor specific attributes exist for C compilers to mark functions as "ok to use with exceptions". -- Marco
Re: Article: D Exceptions and C Callbacks
Am Wed, 07 Aug 2013 12:08:44 +0200 schrieb "Mike Parker" : > On Wednesday, 7 August 2013 at 07:10:21 UTC, Johannes Pfau wrote: > > Am Wed, 07 Aug 2013 03:33:28 +0200 > > schrieb "Mike Parker" : > > > >> > >> > Maybe modifying glfwSetWindowCloseCallback and similar > >> > functions to only accept nothrow functions is a good idea? > >> > >> I'm not sure what you mean. That's a C function, which means > >> it doesn't have any concept of D's nothrow. And, actually, the > >> callbacks should be declared as nothrow anyway. Like: > > > > He meant declaring glfwSetWindowCloseCallback like this: > > alias extern(C) nothrow void function() callback; > > extern(C) void glfwSetWindowCloseCallback(callback cb); > > > > http://dpaste.dzfl.pl/0f23146f > > There's no difference in the generated code, but this way the > > compiler > > will complain if you pass a non-nothrow function to > > glfwSetWindowCloseCallback. > > Yeah, I see. I misunderstood. I agree that the callbacks should > be declared as nothrow. In Derelict, all the C function pointers > are declared as such. Apparently, I overlooked adding it to the > callbacks. I recently proposed using nothrow in GtkD as well. This article was kind of a coincidence now. Yes, it is a real issue with binding to C code unfortunately. Especially on amd64 with GCC compiled stuff since it omits the stack frame that DMD requires to unwind the stack resulting in immediate crashes (druntime HLTs the execution) as soon as you throw an exception from a callback. -- Marco
Re: My first email to Walter, ever
Am Sat, 06 Jul 2013 20:02:16 -0700 schrieb Andrei Alexandrescu : > > * The venerable typeof check > > * For a class, enumerate all of its members, and figure out their > > attributes (protection level, static or not, type...) check > > * For a module/namespace, enumerate all of its symbols and figure > > out their attributes. check > > * For a function, figure out its return type and the type of each of > > its formal arguments. check > > * Figure out if a certain function exists (that should be easy if > > you have the tools above). That will be useful to decide, for > > example, if an iterator supports random access. check > > * And if you really want to go meta, define metacode that can take > > an AST node as a parameter and can visit the AST and figure out what > > each node is. That would allow things such as loop fusion and other > > advanced stuff. But for now, let's leave those aside. che... oh _that_ topic. I think you have done some really good team work there. And I may add Walter's good intuition to make the front end reusable for GDC and LDC. Btw, you didn't want to start a new discussion about AST manipulation through the back door, did you? ;) -- Marco
Re: Announcing bottom-up-build - a build system for C/C++/D
Am Fri, 28 Jun 2013 07:03:27 + (UTC) schrieb Graham St Jack : > On Fri, 28 Jun 2013 08:28:03 +0200, Marco Leise wrote: > > > How does this build tool handle projects with multiple executables ? For > > example the util-linux package contains dozens of utilities or a project > > might have a CLI and a GUI version. Or there might be slight alterations > > like setting a version or debug flag: -debug=threading -version=demo > > > A central theme in bottom-up-build is to produce any number of build > artifacts: libraries, executables, tests, etc. > > Different versions are catered for with: > bub-config --mode= > > The mode is described in your configuration file, and results in a build > directory set up specifically for that mode. > > If you want multiple modes, use multiple build directories. > > Take a look in example/bub.cfg. Sounds good! -- Marco
Re: Announcing bottom-up-build - a build system for C/C++/D
How does this build tool handle projects with multiple executables ? For example the util-linux package contains dozens of utilities or a project might have a CLI and a GUI version. Or there might be slight alterations like setting a version or debug flag: -debug=threading -version=demo -- Marco
Re: Announcing bottom-up-build - a build system for C/C++/D
Am Thu, 27 Jun 2013 10:26:01 +0200 schrieb "eles" : > On Thursday, 27 June 2013 at 07:32:32 UTC, eles wrote: > > CTRL-Z works for me. I think it expects input. > > Ignore it. It just suspends it. You might want to check how many programs you thought to have "killed" like this so far in your running session. It might eat up your RAM. -- Marco
Re: [Phoronix] D Language Still Showing Promise, Advancements
Am Fri, 21 Jun 2013 14:11:17 +0200 schrieb "qznc" : > The only-thread-local-garbage-collection of Rust is quite > interesting in my opinion. Since many-cores (e.g. Xeon Phi) are > coming, a stop-the-world garbage collector might become > unacceptable. If this is a good solution will be seen (maybe). I > certainly do not want D to adopt this experimental feature. Let > them do the research. ;) Rust is called rust exactly because they don't use bleeding edge features in the language. -- Marco
Re: [Phoronix] D Language Still Showing Promise, Advancements
Am Thu, 20 Jun 2013 22:53:13 +0200 schrieb "Adam D. Ruppe" : > On Thursday, 20 June 2013 at 20:47:19 UTC, Michael wrote: > > Also 3 types of pointers scares me. > > This actually doesn't scare me because it is kinda useful for > certain situations. However, I don't think it needs to be built > into the language because library types can do the same kind of > thing. Yes, once you know about UniquePtr, NotNullable, RefCounted and such, 3 pointer types isn't scary are all. But Rust pointers don't map 100% to library types. They have compiler support which removes the syntactical noise of templates and makes them behave more natural. For example in D everything becomes a struct once you add functionality around it and creates corner cases that aren't supported. I wonder how many D programmers actually use Phobos' library pointer types. -- Marco
Re: DConf 2013 Day 3 Talk 4: LDC by David Nadlinger
Am Mon, 17 Jun 2013 22:03:09 +0200 schrieb "Joseph Rushton Wakeling" : > On Monday, 17 June 2013 at 15:19:27 UTC, Andrej Mitrovic wrote: > > There seems to be some audio glitching every couple of seconds > > (at the > > beginning). I've noticed this in other videos as well. It's > > mostly > > minimal though, not much harm done. > > https://www.youtube.com/watch?v=m6jsXQm5IrM#t=106s :-) "What's this wire doing here? It's dangerous!" *pulls* -- Marco
Re: DConf 2013 Day 2 Talk 6: Higgs, an experimental JIT compiler written in D by Maxime Chevalier-Boisvert
Am Sun, 9 Jun 2013 01:53:23 +0200 schrieb Andrej Mitrovic : > On 6/9/13, bearophile wrote: > > The size of "byte" is easy, it's 1 byte, but if you ask me a byte > > is unsigned. I have learnt to be careful with byte/ubyte in D > > You, me, and Don, and probably others as well. It's easy to forget > though that bytes are signed. I found the integer names and sizes intuitive from the beginning! And there was no confusion for me ever. It starts with a 'u', it is unsigned, signed otherwise. And the sizes are common in other programming languages, too. long being the only confusing one, but since int is already the 32-bit version it should be clear that it must be 64-bit. Though I can understand people who think of a byte as an unsigned type, I was much more confused by C's signed chars. Oh really? A character can be negative? Is this programming or sociology? -- Marco
Re: simpledisplay.d now works on as 64 bit on X
Am Thu, 06 Jun 2013 01:27:58 +0200 schrieb "Adam D. Ruppe" : > cool... I found one on dsource but not github yet. The dsource > one transforms the XML but doesn't seem to implement all needed > functions. Shouldn't be hard to complete anyway though. Does that mirror XCB (independent of XCB, implements RPC) or add a wrapper layer around the C bindings? Because it would be kinda cool to have a real X sever binding in D and not just D bindings to the C bindings to the X server. :p -- Marco
Re: dmd 2.063 released with 260 bugfixes and enhancements
Am Thu, 06 Jun 2013 14:44:57 +0200 schrieb "Dicebot" : > On Thursday, 6 June 2013 at 10:50:30 UTC, Marco Leise wrote: > > Yes please, this is holding me back from updating the Gentoo > > package for dmd 2.063. (Unless I want to add that missing file > > as a patch.) > > Why not use git tag instead? The license doesn't allow redistribution of dmd, probably in order to have a download statistic. I don't want to circumvent that by creating my own .zip file or using an auto-generated archive. (Although I wouldn't mind getting rid of the pre-built binaries. :) ) -- Marco
Re: dmd 2.063 released with 260 bugfixes and enhancements
Am Thu, 06 Jun 2013 01:53:35 +0200 schrieb "Joakim" : > On Thursday, 30 May 2013 at 15:16:28 UTC, Andrei Alexandrescu > wrote: > > Hello, > > > > > > We are pleased to announce that dmd 2.063, the reference > > compiler of the D programming language, is now available for > > download for OSX, Windows, and a variety of Unixen: > > > > http://dlang.org/download.html > I just tried to compile dmd and it failed because the header file > irstate.h was missing. After retrieving it from git, the compile > went through, so it's probably the only file missing. I think > the zip needs to be updated. Yes please, this is holding me back from updating the Gentoo package for dmd 2.063. (Unless I want to add that missing file as a patch.) -- Marco
Re: simpledisplay.d now works on as 64 bit on X
Am Mon, 03 Jun 2013 01:02:12 +0200 schrieb "Adam D. Ruppe" : > On Sunday, 2 June 2013 at 21:53:56 UTC, Marco Leise wrote: > > Also eventually we should generate X bindings for all > > platforms (including Windows) from the XML definitions like > > XCB does. > > hmm I don't see an xml file for Xlib.h on my system, do you know > if there is supposed to be one? Though I'm pretty sure I have > most of it correct now, so I guess it isn't really necessary. XCB is generated from these XML files. They have nothing to do with Xlib. You can find the XML files here: http://cgit.freedesktop.org/xcb/proto/tree/src A generator tool for the target language (C in this case) parses them and creates include files from them that define all the remote procedure calls to the X Server. Internally those calls rely on a small generic set of functions that serializes the requests for transmission. What is needed are those generic functions written in D to establish the connection to the server and handling serialization, probably relying on std.socket. And then an XML parser that would write out .d files for the different modules of the X Server, like xrandr, xkb and so on, which are all available as separate XML files. This is what a RPC from glx.xml looks like: length In general they also include documentation tags which can be made available through DDoc automatically and displayed in IDEs like Mono-D already does. > I know everybody says Xlib is dying but I still use it. It is > simple enough and gets the job done without having to learn a > whole thing library... The big deal about it is asynchronous RCP especially over internet connections to X Servers, since with Xlib you have to way for each response before the next request. -- Marco
Re: simpledisplay.d now works on as 64 bit on X
I'm sorry to hear that you ran into the unsigned long problem. I noticed it in time and asked about it on the NG and was told that the difference is between Windows and Posix. One takes long as int32, the other as the native machine word size on 64-bit. We do the same stupid mistakes over and over. :p Also eventually we should generate X bindings for all platforms (including Windows) from the XML definitions like XCB does. That means a small core written in D that enables RPC to the X Server and a generator tool that creates functions, structs and DDoc from what is the XML files. (See /usr/share/xcb.) -- Marco
Re: dmd 2.063 released with 260 bugfixes and enhancements
Am Sun, 2 Jun 2013 07:40:27 +0200 schrieb Marco Leise : > For some reason I still cannot build dmd 2.063 from the zip > release. I mentioned it before the release on the beta and > internals mailing lists and maybe I'm just overlooking > something trivial, but when I run make I get: > > make: *** Keine Regel vorhanden, um das Target »irstate.h«, > benötigt von »irstate.o«, zu erstellen. Schluss. > > irstate.c is there, but irstate.h is missing. Is that like > last time when VERSION was missing? Ok, a pull request about that got merged. Let's see if it ends up in the .zip ;) -- Marco
Re: dmd 2.063 released with 260 bugfixes and enhancements
For some reason I still cannot build dmd 2.063 from the zip release. I mentioned it before the release on the beta and internals mailing lists and maybe I'm just overlooking something trivial, but when I run make I get: make: *** Keine Regel vorhanden, um das Target »irstate.h«, benötigt von »irstate.o«, zu erstellen. Schluss. irstate.c is there, but irstate.h is missing. Is that like last time when VERSION was missing? Also when you release hotfixes, please give the packages new names reflecting the full version. Package managers run into problems when the same URL has changed content. (e.g. aborting on failed checksum or not recognizing the update) -- Marco
Re: dmd 2.063 beta 5
Am Thu, 23 May 2013 13:06:44 -0400 schrieb Andrei Alexandrescu : > TDPL 8.4 discusses a raw/cooked model of construction. The same > principle should apply to const/immutable member construction: you get > to cook the field, but you can't taste it while raw. You are not making friends with Japanese D users. -- Marco
Re: D 2.062 release
Am Sun, 17 Feb 2013 17:02:20 -0800 schrieb Walter Bright : > http://digitalmars.com/d/download.html > > The dlang.org site isn't updated yet, but the downloads are there. I didn't read the whole thread now. Just reporting that the .zip package cannot be compiled due to missing VERSION file and that all the Phobos documentation now has empty "Examples:" sections (which I thought was already fixed). -- Marco
Re: Another opportunity for a major design win has presented itself
Am Thu, 07 Feb 2013 22:01:10 +0100 schrieb "Maxim Fomin" : > On Thursday, 7 February 2013 at 20:16:03 UTC, Walter Bright wrote: > > (a) get dynamic linking and loading to work > > Wasn't this realized before? By the way, last weeks there seems > to be increasing dynamic linking & loading buzz like it was not > an issue for ages. It is like the news: Things boil up, get covered for a week and disappear from public sight for a year. It has been an issue but people have probably arranged with static linking for now or not started to write a particular program in D. I remember I've tried to use structs with const members for a while for example, but arranged with making every field mutable. Now the topic showed up again after months. -- Marco
Re: Mono-D v0.4.2 Project building fixes
Am Thu, 15 Nov 2012 19:46:15 +0100 schrieb "alex" : > ..and the round goes on with a new bunch of fundamental fixes and > improvements :) > > http://mono-d.alexanderbothe.com > Issues go here https://github.com/aBothe/Mono-D/issues as usual, > > and the direct download for all Mac/Windows users that have > MonoDevelop 3.0.4.7: > http://mono-d.alexanderbothe.com/repo/MonoDevelop.D_0.4.2.0_MD3.0.4.7.mpack Installed! -- Marco
Re: Abstract Database Interface
Am Mon, 29 Oct 2012 08:50:19 +0100 schrieb "BLM768" : > > > I am working on similar project, named SQLd[1]. If you are > > interested, we can join forces and work togheder :) > > > > IRC Nick: Robik > > [1]: http://github.com/robik/SQLd > > That might be a good idea. I haven't done much for supporting > different databases, so getting more backend support would be > quite nice. What might work well is for me to just refactor my > code to sit on top of your existing database classes. So far, > I've really just been playing around, but if people show enough > interest, I'd like to play with the idea a while longer. :) I would use such an ORM library, too. It is definitely worth the effort. Currently I use something that just helps me a bit with fetching and updating entries in tables, but goes in the same direction of using only structs for table rows. Here are some database bindings from wiki4d that others have written: http://mleise.dnsd.info/#DatabaseDrivers Some are for D1 though. But now that you are team you may want to look at those bindings for some inspiration, so you don't get locked in on SQLite3 too much :p -- Marco
Re: Wiki page for C bindings / wrappers and reimplementations
Moved: http://mleise.dnsd.info/ (Note: This uses a free DynDNS service and the only free VPS host in the world. I expect occasional downtimes. ;-) ) -- Marco
Re: dmd 1.075 and 2.060 release
Am Fri, 03 Aug 2012 22:27:27 +0200 schrieb Jordi Sayol : > Al 03/08/12 21:22, En/na Jacob Carlborg ha escrit: > > On 2012-08-03 14:47, Jordi Sayol wrote: > > > >> https://github.com/D-Programming-Language/installer/blob/master/linux/dmd-completion > >> > >> It's based on "d-completion.sh" at https://github.com/Lutger/d_utils > > > > How about that. It should absolutely be bundled with the .zip. > > > > dmd deb and rpm packages include it, but this script do not works out of the > box. bash-completion package must be installed, and the script should be > placed in a specific folder. Yeah, and the Windows executables don't work out of the box on Linux either; Wine has to be installed. :-] Aside from that the "specific folder" likely depends on the used OS. (Gentoo uses /usr/share/bash-completion/ and symlinks in /etc/bash_completion.d/) But I think it is like "rdmd" and should be in the .zip file as "dmd.bashcomp" or similar, maybe in some sort of "extras" folder with a README, that explains what it is and where to put it. For the package I maintain I kept downloading the .zip and the .dep, just to integrate the bash completion file. It is good to know where I find the latest version from now on! -- Marco
Re: dmd 1.075 and 2.060 release
Am Thu, 02 Aug 2012 19:23:15 -0700 schrieb Walter Bright : > The beauty of git is you don't have to be "the maintainer". Anyone can submit > pull requests for any project. I did use GitHub fork & pull request before to fix a small bug in Phobos. What I mean is, the person who wrote the bash completion script will not know that the file moved over to GitHub and keep updating whatever is the original source. Don't you know by chance who is responsible for the Ubuntu packages? -- Marco
Re: dmd 1.075 and 2.060 release
Am Thu, 02 Aug 2012 14:54:01 -0700 schrieb Walter Bright : > On 8/2/2012 1:46 PM, Marco Leise wrote: > > By the way, it would be great if the bash completion script was also > > available in the .zip distribution. > > Please submit a pull request. > I'm not its maintainer. I'd like the person who updates it, to know and/or do the pull request. -- Marco
Re: dmd 1.075 and 2.060 release
Am Thu, 02 Aug 2012 12:18:37 -0700 schrieb Walter Bright : > Another big pile of bug fixes. More contributors than ever! > > http://www.digitalmars.com/d/1.0/changelog.html > http://ftp.digitalmars.com/dmd.1.075.zip > > http://www.digitalmars.com/d/2.0/changelog.html > https://github.com/downloads/D-Programming-Language/dmd/dmd.2.060.zip A new Gentoo ebuild is submitted for review as well. Thanks for the posix.mak changes and zlib update, I could delete quite a bit of my patch. By the way, it would be great if the bash completion script was also available in the .zip distribution. -- Marco
Re: Wiki page for C bindings / wrappers and reimplementations
Am Wed, 01 Aug 2012 21:35:32 +0200 schrieb David : > You could make it output json, and someone else does the fancy html? Maybe I'll make it a little more fancy, settle for a practical database table structure and then output JSON. -- Marco
Re: Dscanner - It exists
Am Wed, 1 Aug 2012 22:39:41 +0200 schrieb Philippe Sigaud : > I just tested the JSON output and it works nicely. Finally, a way to > get imports! What does it do if you import from _inside_ a function ? Not that this would happen often, but it can. :-] -- Marco
Re: Dscanner - It exists
Am Wed, 01 Aug 2012 19:58:46 +0200 schrieb "Brian Schott" : > On Wednesday, 1 August 2012 at 17:36:16 UTC, Walter Bright wrote: > > > > I suggest proposing the D lexer as an addition to Phobos. But > > if that is done, its interface would need to accept a range as > > input, and its output should be a range of tokens. > > It used to be range-based, but the performance was terrible. The > inability to use slicing on a forward-range of characters and the > gigantic block on KCachegrind labeled "std.utf.decode" were the > reasons that I chose this approach. I wish I had saved the > measurements on this I can understand you. I was reading a dictionary file with readText().splitLines(); and wondering why a unicode decoding was performed. Unfortunately ranges work on Unicode units and all structured text files are structured by ASCII characters. While these file formats probably just old or done with some false sense of compatibility in mind, it is also clear to their inventors, that parsing them is easier and faster with single-byte characters to delimit tokens. But we have talked about UTF-8 vs. ASCII and foreach vs. ranges before. I still hope for some super-smart solution, that doesn't need a book of documentation and allows some kind of ASCII-equivalent range. I've heard that foreach over UTF-8 with a dchar loop variable, does an implicit decoding of the UTF-8 string. While this is useful it is also not self-explanatory and needs some reading into the topic. -- Marco
Re: Wiki page for C bindings / wrappers and reimplementations
Am Mon, 30 Jul 2012 16:50:57 +0200 schrieb Marco Leise : I have now written a simple web site the lists C bindings by category. It is updated every day. http://mleise.abcz8.com/d/bindings.php Currently it only lists Deimos repositories and includes them even if they only contain C headers. Oh and it is ugly. -- Marco
