Re: utiliD: A library with absolutely no dependencies for bare-metal programming and bootstrapping other D libraries
On Friday, 10 May 2019 at 17:16:24 UTC, Mike Franklin wrote: On Friday, 10 May 2019 at 05:20:59 UTC, Eugene Wissner wrote: - Memcmp, memcpy, memmove and memset are named equal, copy, copyBackward and fill respectively. I just wanted to create native implementations that are bit safer than their C counterparts. So they do the same job, but accept void[] instead of pointers. There are also templated functions with the same names, that work with ranges. I‘m not very comfortable with GCC‘s inline asm and it doesn‘t support naked asm as DMD does, so I put asm in the .S files and compile them separately. But I‘m fine with inline asm too. Why would you use inline assembly ? (generalizing but: extremely bad portability, bad performance, bad readability) Recent discussion on LLVM mailinglist about the problem of the optimizer recognizing memcmp implementation and substituting it with a call to memcmp: https://lists.llvm.org/pipermail/llvm-dev/2019-April/131973.html cheers, Johan
Re: DConf 2019 Day 3 Slides
On Friday, 10 May 2019 at 08:03:52 UTC, Mike Parker wrote: The slides for Andrei's keynote can be found here: https://drive.google.com/open?id=1nrya9553FSMyBfLUmioYovVbqOoYycMe Thank you Andrei for presenting those last 3 slides. -Johan
Re: utiliD: A library with absolutely no dependencies for bare-metal programming and bootstrapping other D libraries
On Friday, 10 May 2019 at 23:58:37 UTC, Mike Franklin wrote: I don't know how a proper assembly implementation would not be performant. Perhaps you could elaborate. Inline assembly prevents a lot of optimizations that give large performance gains such as constant propagation. Say you implement a memcpy with a different signature than C's mempcy (because of slices instead of pointers), then the optimizer does not know what the semantics of that function are and will need the function to be transparent (not assembly) to do such optimizations. But I'm sure you know all that, so that's not your question. :) In the case of reimplementing memcpy/mem* in a function with the same signature as libc, that is not supposed to be inlined (like the current libc functions), then I also think the use of inline asm will not give a perf penalty. Be careful to recreate the exact same semantics as those libc functions because the optimizer is going to _assume_ it knows _exactly_ what those functions are doing. cheers, Johan
Re: Blog Post: Beating std::visit Without Really Trying
On Saturday, 5 October 2019 at 02:59:58 UTC, Paul Backus wrote: I was curious how C++17's std::variant compared to the options we have in D, like Algebraic and SumType, so I did a simple comparison of the generated assembly for each of them. You can read about it at the link below. And as you can probably guess from the title, D comes out ahead, in the end. https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html This is my first attempt at sharing something like this, so any comment or feedback is very much appreciated! It'd be interesting if you could investigate the case of non-trivial lambdas, where inlining the lambdas is not profitable. Perhaps easiest if you call opaque (declaration-only) functions in the visit list. Conclusion, performance-wise: - SumType is great - std::visit is OK'ish - D Algebraic is terrible Do you agree? cheers, Johan
Re: D code running on the Nintendo 3DS
On Sunday, 20 October 2019 at 06:06:48 UTC, TheGag96 wrote: Hi, all. I wanted to get into the world of 3DS homebrew, but I really didn't feel like coding in C or C++. So, through an effort of sheer will, I somehow got a hello world example written in D up and running, along with bindings for most of libctru and citro3d. https://github.com/TheGag96/3ds-hello-dlang Included are instructions on how to set this up (although it's pretty hacky). I imagine one could easily start writing Switch homebrew in D by following most of these steps as well. Once GCC 10 comes out with an updated GDC, it might become a pretty attractive alternative to C/++ for such projects if some work is put into bindings and stuff. Hope someone finds this interesting! Definitely :-) Did you try with LDC? Just a wild guess but perhaps this triple will work: `-mtriple=armv6k-unknown-eabi`. -Johan
Re: Proposal for porting D runtime to WebAssembly
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe wrote: This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d Perhaps you can explicitly clarify that "port" in this context means that you will add the required version(WebAssembly) blocks in the official druntime, rather than in a fork of druntime. (WebAssembly predefined version now explicitly mentions that it is for 32bit. Do you want to broaden this to 64bit aswell, or add a new version identifier?) I read that Clang uses a triple with explicit mention of WASI: --target wasm32-wasi Are you planning for the same with LDC? Will you need a new predefined version identifier for WASI-libc? Perhaps group all required compiler features in a section (and move the `real` story there). Can you elaborate on how you envision CI testing? Do you want to add that to LDC testing? (this may also mean that you first add a new change to LDC's druntime, confirming functionality with LDC CI, and then upstreaming the change) I'm assuming you already started some work in this area? Where can we track it? Great initiative! Johan
Re: LDC 1.24.0-beta1
On Tuesday, 20 October 2020 at 20:21:56 UTC, aberba wrote: On Tuesday, 20 October 2020 at 17:36:11 UTC, kinke wrote: On Tuesday, 20 October 2020 at 16:08:47 UTC, aberba wrote: It's an option but doesn't fill the need for an installer. Not sure why its hasn't been done. See https://github.com/ldc-developers/ldc/issues/1754. From the discussions, it seems you still don't see the value of an installer...backing it with the idea that LDC is for "developers". I'm a developer myself and I use installers all the time when on Windows...there are very few people I personally know who would go for an archive file to set it up themselves. So not everyone is like you. The reason why I personally go for DMD over LDC is convenience (especially when introducing D to newbies)...even though LDC is more optimized for performance. Unless what you guys are doing is an artificial barrier to get others to not use it. Guys, all points have been made, there is no wrong and right here, let's stop arguing over this. What is needed is someone who thinks it is useful to have an exe installer and wants to do the work. It cannot be done by someone who thinks it is not useful, because there are decisions to be made (like which folder to install it in, whether to overwrite old or not), that can only be made by someone who actually cares about it. There is no point in trying to convince kinke or me. I'm sure noone will be against uploading the installer exe onto github release page once it's been made and checked. -Johan
Re: LDC 1.24.0
On Saturday, 24 October 2020 at 15:11:08 UTC, kinke wrote: Glad to announce LDC 1.24 - some highlights: - Based on D 2.094.1+. - Support for LLVM 11. The prebuilt packages use v11.0.0, and the x86 packages newly include the LLVM backend for AMD GPUs. - Experimental support for targeting macOS on 64-bit ARM. The macOS package includes prebuilt druntime/Phobos libraries for cross-compilation via `-mtriple=arm64-apple-macos`. Full release log and downloads: https://github.com/ldc-developers/ldc/releases/tag/v1.24.0 Thanks to all contributors & sponsors! Thanks Martin! FYI: this is the first release that includes a working ThreadSanitizer. Curious to hear people's experiences with it. Cmdline flag: -fsanitize=thread https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual Examples: https://github.com/ldc-developers/ldc/blob/master/tests/sanitizers/tsan_tiny_race.d https://github.com/ldc-developers/ldc/blob/master/tests/sanitizers/tsan_tiny_race_TLS.d cheers, Johan
Re: LDC 1.25.0
On Monday, 22 February 2021 at 07:20:16 UTC, kinke wrote: On Sunday, 21 February 2021 at 21:58:48 UTC, Dennis wrote: On Sunday, 21 February 2021 at 18:26:38 UTC, kinke wrote: - Profile/trace LDC invocations via --ftime-trace. Is there more info on this? I tried it on a simple file, it generates a large json file. Can this be inspected to reduce compile times? The changelog and pull request do not reveal much info on how to use it for the uninformed. See Johan's comment here: https://forum.dlang.org/post/usyrbgwzzwxnzzhqe...@forum.dlang.org Thanks kinke. This article explains the basics: https://www.snsystems.com/technology/tech-blog/clang-time-trace-feature I'm aware there is very little documentation about this... I lack the time to write up more details and promote it better. Now that it's released, I hope promotion can be picked up by LDC's users. Here is an example of how it can help: https://github.com/dlang/phobos/pull/7676#issuecomment-715373535 There is definitely room for improvement, but it requires user feedback and consideration of their different use cases. -Johan
Re: From the D Blog: Driving with D
On Friday, 4 June 2021 at 21:47:21 UTC, Iain Buclaw wrote: On Friday, 4 June 2021 at 21:28:00 UTC, Max Samukha wrote: This doesn't [work]: ubyte[] slice; foreach (ubyte i; slice) { } Invalid bitcast %17 = bitcast i16 %15 to i32 I guess the cause is the same - slice.length.sizeof == 4, while slice.sizeof == 4, slice.ptr.sizeof == 2, and size_t.sizeof == 2. You should have better luck using gdc on avr. https://explore.dgnu.org/z/bos5ee Hi Max, GDC and LDC are both happy to support many platforms (it's fun), so keep filing bug reports in our bug trackers! (LDC's bug tracker is on Github, please also mention the triple you are using) cheers, Johan
Re: LDC talk @ FOSDEM'17
On Saturday, 21 January 2017 at 12:37:26 UTC, Kai Nacke wrote: Hi everybody! Last year turned out to be difficult for D development for me. I try to come back this year. Great! :-) Keep me in the loop when preparing your slides! ;-) cheers, Johan
Re: A New Import Idiom`
On Monday, 13 February 2017 at 15:04:53 UTC, Daniel N wrote: On Monday, 13 February 2017 at 15:00:24 UTC, jmh530 wrote: With the Jack Stouffer comparison, wouldn't it be fairer to do: import std.datetime : Systime; import std.traits : isIntegral; Probably, please help measuring your idea and post it here. I intentionally chose a test made by someone else to avoid biasing from my part. It was the only test I found at time of writing. In case you don't actually use the function, things are faster and you get a smaller object file because the template is never instantiated. So if the function is never used in the module (it is just there for other modules to use), then things are indeed faster and smaller. But you also don't get compile checking on it; putting only ``` void func(T)(from!"std.datetime".SysTime a, T value) if (from!"std.traits".isIntegral!T) { import std.stdio : writeln; writeln(a, value); } ``` in a file compiles fast without complaints about an undefined `from`. But for a full program, compile time is the same (it's really short so you can't really conclude much from it), and binary size as well (LDC 1.1.0, -O3). For my test cases below, the binaries are identical, except for around 30 bytes... My test cases: ``` template from(string moduleName) { mixin("import from = " ~ moduleName ~ ";"); } void func(T)(from!"std.datetime".SysTime a, T value) if (from!"std.traits".isIntegral!T) { import std.stdio : writeln; writeln(a, value); } void main() { func!int(from!"std.datetime".SysTime(), 1); } ``` ``` import std.datetime; import std.traits; void func(T)(SysTime a, T value) if (isIntegral!T) { import std.stdio : writeln; writeln(a, value); } void main() { import std.datetime : SysTime; func!int(SysTime(), 1); } ``` -Johan
Re: A New Import Idiom`
On Monday, 13 February 2017 at 14:28:20 UTC, Mike Parker wrote: Daniel Nielsen put together a post ... [1] https://dlang.org/blog/2017/02/13/a-new-import-idiom/ I also liked the post, nicely written. I didn't check, but if this idiom is not already tested in dmd-testsuite, please add it! -Johan
Re: A New Import Idiom`
On Monday, 13 February 2017 at 20:09:45 UTC, Daniel Nielsen wrote: I still suspect there will be a difference in real-world applications even with LDC. Why would you expect there to be a difference? Module ctors/dtors are still pulled in, regardless of the import being selective or not. What am I missing? Thanks, Johan
Re: Trip notes from Israel
On Monday, 22 May 2017 at 15:05:24 UTC, Andrei Alexandrescu wrote: http://dlang.org/blog/2017/05/22/introspection-introspection-everywhere/ -- Andrei A fun read! "(Late at night, I double checked. Mozilla’s CheckedInt is just as bad as I remembered. They do a division to test for multiplication overflow. Come on, put a line of assembler in there! Portability is worth a price, just not any price.)" Shocked: do you use assembly in Checked and cripple the optimizer?!?! Luckily, no. But LDC and GDC do create the `seto` instruction I think you were hinting at: https://godbolt.org/g/0jUhgs (LDC doesn't do as good as it could, https://github.com/ldc-developers/ldc/issues/2131) cheers, Johan
Re: Trip notes from Israel
On Tuesday, 23 May 2017 at 08:18:26 UTC, Iain Buclaw wrote: So for LDC to be as good as GDC, you need to need to compile with -enable-ldc-amazing-feature-cross-module-inlining? In this case, yep. If you want a hint (though it's not my place to say), LLVM I'm told is a reasonably OK compiler, and any reasonably OK compiler should come with overflow intrinsics - try using them directly. Pleasantly surprised that you think so highly of LDC and LLVM that you assume we are able to get the `seto` instruction in this case without using an overflow intrinsic. - Johan
Re: D for Android beta
On Thursday, 1 June 2017 at 19:31:28 UTC, Joakim wrote: [awesome text] This is great stuff Joakim! It's very nice to see your detailed release notes, with links to the patches. Hope we can get much of that into LDC master soon. Cheers, Johan
Re: Snap packages for DMD and DUB
On Monday, 8 May 2017 at 20:05:01 UTC, Joseph Rushton Wakeling wrote: The `dmd` package also contains `rdmd`, `dustmite` and `ddemangle`. Note that by default the snap package exposes these namespaced on the name of the package (i.e. `dmd.rdmd`, etc.). To avoid this, use the `snap alias` command: sudo snap alias dmd rdmd sudo snap alias dmd dustmite sudo snap alias dmd ddemangle Have you thought about creating a `dtools` package with rdmd, dustmite, and ddemangle in it? They are useful for LDC and GDC too, perhaps people would like to be able to install them without needing to install DMD. (Perhaps the DMD/LDC/GDC package can automatically install such a "dtools" package?) cheers, Johan
Re: Cap'n Proto for D v0.1.2
On Tuesday, 18 April 2017 at 18:09:54 UTC, Thomas Brix Larsen wrote: This is the initial public release of my optimized port of the Java implementation of Cap'n Proto. http://code.dlang.org/packages/capnproto-dlang https://github.com/ThomasBrixLarsen/capnproto-dlang Hi Thomas, Great that you are working on this. I strongly advise you to advertise your work on this page: https://capnproto.org/otherlang.html Cheers, Johan
Re: Work on ARM backend for DMD started
On Tuesday, 4 July 2017 at 21:10:45 UTC, Walter Bright wrote: The backend has also been accused of not doing data flow analysis. It does as good a flow analysis as any compiler. Please... DMD: https://goo.gl/wHTPzz GDC & LDC: https://godbolt.org/g/QFSgaX -Johan
Re: DCompute: GPGPU with Native D for OpenCL and CUDA
On Monday, 17 July 2017 at 13:50:22 UTC, Mike Parker wrote: Nicholas Wilson has put together a blog post on his progress with DCompute Great, Nick!
Re: DMD library available as DUB package
On Tuesday, 18 July 2017 at 12:07:27 UTC, Jacob Carlborg wrote: During the dconf hackathon I set out to create a DUB package for DMD to be used as a library. This has finally been merged [1] and is available here [2]. It contains the lexer and the parser. This is great news of course! But I have some bad news ;-) Now that the Lexer nicely separated, it is very easy for me to testdrive libFuzzer+AddressSanitizer on the lexer and... Expect many bug reports in the next days. I am testing this code: ``` void fuzzDMDLexer(const(char*) data, size_t length) { scope lexer = new Lexer("test", data, 0, length, false, false); lexer.nextToken; do { auto drop = lexer.token.value; } while (lexer.nextToken != TOKeof); } ``` A short list of heap-overflow memory access bugs (params data and length are consistent): 1. length == 0 2. data == "\n" (line feed, 0xa) 3. data == "only_ascii*" (nothing following the "*" is the problem) 4. data == "%%" 5. data == "*ô" 6. data == "\nÜÜÜ" 7. data == "\x0a''" 8. data == ")\xf7" `void scan(Token* t)` is to blame for most of the bugs I found so far. See e.g. line 980 that causes bug 3: https://github.com/dlang/dmd/blob/154aa1bfd36333a8777d571e39690511e670bfcf/src/ddmd/lexer.d#L979-L980 Example of stacktrace (bug 8): ``` ==11222==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200952 at pc 0x0001028915b5 bp 0x7fff5d3941f0 sp 0x7fff5d3941e8 READ of size 1 at 0x60200952 thread T0 #0 0x1028915b4 in _D4ddmd5lexer5Lexer9decodeUTFMFZk lexer.d:2314 #1 0x102887cae in _D4ddmd5lexer5Lexer4scanMFPS4ddmd6tokens5TokenZv lexer.d:1019 #2 0x102875089 in _D4ddmd5lexer5Lexer9nextTokenMFZE4ddmd6tokens3TOK lexer.d:222 #3 0x1028c5d20 in _D9fuzzlexer12fuzzDMDLexerFxPhmZv fuzzlexer.d:31 ``` I am very excited to see the fuzzer+asan working so nicely! :-) Johan
Re: DMD library available as DUB package
On Sunday, 30 July 2017 at 23:41:40 UTC, Johan Engelen wrote: On Tuesday, 18 July 2017 at 12:07:27 UTC, Jacob Carlborg wrote: During the dconf hackathon I set out to create a DUB package for DMD to be used as a library. This has finally been merged [1] and is available here [2]. It contains the lexer and the parser. This is great news of course! But I have some bad news ;-) Now that the Lexer nicely separated, it is very easy for me to testdrive libFuzzer+AddressSanitizer on the lexer and... Expect many bug reports in the next days. OK, this wasn't entirely fair. 1. I didn't read the API: the buffer needs to be null-terminated. 2. With a fix [1] to prevent reading beyond the input buffer, I have yet to find a new bug. The fuzzer is running now... I wonder how long it takes to find the next bug, if any. -Johan [1] https://github.com/dlang/dmd/pull/7050
Re: LDC 1.5.0-beta1
On Monday, 23 October 2017 at 21:26:46 UTC, Guillaume Piolat wrote: Some measurements for some audio processing. Output are identical across compilers (30 measures). Guillaume, can you try with building your own LTO version of the standard library and do new measurements? Something like this should work: ``` ldc-build-runtime --reset --dFlags="-flto=full" BUILD_SHARED_LIBS=OFF ldc2 -your-normal-compile-flags -flto=full -L-L./ldc-build-runtime.tmp/lib ``` Use `--dFlags="-flto=full;-ar="` for now if you are using macOS (will not be needed with the next beta). (you can try with `-flto=thin` too for much faster link times) Cheers, Johan
Re: Interfacing D with C: Getting Started
On Tuesday, 5 December 2017 at 16:06:19 UTC, Mike Parker wrote: This is the first post in a new tutorial series I'm doing on the blog. Hi Mike, Nice. It's always much easier to comment than to write an article from scratch. So take all this with a grain of salt: Overall I find that it'd be much nicer if you focus on C-D interaction only. Currently you've added a lot of things that people really would already know before reading the text: I think preknowledge should be how to create programs in C and in D (including compilers, etc.). Especially on a second read, it's a lot of scrolling back and forth to find the good nuggets of info. - I would make it clearer that you are discussing calling C code from a D application. If you are going to discuss the other way around too, always make it super clear in which language the "main" part of the program is. Perhaps explain the differences in how to setup a "hello world" (e.g. druntime initialization). - I would remove all related to DMC, it's needless extra complication. - Add a sentence that all should work identically for GDC and LDC, except for the `-m32mscoff` (I think `-m32` or `-m64` is superfluous for LDC). - The text about "both files have the same name" is also needless, I think. You are writing a text about how to call C-code from D-code, using the commandline no less! ;-) Instead, I would add the output filename explicitly on the cmdlines. It's short, and more clearly shows which files are created when. - Commandline invocation and installation of compilers is mixed now; pull it apart for clarity. Fit the installation of compilers in one or two sentences (D and C). - typo: "offical" --> "official" - The code fragments don't need to be packed tight. Perhaps add comments inside them, to beef them up a bit and make them stick out more. (you could add comments about "forward declaration", "definition", ...) - Separate the hello world example into its own section: "see how easy it is?" :-) Ideas: - Idea for future texts: how to set this up using dub, and other build mechanisms like cmake, makefile, IDEs, ... - Perhaps an example of why one might want this, external libraries is what I am thinking. Forward reference to future blog article? (dstep?) Cheers, Johan
Re: Silicon Valley D Meetup - December 14, 2017 - "Experimenting with Link Time Optimization" by Jon Degenhardt
On 11/21/2017 11:58 AM, Ali Çehreli wrote: Meetup page: https://www.meetup.com/D-Lang-Silicon-Valley/events/245288287/ LDC[1], the LLVM-based D compiler, has been adding Link Time Optimization capabilities over the last several releases. [...] This talk will look at the results of applying LTO to one set of applications, eBay's TSV utilities[2]. [...] Jon Degenhardt is a member of eBay's Search Science team. [...] D quickly became his favorite programming language, one he uses whenever he can. On Friday, 15 December 2017 at 03:08:35 UTC, Ali Çehreli wrote: This should be live now: http://youtu.be/e05QvoKy_8k Great! I've added some comments there, pasted here: Jon, thanks for the extensive talk and testing on LTO! And thanks for recording / broadcasting :-) (times are approximate) 7:45 Full vs Thin LTO further clarification: Full LTO is single threaded optimization and codegen (comparable with putting all source in one module). Thin LTO loads each module separately and imports functions it needs from other modules, then after the optimization and codegen happen in parallel for each module (and normal linking happens afterwards). LTO's capabilities stem from having access to functions' source code of other modules, and knowing which functions are internal to the program (so that they can be removed, non-ABI-conformant calling convention, etc., also discussed around 41:30); the importing+optim that happens at the start of Thin LTO gives you that, with the added advantage of parallel optim+codegen afterwards. 14:00 If the question was: do you need all libraries to be in IR: no. LTO works with mixed IR-object files and normal object files and libraries. Even if linking with non-IR libraries, it helps to know that no other object file references a symbol (so you can internalize it and generate better code). But indeed, for _much_ better optimization potential: the more source you have compiled with LTO enabled the better. 15:30 Whole source optimization at D-level has indeed higher potential; at the moment I don't think we do many optimizations that are only possible at D-level (and so they are done at IR level; or not at all... I'm working e.g. on devirtualization). Extra remark: the first step towards that is much deeper and well-defined spec of D semantics, in abstract machine terms. 15:45 Testing == contributing! And you're testing has greatly improved LDC's LTO, thanks! 15:50 The ldc-build-runtime tool was made by Martin Kinkelin, and as you mention it is the enabler for most of your work. 16:15 LDC LTO Windows == integrating LLD into LDC (or using lld-link.exe), https://github.com/ldc-developers/ldc/issues/2028 ~30:00 IIRC, the performance regression is due to cross-module inlining/optim (as you mention), which we get for free with LTO :-) (that is not to say that we wouldn't like to do cross-module inlining without LTO) 33:20 Compilation time. LTO skips machine codegen during the normal compilation, as machine codegen is done in the LTO linking step. So the slowdown with Thin LTO may not be too much (Thin LTO being a parallel build). An extreme case where LTO may actually result in faster codegen: if you have 1 million template function instantiations in CTFE, but they are not called during runtime, LTO may easily discard them before they reach the optimization and machine codegen stage. In such a case, LTO may very well be faster (optimized machine codegen is time consuming); however, the IR does have to be created and written to disk, and then read from disk, that takes time too... Overall, Thin LTO is slower than a normal `-O3` build, but only by a small ratio, but it also does more work (the added optimization). The compile speed difference between Full LTO and Thin LTO is very large (Full LTO is several times slower). 39:40 Indeed, D doesn't require codegen of templates if we can prove that it is already codegenned in the library itself: i.e. you _have_ to _link_ with a template-only library. In C++, codegen of templates is mandatory (afaik), and thus you do not have to link with a template-only library (e.g. headers files only). In D, this culling of template codegen is done to increase compile speed; in that sense not a fair comparison with C++. For cross-module inlining / inlining of templated functions: in C++ all template code is available in each codegenned module, so LTO is not needed to improve things; in D, using LTO makes template code available that otherwise wouldn't ---> larger (potentially much larger) relative gains with LTO for D. (this is somewhat particular to LDC currently; GDC does better cross-module inlining; try LDC's `-enable-cross-module-inlining`) 56:40 Fully share your thinking that cross-module inlining is the main source of performance gains Can't wait to see the results of LTO on Weka.io's (LARGE) applications. Work in progress...! Could you add the reference
Re: D's Newfangled Name Mangling
On Wednesday, 20 December 2017 at 13:57:20 UTC, Mike Parker wrote: Many thanks to Rainer for his insightful new article for the D Blog outlining the new name mangling algorithm. Nice! "D and C++ avoid this problem by adding more information to the symbol name, i.e. they encode into a symbol name the scope in which the symbol is defined, the function argument types, and the return type. " I would change it to "... and the return type (D only)". C++ does not include the return type in the mangle for normal functions (it does for templates). An important difference, concerning the earlier remark about programs crashing when "fail to update and recompile all source files that use the new declarartion" -Johan
Re: Silicon Valley D Meetup - December 14, 2017 - "Experimenting with Link Time Optimization" by Jon Degenhardt
On Saturday, 16 December 2017 at 19:40:14 UTC, Jon Degenhardt wrote: On Saturday, 16 December 2017 at 11:52:37 UTC, Johan Engelen wrote: Can't wait to see the results of LTO on Weka.io's (LARGE) applications. Work in progress...! Agreed. It'd be great to see the experience of a few more apps. Don't have performance numbers yet. But the executable size of the release build with ThinLTO drops to a third of the non-LTO executable size...!!! 809 MB to 250 MB. (That's without an LTO build of druntime/Phobos.) -Johan
Article: Finding memory bugs in D code with AddressSanitizer
I've been writing this article since August, and finally found some time to finish it: http://johanengelen.github.io/ldc/2017/12/25/LDC-and-AddressSanitizer.html "LDC comes with improved support for Address Sanitizer since the 1.4.0 release. Address Sanitizer (ASan) is a runtime memory write/read checker that helps discover and locate memory access bugs. ASan is part of the official LDC release binaries; to use it you must build with -fsanitize=address. In this article, I’ll explain how to use ASan, what kind of bugs it can find, and what bugs it will be able to find in the (hopefully near) future." Thanks for your proof-reading. cheers, Johan
Re: Article: Finding memory bugs in D code with AddressSanitizer
On Monday, 25 December 2017 at 20:31:18 UTC, Walter Bright wrote: Thanks for the great article! Some suggestions: Thanks for your comments, I've incorporated them (to my liking). 1. The gray-on-white text is not very legible. Looks great here, I like it, sorry. (made it completely black now, can't see the difference here though) Snips: This comes across as unduly negative. This is a contrived example, and implies that normal D code is written like C++ code. It would be nice to add a paragraph mentioning things about D that make it a more memory safe language. This code should be rejected by the compiler if using -dip1000. It is not, so I filed a bug report I'd rather use examples that didn't rely on compiler/library bugs. You're right, the examples are (of course) contrived. However, I didn't want to write a marketing article, and I also want to show examples found in the wild. I think one of the use cases of ASan is exactly that it can help discover bugs whereever they are, even in the compiler / standard library. I've added bits and pieces to indicate some facilities of D to mitigate these kinds of bugs, but the reality is that a lot of D code is not idiomatic and does not use the safety features (for diverse reasons). The article is not meant as a marketing piece (only for ASan), but also shouldn't be overly critical of D. Hope that the balance is a bit better now with the modifications. -Johan
Re: Article: Finding memory bugs in D code with AddressSanitizer
On Tuesday, 26 December 2017 at 22:11:18 UTC, Jon Degenhardt wrote: On Monday, 25 December 2017 at 17:03:37 UTC, Johan Engelen wrote: I've been writing this article since August, and finally found some time to finish it: http://johanengelen.github.io/ldc/2017/12/25/LDC-and-AddressSanitizer.html Nice article. Main question / comment is about the need for blacklisting D standard libraries (druntime/phobos). If someone wants to try ASan out on their own code, can they start by ignoring the D standard libraries? And, for programs that use druntime/phobos, will this be effective? If I understand the post, the answer is "yes", but I think it could be more explicit. Indeed, yes. I've used ASan successfully on the ddmd lexer. "successfully" = I found and fixed an actual bug with it. Without ASan-enabled standard libs, ASan testing will cover your code and (most) std lib _templated_ code. A blacklist may be needed for templated std lib code that doesn't work with ASan (yet), either because of a bug in the std lib (not very likely I think) or something else. We need much more testing of LDC+ASan. Second comment is related - If the reader was to try instrumenting druntime/phobos along with their own code, how much effort should be expected to correctly blacklist druntime/phobos code? Would many programs have smooth sailing if they took the blacklist published in the post? Or is this early stage enough that some real effort should be expected? Very early stage. I myself have not worked on ASan-enabled druntime/phobos for more than 30 minutes. Already found some trouble with cpuid functions (inline asm): `fun:_D4core5cpuid*` must be added to the blacklist. I think the first goal should be to make a blacklist such that all tests pass, adding blacklist items in a "# not reviewed" section. Then afterwards, we can reduce the blacklist bit-by-bit by figuring out exactly why ASan triggers: either a bug, expected behavior, or an ASan bug. A counterpart to the blacklist file is an `@no_sanitize("address")` magic UDA; to disable ASan and document it inside the code. This should be done in such a way that it is upstreamable. (e.g. version(LDC) static import ldc.attributes, alias no_sanitize = ...) Also, if the blacklist file in the post represents a meaningful starting point, it does perhaps it makes sense to check it in and distribute it. This would provide a place for contributors to start making improvements. Definitely makes sense. I think this should be inside the runtime libraries' repos, right? (So one blacklist for druntime, and another for Phobos). (I'm even thinking about adding `-fsanitize-blacklist=<...>` to the shipped blacklist in `ldc.conf`.) I'll figure out how to incorporate your comments into the article, thanks. cheers, Johan
Re: Article: Finding memory bugs in D code with AddressSanitizer
On Monday, 25 December 2017 at 17:03:37 UTC, Johan Engelen wrote: I've been writing this article since August, and finally found some time to finish it: http://johanengelen.github.io/ldc/2017/12/25/LDC-and-AddressSanitizer.html Is it a good fit with /r/programming ? -Johan
Re: Article: Finding memory bugs in D code with AddressSanitizer
On Wednesday, 3 January 2018 at 23:16:45 UTC, Martin Nowak wrote: On 12/25/2017 06:03 PM, Johan Engelen wrote: I've been writing this article since August, and finally found some time to finish it: http://johanengelen.github.io/ldc/2017/12/25/LDC-and-AddressSanitizer.html Just built dmd with AddressSanitizer and ran dmd's, druntime's, and phobos' test-suite. Nice. Plans to make it part of CI ? -Johan
D code on CodeOcean
Hi all, I made two example "capsules" on CodeOcean.com that demonstrate how to use D on that platform. CodeOcean is a platform for publishing reproducible code accompanying scientific publications for example. Hello World: https://codeocean.com/2018/01/04/d-lpar-dlang-rpar-hello-world VectorFlow (using dub): https://codeocean.com/2018/01/04/vectorflow-mnist-example-lpar-dlang-dub-rpar So if any of you is using D code in publications (Andrei?), consider publishing your code there :) Cheers, Johan
Article: Fuzzing D code with LDC
It's been a work-in-progress for half a year, but finished now: http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html "A not-so-well-written article about the fuzzing capability recently added to LDC, using LLVM’s libFuzzer. Compiling code with -fsanitize=fuzzer adds control-flow instrumentation used to guide the fuzzing and links-in the libFuzzer library that drives the fuzz testing (same as Clang). -fsanitize=fuzzer is available from LDC 1.4.0, not on Windows. LDC 1.6.0 was used for the examples in this article." Hope the article gives you enough information to start fuzz testing your own projects. cheers, Johan (per Ali's request, the article has a nice fuzzy font for some of you :P)
Re: Article: Fuzzing D code with LDC
On Sunday, 14 January 2018 at 17:33:45 UTC, Joakim wrote: Nice post, enjoyed reading it. I need to look into trying out the shift sanitizer sometime. Definitely would be *great* to have `-fsanitize=undefined` ! - Johan
Re: Article: Fuzzing D code with LDC
On Saturday, 13 January 2018 at 23:59:52 UTC, Johan Engelen wrote: It's been a work-in-progress for half a year, but finished now: http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html "A not-so-well-written article about the fuzzing capability recently added to LDC Argh, and just now I find out about: - https://www.youtube.com/watch?v=k-Cv8Q3zWNQ - https://github.com/google/oss-fuzz Anybody interested in looking at OSS Fuzz? "OSS-Fuzz - Continuous Fuzzing for Open Source Software Status: Beta. We are now accepting applications from widely-used open source projects." Cheers, Johan
Re: Article: Fuzzing D code with LDC
On Monday, 15 January 2018 at 09:49:24 UTC, Temtaime wrote: // `Throwable`s thrown are not bugs (in contrast to `Errors`). They _can_ be bugs. class Error : Throwable class Exception : Throwable Good catch ( ;-) ) thanks. -Johan
Re: TSV Utilities release with LTO and PGO enabled
On Tuesday, 16 January 2018 at 02:45:39 UTC, Jon Degenhardt wrote: On Tuesday, 16 January 2018 at 00:19:24 UTC, Martin Nowak wrote: On Sunday, 14 January 2018 at 23:18:42 UTC, Jon Degenhardt wrote: Combined, LTO and PGO resulted in performance improvements greater than 25% on three of my standard six benchmarks, and five of the six improved at least 8%. Yay, I'm usually seeing double digit improvements for PGO alone, and single digit improvements for LTO. Meaning PGO has more effect even though LTO seems to be the more hyped one. Have you bothered benchmarking them separately? Last spring I made a few quick tests of both separately. That was just against the app code, without druntime/phobos. Saw some benefit from LTO, mainly one of the tools, and not much from PGO. Because PGO optimizes for the given profile, it would help a lot if you clarified how you do your PGO benchmarking. What kind of test load profile you used for optimization and what test load you use for the time measurement. Regardless, it's fun to hear your test results :-) Johan
Re: TSV Utilities release with LTO and PGO enabled
On Wednesday, 17 January 2018 at 04:37:04 UTC, Jon Degenhardt wrote: Clearly personal judgment played a role. However, the tools are reasonably task focused, and I did take basic steps to ensure the benchmark data and tests were separate from the training data/tests. For these reasons, my confidence is good that the results are reasonable and well founded. Great, thanks for the details, I agree. Hope it's useful for others to see these details. (btw, did you also check the performance gains when using the profile of the benchmark itself, to learn about the upper-bound of PGO for your program?) I'll merge the IR PGO addition into LDC master soon. Don't know what difference it'll make. -Johan
Re: LDC 1.7.0
On Tuesday, 30 January 2018 at 09:38:26 UTC, aberba wrote: On Monday, 29 January 2018 at 07:40:10 UTC, Dominikus Dittes Scherkl wrote: On Saturday, 27 January 2018 at 21:42:49 UTC, aberba wrote: [...] Ubuntu 16.04 This is a long-term support distribution. Don't expect those to have actual tip versions of any SW package! They rely on stabe versions that don't have the latest features but only those very well tested. The semver 1.7 is not an unstable package. Its that their reason for no updates? LDC 1.7.0 includes major changes to the frontend and is not well-tested. If you want a better-tested recent LDC, I recommend LDC 1.6.0, which is used in production at Weka. - Johan
llvm-mca: a static performance analysis tool
I find this such great LLVM news that I'd share it here: llvm-mca: a static performance analysis tool http://lists.llvm.org/pipermail/llvm-dev/2018-March/121490.html cheers, Johan
Re: Updated D packages in GNU Guix for x86_64-linux, i686-linux, armhf-linux
On Sunday, 11 March 2018 at 08:43:25 UTC, Pjotr Prins wrote: The GNU package manager updated LDC to 1.7.0 which now provides x86_64-linux, see https://www.gnu.org/software/guix/packages/L/. Runtime dependencies and build are listed at https://hydra.gnu.org/build/2525193#tabs-runtime-deps, including 0.17.4 (bootstrap) and LLVM 3.8.1. LDC 0.17.4 provides x86_64-linux, i686-linux, armhf-linux. We aim to add D compilers for all supported architectures. Kai has been working recently [1] on updating LDC 0.17.x (ltsmaster) to work with LLVM 6, PowerPC, AArch64, ... You may want to take advantage of that activity to get better bootstrap compiler support for your platforms. For example by filing bug reports, or better: helping out with testing and fixing things ;-) -Johan [1] See e.g. https://github.com/ldc-developers/dmd-testsuite/pull/34
Re: Why think unit tests should be in their own source code hierarchy instead of side-by-side
On Thursday, 22 March 2018 at 15:18:40 UTC, Jacob Carlborg wrote: On Thursday, 22 March 2018 at 11:00:31 UTC, Atila Neves wrote: Direct link: https://atilanevesoncode.wordpress.com/2018/03/22/keep-d-unittests-separated-from-production-code/ I completely agree. Although my reason is mostly because there will be too much code in a single file if the regular code and unit tests are mixed in the same file. Fully agree with this "too much code in a single file" point. I am confident that part of the reason of Phobos unittesting being very incomplete, is that adding unittests further clutters the codebase. Moving all unittests to the bottom of the file (pulling them out of classes too) would resolve this issue in part. - Johan
Re: LDC 1.11.0 beta
On Saturday, 7 July 2018 at 18:17:49 UTC, Seb wrote: Would be great to include https://github.com/dlang/dmd/pull/8456 as it's a serious regression and the reason for the early 2.081.1 release. Because the quality of new DMD releases is often subpar, the LDC release plan is to only release after a good point release of DMD (usually *.*.1, but if this is a hasty .1 release, we probably better wait until 2.081.2) -Johan
Re: LDC 1.13.0-beta1
On Friday, 2 November 2018 at 21:04:13 UTC, kinke wrote: Glad to announce the first beta for LDC 1.13: Note how fast Martin produced this beta release after the DMD 2.083 release. Thanks to all contributors! The main contributor by far is you Martin, thank _you_! -Johan
Re: Fuzzed - a program to find DMDFE parser crash
On Saturday, 15 December 2018 at 11:29:45 UTC, Basile B. wrote: Fuzzed [1] is a simple fuzzer for the D programming language. Are you familiar with libFuzzer and LDC's integration? https://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html You can feed libFuzzer with a dictionary of keywords to speed up the initial fuzzing phase, where the keywords are the tokens strings that you use. Besides finding crashes, it's also good to enable ASan to find memory-related bugs that by luck didn't crash the program. The time to write this announce, already 5 "crashers" found. Great :) The other day I was reminded of OSS Fuzz and that it'd be nice if we would setup fuzzing for the frontend and phobos there... -Johan
LDC "nightly" or latest CI build
Hi all, The dlang.org install.sh script is now able to install the latest successful CI build of LDC master, called "ldc-latest-ci" ! That should make it easier for you to test the newest of the newest of LDC master. On Travis, you can use "d: ldc-latest-ci" to test your project with it. On d.godbolt.org, you can find it as ldc latest CI (dmd nightly is there too). Note: the CI build includes assertions for extra testing, which means that the compiler will be much slower than normal releases. Cheers, Johan
Re: Beta 2.084.1
On Tuesday, 5 February 2019 at 04:18:55 UTC, Martin Nowak wrote: Glad to announce the first beta for the 2.084.1 point release, Any chance of getting this in? https://github.com/dlang/phobos/pull/6814 Phobos std.file has become unusable after 2.082 because of [1] leading to memory corruption. It's pretty bad: dmd 2.082 and 2.083, and LDC 1.12 and 1.13 cannot be used for serious work (unless you prevent people from using std.file by deleting it?). Cheers, Johan [1] the trigger of the bug: https://github.com/dlang/phobos/pull/6584
Re: Terminal ANSI Colouring Library
On Monday, 16 March 2015 at 15:39:52 UTC, Kingsley wrote: Hi, As a learning project I've knocked up a tiny library to output ansi coloured text on linux/osx. Have a look here for an earlier attempt: http://forum.dlang.org/thread/mailman.41.1317934445.28623.digitalmar...@puremagic.com This would be great to have in Phobos!
Re: Testing Nightly Build Service
On Saturday, 12 December 2015 at 12:08:50 UTC, Martin Nowak wrote: As you might already know from the last sprint review (http://forum.dlang.org/post/56592679.3010604@dawg.), we've setup a server to build nightlies. The service is still in a test phase but seems to work steadily. Great! Are there plans for adding nightlies for GDC and LDC?
Re: Testing Nightly Build Service
On Sunday, 13 December 2015 at 12:26:12 UTC, Martin Nowak wrote: On Sunday, 13 December 2015 at 10:22:56 UTC, Suliman wrote: https://builds.dawg.eu/dmd-nightly/ Why not https://builds.dlang.org ? Because we're testing the service, once it's reliable, we'll move this to a dlang subdomain or integrate it with downloads.dlang.org. Hi Martin, Any news on this? (and it'd be great if LDC could be added too! ;-) cheers, Johan
Re: Sublime Text 3 Gets Better D Support
On Wednesday, 27 January 2016 at 17:34:35 UTC, Jack Stouffer wrote: Sublime Text is a very popular text editor, and for a while now it's had marginal D support. What has changed recently is updated syntax highlighting to support all the new keywords that have come in the last couple of years and UDAs Great news! Recently, I worked a little on adding rudimentary dfmt support (like clang-format) to Sublime: https://github.com/Hackerpilot/dfmt/pull/202
Re: DigitalWhip
On Saturday, 13 February 2016 at 18:48:12 UTC, artemalive wrote: DigitalWhip is a performance benchmark of statically typed programming languages that compile to native code: https://github.com/artemalive/DigitalWhip Could you add the compiler versions to the outputted .txt file, e.g. `dmd --version`? (the example output files don't have it)
Re: DigitalWhip
On Saturday, 13 February 2016 at 19:26:39 UTC, artemalive wrote: On Saturday, 13 February 2016 at 19:19:46 UTC, Johan Engelen wrote: Could you add the compiler versions to the outputted .txt file, e.g. `dmd --version`? (the example output files don't have it) These files are just for demonstration of expected output. The benchmark should work fine with the latest versions of D compilers. No guarantee for older versions;) I tested with dmd v2.069.2. What I meant to say is that the framework should output the compiler version, instead of "READY". At least for the D compilers, the performance depends so much on which version is used, that the output is unusable without version numbers, I feel.
Re: DigitalWhip
On Sunday, 14 February 2016 at 18:07:00 UTC, artemalive wrote: Thanks. Good suggestion. I'll check if the version information can be retrieved automatically for all compilers. If that's the case then version information will be added soon. I would simply print the output of "--version" for all compilers. But version information will help with archiving/sharing results. Exactly.
Re: LLVM 3.8 released - and LDC is already able to use it!
On Tuesday, 8 March 2016 at 19:12:57 UTC, Kai Nacke wrote: Hi all! LLVM 3.8 has been released half an hour ago! See the release notes here: http://www.llvm.org/releases/3.8.0/docs/ReleaseNotes.html Downloads: http://www.llvm.org/releases/download.html#3.8.0 Also note that LDC is mentioned in the release notes as the only external project who is already supporting LLVM 3.8. Just recompile LDC using master or ltsmaster branch from GitHub to use this new LLVM release. This is the 8th time that LDC and D are mentioned in the LLVM release notes! Very cool :) (you forgot to mention that we also support LLVM 3.9 ;-))
Blog post: PGO: Optimizing D's virtual calls
Hi all, I've written an article about how I implemented profile-guided optimization (PGO) of virtual calls to direct calls (a micro-micro-optimization, expected performance gain of just a few percent if any!). I hope it's interesting for those of you who like to read about compiler internals. https://johanengelen.github.io/ldc/2016/04/13/PGO-in-LDC-virtual-calls.html It's my first programming article, please be gentle with your comments :-) cheers, Johan
Re: Blog post: PGO: Optimizing D's virtual calls
On Wednesday, 13 April 2016 at 13:08:30 UTC, Andrei Alexandrescu wrote: Noice. https://twitter.com/D_Programming/status/720236648034877440 https://www.facebook.com/dlang.org/posts/1275322695814742 https://www.reddit.com/r/programming/comments/4elfok/profile_guided_optimization_optimizing_ds_virtual/ http://hackerne.ws Thanks for posting these! -Johan
Re: Blog post: PGO: Optimizing D's virtual calls
On Wednesday, 13 April 2016 at 11:34:26 UTC, Johan Engelen wrote: Hi all, I've written an article about how I implemented profile-guided optimization (PGO) of virtual calls to direct calls (a micro-micro-optimization, expected performance gain of just a few percent if any!). I've updated the article with performance measurements of just the D code of LDC: the improvement with PGO is about 7% !
Re: 2k questions on SO
On Sunday, 17 April 2016 at 12:10:47 UTC, Daniel Kozak wrote: http://stackoverflow.com/tags/d/info Awww, no [ldc] tag!
Re: Blog post: PGO: Optimizing D's virtual calls
On Wednesday, 20 April 2016 at 22:10:53 UTC, Marco Leise wrote: 7% is quite a bit more than any other option like -cpu=native could achieve. If feasible I'd enjoy if ldc2 shipped with a pgo configuration option. If you use the compiler in a lot of edit/compile cycles it will pay off. This was the use-case I was optimizing for, and so I will add an option to pass a PGO profile to the CMake script :-) Note that it is a 7% improvement in the D part of LDC. The optimizer (LLVM, C++) takes most of the time during compilation and so the overall performance increase is smaller.
Re: Blog post: PGO: Optimizing D's virtual calls
On Wednesday, 13 April 2016 at 11:34:26 UTC, Johan Engelen wrote: Hi all, I've written an article about how I implemented profile-guided optimization (PGO) I packaged LDC v1.0.0 with PGO for OS X: https://github.com/JohanEngelen/ldc/releases/tag/v1.0.0-beta1-pgo All your feedback is very welcome :-) cheers, Johan
Re: Web page listing all D compilers (and DMDFE version!) on travis-ci
On Tuesday, 26 April 2016 at 06:42:11 UTC, Nick Sabalausky wrote: https://semitwist.com/travis-d-compilers That's an auto-generated listing of all versions of DMD, GDC and LDC available on travis-ci. Very useful, but perhaps you can list the string one should use to tell Travis to use that version? What I mean is: currently the Name column says e.g. "ldc2-0.17.1", but in the travis.yml file you must specify "ldc-0.17.1" to get it (without the "2"). (If the string is incorrect, the install script will give an error message from which it is not clear at all that the compiler name is not recognized.) Thanks, Johan
Re: It's alive! D building D building D, all on Android
Fantastic news! I hope we can find a good way to integrate automated testing of github branches/PRs for Android. cheers, Johan
[OT] Re: pl0stuff an optimizing pl0 > c transcompiler
On Friday, 20 May 2016 at 18:04:55 UTC, Stefan Koch wrote: Update I have implemented D codegen. The CodeGenerator as well as the optimizer work at CTFE. Therefore you can transcompile code at compileTime at call PL/0 functions as there were naively implemented in D. This is pretty cool :D Now I understand why you want to improve CTFE. You actually want better C++ interop. And to do that, you are going to CTFE compile the C++ code to D code, which is then mixed in, and all is good. Excellent idea! ;-) cheers, Johan
Re: amoeba, a chess engine written in D
On Friday, 20 May 2016 at 23:16:01 UTC, Richard Delorme wrote: The source can be compiled with dmd, ldc or gdc, but the best performance are obtained with the latter (almost twice faster). Can you give cmdline details and compiler version data for this? (how much faster is gdc relative to ldc?) Thanks, Johan
Re: amoeba, a chess engine written in D
On Friday, 20 May 2016 at 23:16:01 UTC, Richard Delorme wrote: I am pleased to announce the release of a chess engine written in D: https://github.com/abulmo/amoeba In the makefile, I see you tried LDC's PGO, awesome! I hope to get (part of) it into LDC master very soon. All feedback is very welcome. (small typo: better use ldc-profdata instead of llvm-profdata) cheers, Johan
Re: amoeba, a chess engine written in D
On Saturday, 21 May 2016 at 16:25:27 UTC, Richard Delorme wrote: With the above versions & my poor knowledge of the compilers, pgo & BUILD=popcount only worked with gdc. I will try to upgrade to more recent versions to see if things changed. With LDC, you can pass "-mattr=+popcnt" to use the popcount instruction. Or "-mattr=+sse42" to enable SSE4.2 (like you seem to do in the Makefile).
Re: My ACCU 2016 keynote video available online
On Thursday, 19 May 2016 at 12:54:48 UTC, Jens Müller wrote: For example what's equivalent to gdc's -ffast-math in ldc. This is: https://github.com/ldc-developers/ldc/pull/1472 Working on performance improvements is a lot of fun. Please feed us with code that doesn't run as fast as it should! :) Johan
Re: Beta D 2.071.1-b2
On Sunday, 29 May 2016 at 21:53:23 UTC, Martin Nowak wrote: Second beta for the 2.071.1 release. http://dlang.org/download.html#dmd_beta http://dlang.org/changelog/2.071.1.html Please report any bugs at https://issues.dlang.org I've merged it into LDC, CI testers are running!
Re: Beta D 2.071.1-b2
On Monday, 30 May 2016 at 09:24:21 UTC, Johan Engelen wrote: On Sunday, 29 May 2016 at 21:53:23 UTC, Martin Nowak wrote: Second beta for the 2.071.1 release. http://dlang.org/download.html#dmd_beta http://dlang.org/changelog/2.071.1.html Please report any bugs at https://issues.dlang.org I've merged it into LDC, CI testers are running! Thumbs up! LDC master is now at 2.071.1-b2, and it can compile Weka's codebase fine (one remaining issue, but most likely not frontend related). cheers, Johan
Re: D syntax highlighting support for Jekyll websites in v1.11.0
On Monday, 6 June 2016 at 20:03:11 UTC, Borislav Kosharov wrote: Hey guys I implemented D syntax highlighting for rouge that is the default syntax highlighter for Jekyll. If you have a github pages jekyll site you will be able to post D code and get highlighting when pages moves to version 1.11.0 . Awesome! Do you know when Github Pages will upgrade, roughly? cheers, Johan
Re: LDC 1.0.0 has been released!
On Tuesday, 7 June 2016 at 16:54:55 UTC, Daniel N wrote: On Monday, 6 June 2016 at 10:45:39 UTC, Guillaume Piolat wrote: very nice! how about a howto for windows/VS users? 1) Download LDC unpack anywhere 2) Download VisualD install https://github.com/dlang/visuald/releases 3) Setup VisualD to find LDC Tools->Options->Projects and Solutions->Visual D Settings->LDC Directories ... Profit! Excellent :-) https://github.com/dlang/visuald/pull/61 (I am just "copy-pasting" there! Please create your own PR with better info! )
Re: LDC+Dub+Vibe.d work on SmartOS 64bit now
On Wednesday, 8 June 2016 at 13:41:59 UTC, Alexandr Basko wrote: On Wednesday, 8 June 2016 at 12:19:08 UTC, flamencofantasy wrote: On Wednesday, 8 June 2016 at 08:00:03 UTC, Alexandr Basko wrote: [...] Excellent! Please post a howto. I would very much like to move my server from an LX branded zone to SmartOS. Can I post howto in this thread? If yes, than i`m post it tomorrow also You could make a new page on the wiki so the Howto will be easier to locate.
Re: LDC+Dub+Vibe.d work on SmartOS 64bit now
On Wednesday, 8 June 2016 at 10:06:00 UTC, Oleg Nykytenko wrote: On Wednesday, 8 June 2016 at 08:43:59 UTC, Joakim wrote: Have you run the standard library's tests? There was a thread last summer about this: http://forum.dlang.org/thread/zhcduibirwprgbzqk...@forum.dlang.org We haven't run library's tests. What right way to run this tests? But: We ported our sufficiently big server application from Linux to SmartOS x64. All application tests is OK. Application working now on testing server in production like regime under full load. And it's working fine. We're planning to use it in production in future. Are you interested in running a buildbot for LDC on your platform to detect issues early? See: http://forum.dlang.org/thread/hikcbbqqnpvvyirxg...@forum.dlang.org Thanks, Johan
Re: Recursive SymbolNames solved.
On Thursday, 9 June 2016 at 04:58:45 UTC, Stefan Koch wrote: On Wednesday, 8 June 2016 at 13:28:19 UTC, Stefan Koch wrote: Hi, I solved the issue. PR is coming shortly. Solution is as follows: Keep a list of already visited symbols in the mangler. And information where we store the mangle. meaning the position in the mangle string. If a symbol is encountered for the second time store a backrefernce rather then the symbol name again. You mean this? https://github.com/weka-io/ldc/commit/ffd6b55a4b83b3007d9690cbd9a25d07b243a00e It did not result in a large reduction of symbol sizes. http://forum.dlang.org/post/szodxrizfmufqdkpd...@forum.dlang.org
Re: [Semi OT] About code review
On Monday, 27 June 2016 at 00:01:34 UTC, deadalnix wrote: Several people during DConf asked abut tips and tricks on code review. So I wrote an article about it: http://www.deadalnix.me/2016/06/27/on-code-review/ It's a nice read. One comment: perhaps the balance has tipped a bit much to "making a good PR", rather than "doing a good review". I feel the merit of a review is to improve the contribution, rather than to decide whether it is mergable or not. Although it is in the article, I think it could be given a little more attention: the review itself should contribute to the project, i.e. the reviewer should (try hard to) propose alternatives if something should/could be improved. Criticism is very easy, _constructive_ criticism isn't; I think the latter is needed to gain a contributor, and the first does the opposite. -Johan
Re: Release D 2.071.1
On Monday, 27 June 2016 at 22:11:53 UTC, Martin Nowak wrote: Glad to announce D 2.071.1. Thanks. LDC master is now also at 2.071.1. -Johan
Re: First dmd nightly shipping with dub
On Friday, 8 July 2016 at 20:43:58 UTC, Martin Nowak wrote: On 07/08/2016 11:22 AM, Dicebot wrote: On Friday, 8 July 2016 at 09:13:08 UTC, Martin Nowak wrote: What would be the use-case for those? Using newer dub versions with an older compiler? Or simply using dub with ldc/gdb without also installing dmd. Kai is working on shipping LDC releases with Dub too! -Johan
Re: First dmd nightly shipping with dub
When packaging dub with a compiler, it'd be nice if dub per default selected the compiler it came packaged with, I think. (it currently doesn't) Perhaps people can chime in here with ideas? https://github.com/dlang/dub/issues/895 https://github.com/dlang/dub/pull/875 Thanks, Johan
Article: "Profile-Guided Optimization with LDC"
Because you can all download binaries for LDC 1.1.0-alpha1 now, it is time to release the article on how to use profile-guided optimization with the new LDC release. Please test and report back to us! :) https://johanengelen.github.io/ldc/2016/07/15/Profile-Guided-Optimization-with-LDC.html " Three months ago, I measured a 7% performance gain of the compiler front-end D code using Profile-Guided Optimization (PGO) with LDC. Now, part of my PGO work was merged into LDC master on Jun 20, 2016! This means it will be available in the next LDC release (version 1.1.0, needs LLVM 3.7 or newer). You can play with it with the LDC 1.1.0-alpha1 release. Here I’ll discuss how to use it, saving the implementation details for another article. Using PGO with LDC is similar to using PGO with Clang, and much of this article also applies to Clang. Profile-Guided Optimization (PGO) “Profile-guided optimizations” are optimizations that the compiler can do when it is given information about the typical execution flow of your program, such that the typical execution flow of your program runs faster (and rare execution flows run slower). The typical execution flow information is the “profile” of your program: how many times was this function called, how often was a control-flow branch taken, how often was function A called by function B, what are likely values of variable X, how much memory is usually allocated for something, etc. ... "
Re: Beta D 2.071.2-b1
On Monday, 1 August 2016 at 11:02:41 UTC, Martin Nowak wrote: First beta for the 2.071.2 point release. Green on all testers: merged into LDC master! cheers, Johan
Re: ndBenchmarks #1: ndslice.algorithm vs std.numeric vs std.algorithm
On Wednesday, 3 August 2016 at 20:53:59 UTC, Ilya Yaroshenko wrote: Hi all, There are two first [1] benchmarks for upcoming ndslice.algorithm [2]. Recent LDC alpha based on LLVM 3.8 and recent Mir v0.16.0-alpha3 are required. @fasmath syntax may be changed a little bit and will be simplified anyway. Dot Product: ndReduce vectorized = 3 ms, 314 μs ndReduce = 14 ms, 767 μs **That's** the difference with or without fastmath?? (awesome work of course!) -Johan
Re: Running a D game in the browser
On Friday, 5 August 2016 at 05:39:57 UTC, Sebastien Alaiwan wrote: On Thursday, 4 August 2016 at 19:17:34 UTC, Sebastien Alaiwan wrote: at the moment, I have a patch to making the build work (only for the binary "ldc2", not other tools of the package). I created a dedicated github branch "fastcomp-ldc". The patch: https://github.com/Ace17/dscripten/blob/fastcomp-ldc/ldc2.patch That patch doesn't look too bad. Could you introduce a CMake option for building with Emscripten-fastcomp? And a #define "LDC_LLVM_EMSCRIPTEN" or something like that, so that you can change `#if LDC_LLVM_VER >= 309 && 0` to `#if LDC_LLVM_VER >= 309 && !LDC_LLVM_EMSCRIPTEN`. Should be mergable into LDC master then! cheers, Johan
Re: Battle-plan for CTFE
On Sunday, 7 August 2016 at 16:52:28 UTC, Stefan Koch wrote: On Saturday, 6 August 2016 at 23:04:48 UTC, Stefan Koch wrote: No, not now, but very soon. I want to have _basic_ utf8 support before I am comfortable with enabling string operations. Tomorrow this is going to work. Hi Stefan, How much effort will it be to integrate your work into LDC? I'd like to test it on some Weka code. Definitely let us know once you can CTFE something like std.traits.ParameterStorageClassTuple. cheers, Johan
Re: Beta D 2.071.2-b2
On Tuesday, 9 August 2016 at 15:37:27 UTC, Martin Nowak wrote: Second beta for the 2.071.2 release. Nice. Merged into LDC master. -Johan
Re: Beta D 2.071.2-b2
On Tuesday, 9 August 2016 at 15:37:27 UTC, Martin Nowak wrote: Please report any bugs at https://issues.dlang.org I'd appreciate it if someone could have a look at this regression: https://issues.dlang.org/show_bug.cgi?id=16369
Re: [GSoC] Mir.random.flex - Generic non-uniform random sampling
On Tuesday, 23 August 2016 at 13:01:29 UTC, Seb wrote: On Tuesday, 23 August 2016 at 12:12:30 UTC, Seb wrote: [...] I fixed that and also added how many samples were generated per run (10M), thanks! Btw I quickly added the run-time for C++ (for G++ and clang++ with -O3) I think you should add compiler/runtimelib versions for all measurements, and all relevant flags. I'm happy to see you are using LDC for benchmarking; don't forget to mention the LLVM version for LDC's version. You already wrote `-O3` for the C++ measurements, did you not use -mcpu=native or some other performance changing flags? Cheers, Johan
Re: Beta D 2.071.2-b3
On Tuesday, 30 August 2016 at 23:54:45 UTC, Martin Nowak wrote: Allowing access to private members has a lot of implications, e.g. breaks lots of optimizations b/c you can't know who accesses sth. "lots of optimizations" Can you mention a few? (I can only think of complicated stuff that requires pretty much whole-program analysis to prove validity, and in that case adding `private` doesn't help any more) -Johan
Re: Beta D 2.071.2-b3
On Thursday, 1 September 2016 at 20:46:50 UTC, Steven Schveighoffer wrote: On 8/31/16 5:56 AM, Johan Engelen wrote: On Tuesday, 30 August 2016 at 23:54:45 UTC, Martin Nowak wrote: Allowing access to private members has a lot of implications, e.g. breaks lots of optimizations b/c you can't know who accesses sth. "lots of optimizations" Can you mention a few? I'm not sure optimizations are had by private variables. Some optimizations can be had because the compiler can prove a variable could not have been changed. But I don't know that private suits that need, it's more due to const or immutable. Yep. I was trying to be polite :-)
Re: Beta D 2.071.2-b3
On Saturday, 3 September 2016 at 18:06:37 UTC, Dicebot wrote: On Saturday, 3 September 2016 at 14:40:37 UTC, Martin Nowak wrote: On Wednesday, 31 August 2016 at 09:56:17 UTC, Johan Engelen wrote: (I can only think of complicated stuff that requires pretty much whole-program analysis to prove validity, and in that case adding `private` doesn't help any more) Yes, it does help. As private prevents usage outside of a module it allows to do some optimizations that required whole program analysis otherwise, e.g. variables and functions can get internal linkage, thus reducing loads/stores and indirect calls. This is a very naive viewpoint. There are enough ways to leak access to private entity (alias, template argument, taking address, some compile-time introspection) to make such optimizations impossible without extensive program analysis. Exactly.
Re: Beta D 2.071.2-b4
On Monday, 12 September 2016 at 07:47:19 UTC, Martin Nowak wrote: Fourth beta for the 2.071.2 release. This comes with a different fix for Issue 15907 than 2.071.2-b3. http://dlang.org/changelog/2.071.2.html I think the changelog has not been updated yet?
Re: Beta D 2.071.2-b4
On Tuesday, 13 September 2016 at 14:00:26 UTC, Martin Nowak wrote: On Monday, 12 September 2016 at 07:47:19 UTC, Martin Nowak wrote: This comes with a different fix for Issue 15907 than 2.071.2-b3. There will be another beta tomorrow or so to include at least one more fix (for Issue 16460) and we'll soon release 2.071.2. I have merged 2.071.2-b4+the fix for issue 16460 into LDC master. As far as deprecations go, it's looking good on Weka's codebase. cheers, Johan
Re: Beta D 2.071.2-b5
On Wednesday, 14 September 2016 at 12:16:51 UTC, Martin Nowak wrote: Fifth and hopefully last beta for the 2.071.2 release. This comes with two more fixes for Issue 16031 and 16460. LDC master is up-to-date with 2.071.2-b5!
LDC: Speed up incremental builds with object file caching
I just finished another post about LDC: https://johanengelen.github.io/ldc/2016/09/17/LDC-object-file-caching.html Thanks in advance for letting me know about any bugs you find, in the text or in the code :) -Johan
Re: LDC: Speed up incremental builds with object file caching
On Saturday, 17 September 2016 at 19:17:44 UTC, Daniel Kozak wrote: Nice article, however font on this site is too small to read, I have to zoom in :( Do more people have this problem? For me, the font size is similar to, say, Github.
Re: LDC: Speed up incremental builds with object file caching
On Sunday, 18 September 2016 at 08:14:47 UTC, Martin Nowak wrote: Interesting approach to speed up compilation without running into dmd's problems of template instance emission when compiling multiple modules to multiple objects. I think LDC has the same problems (template instances are emitted in the first module not the one that needed the instantiation?). Knowing this, you may be able to set up a case where things break, but I think it would have to involve recompilation with a different set of sources than the first compile. E.g. `ldc2 -c a.d b.d c.d` first, then `ldc2 -c b.d c.d`, and then trying to link. Which probably doesn't work well without caching either... We should deprecate non-singleobj compiles. Many people use noatime or relatime when mounting their filesystems, so access time isn't the best eviction strategy unless you touch the files. I forgot to mention this in the article, thanks! I read that a file's last access timestamp may also not be updated on Windows (default Windows 7 setting), so I already had LDC `touch` the cached file upon a cache hit. Added it as a footnote in the article.
Re: LDC: Speed up incremental builds with object file caching
On Sunday, 18 September 2016 at 12:15:12 UTC, Daniel Kozak wrote: But on my android phone it is OK. So maybe I should bought some 2k or 4k monitor ;) Definitely one of the best upgrades I ever did (4k monitor). Warning: there's no way back.
Re: LDC 1.1.0-beta3 has been released!
On Tuesday, 11 October 2016 at 07:29:00 UTC, Sönke Ludwig wrote: Just noticed that the release binaries are missing from https://github.com/ldc-developers/ldc/releases/tag/v1.1.0-beta3 This means that the beta currently cannot be tested with TravisCI (wanted to test a DUB related regression fix). Whoops, that's my bad :( (I editted a little and clicked the "save draft" button which turned it into a draft again, I think) Thanks for letting us know! Fixed it. I also updated LATEST_BETA, so `ldc-beta` on Travis should now automatically use 1.1.0-beta3. -Johan