Re: Release D 2.077.0
On Saturday, 4 November 2017 at 08:19:17 UTC, Walter Bright wrote: On 11/3/2017 1:20 PM, Dmitry Olshansky wrote: Sadly array ops would be insufficient for said problem. It wasn’t a direct element wise expression. That sounds like that might be why it failed vectorization :-) As I recall it there were no trivial loops there. Usually these 2 magicians could make compiler eat it in a few hours of shuffling the code. They vectorized about half a dozen loops that way. The last one took 10 times more then the others taken together ;) If you recall the expression, it would be interesting to see it. Even if I had it saved somewhere the place was NDA-ed to death. I traded 3 months of intellectual work (and property) for a modest amount of money. Interesting experience but no illusions about R&D centers anymore.
Re: Release D 2.077.0
On 11/3/2017 1:20 PM, Dmitry Olshansky wrote: Sadly array ops would be insufficient for said problem. It wasn’t a direct element wise expression. That sounds like that might be why it failed vectorization :-) If you recall the expression, it would be interesting to see it.
Re: Release D 2.077.0
On Friday, 3 November 2017 at 14:00:38 UTC, Arun Chandrasekaran wrote: Mike, thanks for the blog post. Few lines about how the name mangling issue was addressed would've been interesting know on the blog. The regarding main PR contains a lot of info: https://github.com/dlang/dmd/pull/5855 For example, some quick stats on the reduced size of Phobos: https://github.com/dlang/dmd/pull/5855#issuecomment-315565256
Re: Release D 2.077.0
On Friday, 3 November 2017 at 19:46:58 UTC, Walter Bright wrote: On 11/3/2017 3:02 AM, Mike Parker wrote: For clarity, where the changeling says that GDC & LDC use auto-vectorization, that's actually happening with the array operations and core.simd is not required, correct? I think that GDC and LDC do do auto-vectorization, but I haven't verified it myself. Auto-vectorization is a fundamentally bizarre feature. It takes low level code and reverse-engineers it back into a higher level construct, and then proceeds to generate code for that higher level construct. Everything else a compiler does is start from a high level construct and then generate low level code. The trouble with AV is whether it succeeds or not depends on peculiarities (and I mean peculiarities) of the particular vector instruction set target. It can decided to not vectorize based on seemingly trivial and innocuous changes to the loop. I’ll share an anecdotal experience from a time I worked in reasearch lab of a well known tech giant. 2 senior researchers spent literally 2 weeks trying to coerce compiler into vectorizing an inner loop of a non-trivial matrix algorithm. The only diagnostic from compiler was “loop form is not correct”. Thankfully it did tell them it failed, else they’d have to disassemble it each time. I think eventually they either rewritten it to fit heuristic or just carried on with explicit intrinsics. What's needed is a language feature that is straightforwardly vectorizable. That would be D's array operations. Sadly array ops would be insufficient for said problem. It wasn’t a direct element wise expression.
Re: Release D 2.077.0
On 11/3/2017 3:02 AM, Mike Parker wrote: For clarity, where the changeling says that GDC & LDC use auto-vectorization, that's actually happening with the array operations and core.simd is not required, correct? I think that GDC and LDC do do auto-vectorization, but I haven't verified it myself. Auto-vectorization is a fundamentally bizarre feature. It takes low level code and reverse-engineers it back into a higher level construct, and then proceeds to generate code for that higher level construct. Everything else a compiler does is start from a high level construct and then generate low level code. The trouble with AV is whether it succeeds or not depends on peculiarities (and I mean peculiarities) of the particular vector instruction set target. It can decided to not vectorize based on seemingly trivial and innocuous changes to the loop. The only way to tell is to benchmark it or look at the object file - methods that are unreliable (benchmarking) or do not scale (manually looking at the object file). What's needed is a language feature that is straightforwardly vectorizable. That would be D's array operations.
Re: Release D 2.077.0
I have SIGSEGV when using DMD and simd types. This code works ok with GDC and LDC fine, but SIGSEGV with latest DMD (maybe even with previous versions I do not know) https://dpaste.dzfl.pl/5476f5bef828 On Fri, Nov 3, 2017 at 5:04 PM, Steven Schveighoffer via Digitalmars-d-announce wrote: > On 11/3/17 10:00 AM, Arun Chandrasekaran wrote: > >> On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote: >> >>> On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote: >>> Glad to announce D 2.077.0. This release comes with a new, more compact mangling, templated vector operations, reproducible dmd builds, and various fixes. Thanks to everyone involved in this 👏. http://downloads.dlang.org/releases/2.x/2.077.0/ http://dlang.org/changelog/2.077.0.html The dlang.org website will get updated soon. -Martin >>> >>> Blog: >>> https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/ >>> >> >> Mike, thanks for the blog post. Few lines about how the name mangling >> issue was addressed would've been interesting know on the blog. >> > > A blog post I wrote about the issue itself (and a workaround that I > employed to achieve the same result) is here: > > http://www.schveiguy.com/blog/2016/05/have-your-voldemort-ty > pes-and-keep-your-disk-space-too/ > > I hope Rainer agrees to the blog post as well. While I understand the > concept, a detailed description of how the back references work would be > very interesting. > > -Steve >
Re: Release D 2.077.0
On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote: On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote: Glad to announce D 2.077.0. This release comes with a new, more compact mangling, templated vector operations, reproducible dmd builds, and various fixes. Thanks to everyone involved in this 👏. http://downloads.dlang.org/releases/2.x/2.077.0/ http://dlang.org/changelog/2.077.0.html The dlang.org website will get updated soon. -Martin Blog: https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/ Reddit: https://www.reddit.com/r/programming/comments/7ajg71/dmd_20770_released/ Typo: particulary "case so that," -> "case, so that" (I'd also remove the comma after that)
Re: Release D 2.077.0
On 11/3/17 10:00 AM, Arun Chandrasekaran wrote: On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote: On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote: Glad to announce D 2.077.0. This release comes with a new, more compact mangling, templated vector operations, reproducible dmd builds, and various fixes. Thanks to everyone involved in this 👏. http://downloads.dlang.org/releases/2.x/2.077.0/ http://dlang.org/changelog/2.077.0.html The dlang.org website will get updated soon. -Martin Blog: https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/ Mike, thanks for the blog post. Few lines about how the name mangling issue was addressed would've been interesting know on the blog. A blog post I wrote about the issue itself (and a workaround that I employed to achieve the same result) is here: http://www.schveiguy.com/blog/2016/05/have-your-voldemort-types-and-keep-your-disk-space-too/ I hope Rainer agrees to the blog post as well. While I understand the concept, a detailed description of how the back references work would be very interesting. -Steve
Re: Release D 2.077.0
On Friday, 3 November 2017 at 14:41:06 UTC, Gerald wrote: On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote: Glad to announce D 2.077.0. ... The new version is much better with only a 6 MB difference between the stripped and non-stripped versions. My vibe-d application debug build reduced from 56Mb to 44Mb (release = 19Mb).
Re: Release D 2.077.0
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote: Glad to announce D 2.077.0. This release comes with a new, more compact mangling, templated vector operations, reproducible dmd builds, and various fixes. Thanks to everyone involved in this 👏. http://downloads.dlang.org/releases/2.x/2.077.0/ http://dlang.org/changelog/2.077.0.html The dlang.org website will get updated soon. -Martin Great release, in Tilix the dmd executable was quite large due to all the symbols generated in GtkD for event handling (mea culpa since I did that PR). The new version is much better with only a 6 MB difference between the stripped and non-stripped versions.
Re: Release D 2.077.0
On Friday, 3 November 2017 at 14:00:38 UTC, Arun Chandrasekaran wrote: Mike, thanks for the blog post. Few lines about how the name mangling issue was addressed would've been interesting know on the blog. There's a link in the post to the documentation describing the enhancement. As for how Rainer settled on that solution, I'm hoping to get a guest post out of him (though I haven't asked him yet, so s!). [1] https://dlang.org/spec/abi.html#back_ref
Re: Release D 2.077.0
On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote: On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote: Glad to announce D 2.077.0. This release comes with a new, more compact mangling, templated vector operations, reproducible dmd builds, and various fixes. Thanks to everyone involved in this 👏. http://downloads.dlang.org/releases/2.x/2.077.0/ http://dlang.org/changelog/2.077.0.html The dlang.org website will get updated soon. -Martin Blog: https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/ Mike, thanks for the blog post. Few lines about how the name mangling issue was addressed would've been interesting know on the blog. Reddit: https://www.reddit.com/r/programming/comments/7ajg71/dmd_20770_released/
Re: Release D 2.077.0
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote: Glad to announce D 2.077.0. This release comes with a new, more compact mangling, templated vector operations, reproducible dmd builds, and various fixes. Thanks to everyone involved in this 👏. http://downloads.dlang.org/releases/2.x/2.077.0/ http://dlang.org/changelog/2.077.0.html The dlang.org website will get updated soon. -Martin Blog: https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/ Reddit: https://www.reddit.com/r/programming/comments/7ajg71/dmd_20770_released/
Re: Release D 2.077.0
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote: Glad to announce D 2.077.0. This release comes with a new, more compact mangling, templated vector operations, reproducible dmd builds, and various fixes. Thanks to everyone involved in this 👏. http://downloads.dlang.org/releases/2.x/2.077.0/ http://dlang.org/changelog/2.077.0.html The dlang.org website will get updated soon. -Martin It's time to update documentation on devdocs
Re: Release D 2.077.0
On Friday, 3 November 2017 at 10:14:27 UTC, Joakim wrote: See the linked druntime pull, core.simd is only imported for dmd: https://github.com/dlang/druntime/pull/1891/files#diff-c17bbc97c8719ab709a4a54e2f6924ceR67 Ah, I see. I misunderstood Walter to be saying the user needed core.simd to get the vectorization. Thanks!
Re: Release D 2.077.0
On Friday, 3 November 2017 at 09:28:37 UTC, Daniel Kozak wrote: How should I compile my program to enable array vectorization? Array operations refers to https://dlang.org/spec/arrays.html#array-operations. I have tried dmd -march=native, -march=avx2 as changlog suggest It's -mcpu= not -march= for dmd, my bad. Unfortunate that dmd uses different switches than gcc. If you're compiling for 64-bit, you'll get SSE2 by default.
Re: Release D 2.077.0
On Friday, 3 November 2017 at 10:07:25 UTC, Mike Parker wrote: On Friday, 3 November 2017 at 10:02:18 UTC, Mike Parker wrote: [...] OK, I'm a bit confused here. This gives the impression that the vectorization happens automatically with array operations: "Array operations have been converted from dedicated assembly routines for some array operations to a generic template implementation for all array operations. This provides huge performance increases (2-4x higher throughput) for array operations that were not previously vectorized. Furthermore the implementation makes better use of vectorization even for short arrays to heavily reduce latency for some operations (up to 4x)." Where does core.simd fit in? See the linked druntime pull, core.simd is only imported for dmd: https://github.com/dlang/druntime/pull/1891/files#diff-c17bbc97c8719ab709a4a54e2f6924ceR67
Re: Release D 2.077.0
On Friday, 3 November 2017 at 10:02:18 UTC, Mike Parker wrote: On Friday, 3 November 2017 at 09:33:31 UTC, Walter Bright wrote: On 11/3/2017 2:28 AM, Daniel Kozak wrote: How should I compile my program to enable array vectorization? dmd doesn't do what is known as "auto-vectorization". https://en.wikipedia.org/wiki/Automatic_vectorization What D does is have vector data types, and when those are used vector instructions are generated for them. https://dlang.org/spec/simd.html For clarity, where the changeling says that GDC & LDC use auto-vectorization, that's actually happening with the array operations and core.simd is not required, correct? Yes, at least with ldc.
Re: Release D 2.077.0
On Friday, 3 November 2017 at 10:02:18 UTC, Mike Parker wrote: On Friday, 3 November 2017 at 09:33:31 UTC, Walter Bright wrote: On 11/3/2017 2:28 AM, Daniel Kozak wrote: How should I compile my program to enable array vectorization? dmd doesn't do what is known as "auto-vectorization". https://en.wikipedia.org/wiki/Automatic_vectorization What D does is have vector data types, and when those are used vector instructions are generated for them. https://dlang.org/spec/simd.html For clarity, where the changeling says that GDC & LDC use auto-vectorization, that's actually happening with the array operations and core.simd is not required, correct? OK, I'm a bit confused here. This gives the impression that the vectorization happens automatically with array operations: "Array operations have been converted from dedicated assembly routines for some array operations to a generic template implementation for all array operations. This provides huge performance increases (2-4x higher throughput) for array operations that were not previously vectorized. Furthermore the implementation makes better use of vectorization even for short arrays to heavily reduce latency for some operations (up to 4x)." Where does core.simd fit in?
Re: Release D 2.077.0
On Friday, 3 November 2017 at 09:33:31 UTC, Walter Bright wrote: On 11/3/2017 2:28 AM, Daniel Kozak wrote: How should I compile my program to enable array vectorization? dmd doesn't do what is known as "auto-vectorization". https://en.wikipedia.org/wiki/Automatic_vectorization What D does is have vector data types, and when those are used vector instructions are generated for them. https://dlang.org/spec/simd.html For clarity, where the changeling says that GDC & LDC use auto-vectorization, that's actually happening with the array operations and core.simd is not required, correct?
Re: Release D 2.077.0
On 11/3/2017 2:28 AM, Daniel Kozak wrote: How should I compile my program to enable array vectorization? dmd doesn't do what is known as "auto-vectorization". https://en.wikipedia.org/wiki/Automatic_vectorization What D does is have vector data types, and when those are used vector instructions are generated for them. https://dlang.org/spec/simd.html
Re: Release D 2.077.0
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote: Glad to announce D 2.077.0. This release comes with a new, more compact mangling, templated vector operations, reproducible dmd builds, and various fixes. Thanks to everyone involved in this 👏. http://downloads.dlang.org/releases/2.x/2.077.0/ http://dlang.org/changelog/2.077.0.html The dlang.org website will get updated soon. -Martin How should I compile my program to enable array vectorization? I have tried dmd -march=native, -march=avx2 as changlog suggest but does not work I have tried even just -march=native or -march=avx2 but still does not compile Error: unrecognized switch '-march=avx2' Error: unrecognized switch '-march=native'
Re: Release D 2.077.0
On 11/02/2017 06:35 PM, Martin Nowak wrote: reproducible dmd builds, Curiosity: What exactly was preventing this before? Order of source files altering the order of output? Something else? Also, just musing...Regarding the matter of __TIME__(etc) breaking this guarantee (breaking it for obvious reasons). Seems to me it would be helpful to have compiler switches to force lexer symbols to specific values in order to bring at least some level of reproducablity even to projects that do use such symbols.
Release D 2.077.0
Glad to announce D 2.077.0. This release comes with a new, more compact mangling, templated vector operations, reproducible dmd builds, and various fixes. Thanks to everyone involved in this 👏. http://downloads.dlang.org/releases/2.x/2.077.0/ http://dlang.org/changelog/2.077.0.html The dlang.org website will get updated soon. -Martin