Re: Release D 2.077.0

[Dmitry Olshansky via Digitalmars-d-announce]

On Saturday, 4 November 2017 at 08:19:17 UTC, Walter Bright wrote:

On 11/3/2017 1:20 PM, Dmitry Olshansky wrote:
Sadly array ops would be insufficient for said problem. It 
wasn’t a direct element wise expression.


That sounds like that might be why it failed vectorization :-)


As I recall it there were no trivial loops there. Usually these 2 
magicians could make compiler eat it in a few hours of shuffling 
the code. They vectorized about half a dozen loops that way.


The last one took 10 times more then the others taken together ;)



If you recall the expression, it would be interesting to see it.


Even if I had it saved somewhere the place was NDA-ed to death. I 
traded 3 months of intellectual work (and property) for a modest 
amount of money. Interesting experience but no illusions about 
R centers anymore.

Re: Release D 2.077.0

[Walter Bright via Digitalmars-d-announce]

On 11/3/2017 1:20 PM, Dmitry Olshansky wrote:
Sadly array ops would be insufficient for said problem. It wasn’t a direct 
element wise expression.


That sounds like that might be why it failed vectorization :-)

If you recall the expression, it would be interesting to see it.

Re: Release D 2.077.0

[Seb via Digitalmars-d-announce]

On Friday, 3 November 2017 at 14:00:38 UTC, Arun Chandrasekaran 
wrote:
Mike, thanks for the blog post. Few lines about how the name 
mangling issue was addressed would've been interesting know on 
the blog.


The regarding main PR contains a lot of info: 
https://github.com/dlang/dmd/pull/5855

For example, some quick stats on the reduced size of Phobos:

https://github.com/dlang/dmd/pull/5855#issuecomment-315565256

Re: Release D 2.077.0

[Dmitry Olshansky via Digitalmars-d-announce]

On Friday, 3 November 2017 at 19:46:58 UTC, Walter Bright wrote:

On 11/3/2017 3:02 AM, Mike Parker wrote:
For clarity, where the changeling says that GDC & LDC use 
auto-vectorization, that's actually happening with the array 
operations and core.simd is not required, correct?


I think that GDC and LDC do do auto-vectorization, but I 
haven't verified it myself.


Auto-vectorization is a fundamentally bizarre feature. It takes 
low level code and reverse-engineers it back into a higher 
level construct, and then proceeds to generate code for that 
higher level construct.


Everything else a compiler does is start from a high level 
construct and then generate low level code.


The trouble with AV is whether it succeeds or not depends on 
peculiarities (and I mean peculiarities) of the particular 
vector instruction set target. It can decided to not vectorize 
based on seemingly trivial and innocuous changes to the loop.


I’ll share an anecdotal experience from a time I worked in 
reasearch lab of a well known tech giant. 2 senior researchers 
spent literally 2 weeks trying to coerce compiler into 
vectorizing an inner loop of a non-trivial matrix algorithm.


The only diagnostic from compiler was “loop form is not correct”. 
Thankfully it did tell them it failed, else they’d have to 
disassemble it each time.


I think eventually they either rewritten it to fit heuristic or 
just carried on with explicit intrinsics.



What's needed is a language feature that is straightforwardly 
vectorizable. That would be D's array operations.


Sadly array ops would be insufficient for said problem. It wasn’t 
a direct element wise expression.

Re: Release D 2.077.0

[Walter Bright via Digitalmars-d-announce]

On 11/3/2017 3:02 AM, Mike Parker wrote:
For clarity, where the changeling says that GDC & LDC use auto-vectorization, 
that's actually happening with the array operations and core.simd is not 
required, correct?


I think that GDC and LDC do do auto-vectorization, but I haven't verified it 
myself.

Auto-vectorization is a fundamentally bizarre feature. It takes low level code 
and reverse-engineers it back into a higher level construct, and then proceeds 
to generate code for that higher level construct.


Everything else a compiler does is start from a high level construct and then 
generate low level code.


The trouble with AV is whether it succeeds or not depends on peculiarities (and 
I mean peculiarities) of the particular vector instruction set target. It can 
decided to not vectorize based on seemingly trivial and innocuous changes to the 
loop. The only way to tell is to benchmark it or look at the object file - 
methods that are unreliable (benchmarking) or do not scale (manually looking at 
the object file).


What's needed is a language feature that is straightforwardly vectorizable. That 
would be D's array operations.

Re: Release D 2.077.0

[Daniel Kozak via Digitalmars-d-announce]

I have SIGSEGV when using DMD and simd types. This code works ok with GDC
and LDC fine, but SIGSEGV with latest DMD (maybe even with previous
versions I do not know)

https://dpaste.dzfl.pl/5476f5bef828

On Fri, Nov 3, 2017 at 5:04 PM, Steven Schveighoffer via
Digitalmars-d-announce  wrote:

> On 11/3/17 10:00 AM, Arun Chandrasekaran wrote:
>
>> On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote:
>>
>>> On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:
>>>
 Glad to announce D 2.077.0.

 This release comes with a new, more compact mangling, templated vector
 operations, reproducible dmd builds, and various fixes.
 Thanks to everyone involved in this .

 http://downloads.dlang.org/releases/2.x/2.077.0/
 http://dlang.org/changelog/2.077.0.html
 The dlang.org website will get updated soon.

 -Martin

>>>
>>> Blog:
>>> https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/
>>>
>>
>> Mike, thanks for the blog post. Few lines about how the name mangling
>> issue was addressed would've been interesting know on the blog.
>>
>
> A blog post I wrote about the issue itself (and a workaround that I
> employed to achieve the same result) is here:
>
> http://www.schveiguy.com/blog/2016/05/have-your-voldemort-ty
> pes-and-keep-your-disk-space-too/
>
> I hope Rainer agrees to the blog post as well. While I understand the
> concept, a detailed description of how the back references work would be
> very interesting.
>
> -Steve
>

Re: Release D 2.077.0

[Joakim via Digitalmars-d-announce]

On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote:
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak 
wrote:

Glad to announce D 2.077.0.

This release comes with a new, more compact mangling, 
templated vector operations, reproducible dmd builds, and 
various fixes.

Thanks to everyone involved in this .

http://downloads.dlang.org/releases/2.x/2.077.0/
http://dlang.org/changelog/2.077.0.html
The dlang.org website will get updated soon.

-Martin


Blog:
https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/

Reddit:
https://www.reddit.com/r/programming/comments/7ajg71/dmd_20770_released/


Typo: particulary

"case so that," -> "case, so that" (I'd also remove the comma 
after that)

Re: Release D 2.077.0

[Steven Schveighoffer via Digitalmars-d-announce]

On 11/3/17 10:00 AM, Arun Chandrasekaran wrote:

On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote:

On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:

Glad to announce D 2.077.0.

This release comes with a new, more compact mangling, templated 
vector operations, reproducible dmd builds, and various fixes.

Thanks to everyone involved in this .

http://downloads.dlang.org/releases/2.x/2.077.0/
http://dlang.org/changelog/2.077.0.html
The dlang.org website will get updated soon.

-Martin


Blog:
https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/


Mike, thanks for the blog post. Few lines about how the name mangling 
issue was addressed would've been interesting know on the blog.


A blog post I wrote about the issue itself (and a workaround that I 
employed to achieve the same result) is here:


http://www.schveiguy.com/blog/2016/05/have-your-voldemort-types-and-keep-your-disk-space-too/

I hope Rainer agrees to the blog post as well. While I understand the 
concept, a detailed description of how the back references work would be 
very interesting.


-Steve

Re: Release D 2.077.0

[crimaniak via Digitalmars-d-announce]

On Friday, 3 November 2017 at 14:41:06 UTC, Gerald wrote:
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak 
wrote:

Glad to announce D 2.077.0.


... The new version is much better with only a 6 MB difference 
between the stripped and non-stripped versions.


My vibe-d application debug build reduced from 56Mb to 44Mb 
(release = 19Mb).

Re: Release D 2.077.0

[Gerald via Digitalmars-d-announce]

On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:

Glad to announce D 2.077.0.

This release comes with a new, more compact mangling, templated 
vector operations, reproducible dmd builds, and various fixes.

Thanks to everyone involved in this .

http://downloads.dlang.org/releases/2.x/2.077.0/
http://dlang.org/changelog/2.077.0.html
The dlang.org website will get updated soon.

-Martin


Great release, in Tilix the dmd executable was quite large due to 
all the symbols generated in GtkD for event handling (mea culpa 
since I did that PR). The new version is much better with only a 
6 MB difference between the stripped and non-stripped versions.

Re: Release D 2.077.0

[Mike Parker via Digitalmars-d-announce]

On Friday, 3 November 2017 at 14:00:38 UTC, Arun Chandrasekaran 
wrote:


Mike, thanks for the blog post. Few lines about how the name 
mangling issue was addressed would've been interesting know on 
the blog.






There's a link in the post to the documentation describing the 
enhancement. As for how Rainer settled on that solution, I'm 
hoping to get a guest post out of him (though I haven't asked him 
yet, so s!).


[1] https://dlang.org/spec/abi.html#back_ref

Re: Release D 2.077.0

[Arun Chandrasekaran via Digitalmars-d-announce]

On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote:
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak 
wrote:

Glad to announce D 2.077.0.

This release comes with a new, more compact mangling, 
templated vector operations, reproducible dmd builds, and 
various fixes.

Thanks to everyone involved in this .

http://downloads.dlang.org/releases/2.x/2.077.0/
http://dlang.org/changelog/2.077.0.html
The dlang.org website will get updated soon.

-Martin


Blog:
https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/


Mike, thanks for the blog post. Few lines about how the name 
mangling issue was addressed would've been interesting know on 
the blog.




Reddit:
https://www.reddit.com/r/programming/comments/7ajg71/dmd_20770_released/

Re: Release D 2.077.0

[Mike Parker via Digitalmars-d-announce]

On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:

Glad to announce D 2.077.0.

This release comes with a new, more compact mangling, templated 
vector operations, reproducible dmd builds, and various fixes.

Thanks to everyone involved in this .

http://downloads.dlang.org/releases/2.x/2.077.0/
http://dlang.org/changelog/2.077.0.html
The dlang.org website will get updated soon.

-Martin


Blog:
https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/

Reddit:
https://www.reddit.com/r/programming/comments/7ajg71/dmd_20770_released/

Re: Release D 2.077.0

[ANtlord via Digitalmars-d-announce]

On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:

Glad to announce D 2.077.0.

This release comes with a new, more compact mangling, templated 
vector operations, reproducible dmd builds, and various fixes.

Thanks to everyone involved in this .

http://downloads.dlang.org/releases/2.x/2.077.0/
http://dlang.org/changelog/2.077.0.html
The dlang.org website will get updated soon.

-Martin


It's time to update documentation on devdocs

Re: Release D 2.077.0

[Mike Parker via Digitalmars-d-announce]

On Friday, 3 November 2017 at 10:14:27 UTC, Joakim wrote:
See the linked druntime pull, core.simd is only imported for 
dmd:


https://github.com/dlang/druntime/pull/1891/files#diff-c17bbc97c8719ab709a4a54e2f6924ceR67


Ah, I see. I misunderstood Walter to be saying the user needed 
core.simd to get the vectorization. Thanks!

Re: Release D 2.077.0

[Martin Nowak via Digitalmars-d-announce]

On Friday, 3 November 2017 at 09:28:37 UTC, Daniel Kozak wrote:

How should I compile my program to enable array vectorization?


Array operations refers to 
https://dlang.org/spec/arrays.html#array-operations.



I have tried dmd -march=native, -march=avx2 as changlog suggest


It's -mcpu= not -march= for dmd, my bad.
Unfortunate that dmd uses different switches than gcc.
If you're compiling for 64-bit, you'll get SSE2 by default.

Re: Release D 2.077.0

[Joakim via Digitalmars-d-announce]

On Friday, 3 November 2017 at 10:07:25 UTC, Mike Parker wrote:

On Friday, 3 November 2017 at 10:02:18 UTC, Mike Parker wrote:

[...]


OK, I'm a bit confused here. This gives the impression that the 
vectorization happens automatically with array operations:


"Array operations have been converted from dedicated assembly 
routines for some array operations to a generic template 
implementation for all array operations. This provides huge 
performance increases (2-4x higher throughput) for array 
operations that were not previously vectorized. Furthermore the 
implementation makes better use of vectorization even for short 
arrays to heavily reduce latency for some operations (up to 
4x)."


Where does core.simd fit in?


See the linked druntime pull, core.simd is only imported for dmd:

https://github.com/dlang/druntime/pull/1891/files#diff-c17bbc97c8719ab709a4a54e2f6924ceR67

Re: Release D 2.077.0

[Joakim via Digitalmars-d-announce]

On Friday, 3 November 2017 at 10:02:18 UTC, Mike Parker wrote:

On Friday, 3 November 2017 at 09:33:31 UTC, Walter Bright wrote:

On 11/3/2017 2:28 AM, Daniel Kozak wrote:

How should I compile my program to enable array vectorization?


dmd doesn't do what is known as "auto-vectorization".

https://en.wikipedia.org/wiki/Automatic_vectorization

What D does is have vector data types, and when those are used 
vector instructions are generated for them.


https://dlang.org/spec/simd.html


For clarity, where the changeling says that GDC & LDC use 
auto-vectorization, that's actually happening with the array 
operations and core.simd is not required, correct?


Yes, at least with ldc.

Re: Release D 2.077.0

[Mike Parker via Digitalmars-d-announce]

On Friday, 3 November 2017 at 10:02:18 UTC, Mike Parker wrote:

On Friday, 3 November 2017 at 09:33:31 UTC, Walter Bright wrote:

On 11/3/2017 2:28 AM, Daniel Kozak wrote:

How should I compile my program to enable array vectorization?


dmd doesn't do what is known as "auto-vectorization".

https://en.wikipedia.org/wiki/Automatic_vectorization

What D does is have vector data types, and when those are used 
vector instructions are generated for them.


https://dlang.org/spec/simd.html


For clarity, where the changeling says that GDC & LDC use 
auto-vectorization, that's actually happening with the array 
operations and core.simd is not required, correct?


OK, I'm a bit confused here. This gives the impression that the 
vectorization happens automatically with array operations:


"Array operations have been converted from dedicated assembly 
routines for some array operations to a generic template 
implementation for all array operations. This provides huge 
performance increases (2-4x higher throughput) for array 
operations that were not previously vectorized. Furthermore the 
implementation makes better use of vectorization even for short 
arrays to heavily reduce latency for some operations (up to 4x)."


Where does core.simd fit in?

Re: Release D 2.077.0

[Mike Parker via Digitalmars-d-announce]

On Friday, 3 November 2017 at 09:33:31 UTC, Walter Bright wrote:

On 11/3/2017 2:28 AM, Daniel Kozak wrote:

How should I compile my program to enable array vectorization?


dmd doesn't do what is known as "auto-vectorization".

https://en.wikipedia.org/wiki/Automatic_vectorization

What D does is have vector data types, and when those are used 
vector instructions are generated for them.


https://dlang.org/spec/simd.html


For clarity, where the changeling says that GDC & LDC use 
auto-vectorization, that's actually happening with the array 
operations and core.simd is not required, correct?

Re: Release D 2.077.0

[Walter Bright via Digitalmars-d-announce]

On 11/3/2017 2:28 AM, Daniel Kozak wrote:

How should I compile my program to enable array vectorization?


dmd doesn't do what is known as "auto-vectorization".

https://en.wikipedia.org/wiki/Automatic_vectorization

What D does is have vector data types, and when those are used vector 
instructions are generated for them.


https://dlang.org/spec/simd.html

Re: Release D 2.077.0

[Daniel Kozak via Digitalmars-d-announce]

On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:

Glad to announce D 2.077.0.

This release comes with a new, more compact mangling, templated 
vector operations, reproducible dmd builds, and various fixes.

Thanks to everyone involved in this .

http://downloads.dlang.org/releases/2.x/2.077.0/
http://dlang.org/changelog/2.077.0.html
The dlang.org website will get updated soon.

-Martin


How should I compile my program to enable array vectorization?

I have tried dmd -march=native, -march=avx2 as changlog suggest 
but does not work


I have tried even just -march=native or -march=avx2 but still 
does not compile


Error: unrecognized switch '-march=avx2'

Error: unrecognized switch '-march=native'

Re: Release D 2.077.0

[Nick Sabalausky (Abscissa) via Digitalmars-d-announce]

On 11/02/2017 06:35 PM, Martin Nowak wrote:

reproducible dmd builds,


Curiosity: What exactly was preventing this before? Order of source 
files altering the order of output? Something else?


Also, just musing...Regarding the matter of __TIME__(etc) breaking this 
guarantee (breaking it for obvious reasons). Seems to me it would be 
helpful to have compiler switches to force lexer symbols to specific 
values in order to bring at least some level of reproducablity even to 
projects that do use such symbols.