Re: Encouraging preliminary results implementing memcpy in D
On Monday, 18 June 2018 at 02:31:25 UTC, Mike Franklin wrote: Unfortunately the code gen is quite a bit worse: Scratch that. If compiling with -O it seems to do the right thing. Mike
Re: Encouraging preliminary results implementing memcpy in D
On Sunday, 17 June 2018 at 17:00:00 UTC, David Nadlinger wrote:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
https://github.com/JinShil/memcpyD
[…]
Feedback, advise, and pull requests to improve the
implementation are most welcome.
The memcpyD implementation is buggy; it assumes that all
arguments are aligned to their size. This isn't necessarily
true. For example, `ubyte[1024].alignof == 1`, and struct
alignment can also be set explicitly using align(N).
Yes, I'm already aware of that. My plan is to create optimized
implementations for aligned data, and then handled unaligned data
as compositions of the various aligned implementations. For
example a 3 byte copy would be a short copy plus a byte copy.
That may not be appropriate for all cases. I'll have to measure,
and adapt.
On x86, you can get away with this in a lot of cases even
though it's undefined behaviour [1], but this is not
necessarily the case for SSE/AVX instructions. In fact, that's
probably a pretty good guess as to where those weird crashes
you mentioned come from.
Thanks! I think you're right.
For loading into vector registers, you can use
core.simd.loadUnaligned instead (ldc.simd.loadUnaligned for LDC
– LDC's druntime has not been updated yet after {load,
store}Unaligned were added upstream as well).
Unfortunately the code gen is quite a bit worse:
Exibit A:
https://run.dlang.io/is/jIuHRG
*(cast(void16*)()) = *(cast(const void16*)());
_Dmain:
pushRBP
mov RBP,RSP
sub RSP,020h
lea RAX,-020h[RBP]
xor ECX,ECX
mov [RAX],RCX
mov 8[RAX],RCX
lea RDX,-010h[RBP]
mov [RDX],RCX
mov 8[RDX],RCX
movdqa XMM0,-020h[RBP]
movdqa -010h[RBP],XMM0
xor EAX,EAX
leave
ret
add [RAX],AL
.text._Dmainends
Exhibit B:
https://run.dlang.io/is/PLRfhW
storeUnaligned(cast(void16*)(), loadUnaligned(cast(const
void16*)()));
_Dmain:
pushRBP
mov RBP,RSP
sub RSP,050h
lea RAX,-050h[RBP]
xor ECX,ECX
mov [RAX],RCX
mov 8[RAX],RCX
lea RDX,-040h[RBP]
mov [RDX],RCX
mov 8[RDX],RCX
mov -030h[RBP],RDX
mov -010h[RBP],RAX
movdqu XMM0,[RAX]
movdqa -020h[RBP],XMM0
movdqa XMM1,-020h[RBP]
movdqu [RDX],XMM1
xor EAX,EAX
leave
ret
add [RAX],AL
.text._Dmainends
If the code gen was better, that would definitely be the way to
go; to have unaligned and aligned share the same implementation.
Maybe I can fix the DMD code gen, or implement a `copyUnaligned`
intrinsic.
Also, there doesn't seem to be any equivalent 32-byte
implementations in `core.simd`. Is that just because noone's
bother to implement them yet? And with AVX512, we should
probably have 64-byte implementations as well.
Mike
Re: Encouraging preliminary results implementing memcpy in D
On Sunday, 17 June 2018 at 17:00:00 UTC, David Nadlinger wrote: core.simd.loadUnaligned instead Ah, well… https://issues.dlang.org/show_bug.cgi?id=19001 — David
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
https://github.com/JinShil/memcpyD
[…]
Feedback, advise, and pull requests to improve the
implementation are most welcome.
The memcpyD implementation is buggy; it assumes that all
arguments are aligned to their size. This isn't necessarily true.
For example, `ubyte[1024].alignof == 1`, and struct alignment can
also be set explicitly using align(N).
On x86, you can get away with this in a lot of cases even though
it's undefined behaviour [1], but this is not necessarily the
case for SSE/AVX instructions. In fact, that's probably a pretty
good guess as to where those weird crashes you mentioned come
from.
On other architectures, unaligned accesses might be outright
unsupported.
For loading into vector registers, you can use
core.simd.loadUnaligned instead (ldc.simd.loadUnaligned for LDC –
LDC's druntime has not been updated yet after {load,
store}Unaligned were added upstream as well).
— David
[1] This is applying C rules to D, where creation of unaligned
pointers is undefined behaviour. The D specification doesn't
mention
Re: Encouraging preliminary results implementing memcpy in D
Well, e-mail was never meant to be reliable or secure... BTW, there are already solutions to prevent impersonation: sign the messages with either PGP or S/MIME... the former is more decentralised, and the later usually comes together with stronger verifications, like personal identification of the sender (I mean, whoever issues the certificate for S/MIME should ideally check your name, etc.) :-) Although admittedly that'd work only for the NNTP / mailing list, I don't know how that could be adapted to the web "forum" interface... On 06/14/2018 05:59 AM, Cym13 wrote: Hopefully once that particular user gets discouraged or we find a way to actually avoid user impersonification, then things will be able to come back to normal.
Re: Encouraging preliminary results implementing memcpy in D
On Friday, 15 June 2018 at 00:15:35 UTC, Mike Franklin wrote: On Thursday, 14 June 2018 at 20:35:23 UTC, baz wrote: [...] Correct! D already has features like `a[] = b[]` so there is no reason to call `memcpy` directly; that is a job for the druntime. `memcpyD` is intended to be an internal druntime utility. [...] Ok, thanks for the explanation.
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 20:35:23 UTC, baz wrote: I asked on IRC yesterday and actually tHose memcpy are not the memcpy we use to copy wide chunks, apparently it's rather for an internal druntime thing, i.e cpy type to type so likely always aligned. Correct! D already has features like `a[] = b[]` so there is no reason to call `memcpy` directly; that is a job for the druntime. `memcpyD` is intended to be an internal druntime utility. However, we should still be good D citizens when we write our low level druntime code, so the interface will be `memcpy(T)(T a, T b)` and `memcpy(T)(T[] a, T[] b)` instead of `memcpy(void* a, void* b, size_t size)`. This will ensure the code, at compile-time, adheres to D's guarantees such as `@safe`, `nothrow`, and `pure`. And, given that it won't require `TypeInfo` it will be usable in -betterC. Although rare, I believe it is still possible to have misaligned memory in D. My understanding is that misaligned copies usually involve copying smaller chunks of the memory until they are aligned, and then copying the aligned memory in larger chunks. I suspect that will work well with the D implementation. TL;DR, Unaligned memory will be handled after optimized aligned memory copies are implemented. Mike
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, June 14, 2018 16:04:32 Nick Sabalausky via Digitalmars-d- announce wrote: > On 06/14/2018 05:01 AM, AnotherTorUser wrote: > > If all such people stopped working for such companies, what do you think > > the economic impact would be? > > What do you think is the social impact if they don't? And don't even try > to pretend the companies can't trivially solve the "economic" issues for > themselves in an instant by knocking off the behaviour that causes loss > of talent. But that would imply that they have a frontal lobe. :) In all seriousness, it is surprising how frequently companies seem to be incapable of making decisions that would fix a lot of their problems, and they seem to be incredibly prone to thinking about things in a shortsighted manner. I'm reminded of an article by Joel Spoelskey where he talks about how one of the key things that a source control software solution can do to make it more likely for folks to be willing to try it is to make it easy to get your source code and history back out again and into another source control system. However, companies typically freak out at the idea of making it easy to switch from their product to another product. They're quite willing to make it easy to switch _to_ their product so that they can start making money off of you, but the idea that making it low cost to leave could actually improve the odds of someone trying their product - and thus increase their profits - seems to be beyond them. Another case which is closer to the exact topic at hand is that many companies seem to forget how much it costs to hire someone when they consider what they should do to make it so that their employees are willing - or even eager - to stay. Spending more money on current employees (be that on salary or something else to make the workplace desirable) or avoiding practices that tick employees off so that they leave can often save money in the long run, but companies frequently ignore that fact. They're usually more interested in saving on the bottom line right now than making decisions that save money over time. So, while I completely agree that companies can technically make decisions that solve some of their problems with things like retaining talent, it seems like it's frequently the case that they're simply incapable of doing it in practice - though YMMV; some companies are better about it than others. - Jonathan M Davis
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 17:27:27 UTC, Patrick Schluter wrote: On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Just a little remark. Alignment has also an extremely heavy impact on the performance of memcpy(). Does your test include it? I asked on IRC yesterday and actually tHose memcpy are not the memcpy we use to copy wide chunks, apparently it's rather for an internal druntime thing, i.e cpy type to type so likely always aligned.
Re: Encouraging preliminary results implementing memcpy in D
On 06/14/2018 05:01 AM, AnotherTorUser wrote: sorry. but what world do you live in? Note this part of what I said: >> If I worked in such an organization that tracked its employees >> activities *outside the workplace* That part is KEY, but it sounds like you an errExit have chosen to ignore that part. If all such people stopped working for such companies, what do you think the economic impact would be? What do you think is the social impact if they don't? And don't even try to pretend the companies can't trivially solve the "economic" issues for themselves in an instant by knocking off the behaviour that causes loss of talent.
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Just a little remark. Alignment has also an extremely heavy impact on the performance of memcpy(). Does your test include it?
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 13:42:14 UTC, Steven Schveighoffer wrote: Well, for me, it wasn't attacks. There have been quite a few people in this community who were rude, insulting, and IMO, that is not worth moderation. In fact some of them have even came around and become quite good D contributors. Those problems usually work themselves out because we have a great community which does not give trolls the attention they desire. That seems to be the opinion of most. The problem I have is that this forum is the main way to communicate about the language, and thus it is how others form their opinion of the language.
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 07:54:20 UTC, Jonathan M Davis wrote: On Wednesday, June 13, 2018 14:34:28 Uknown via Digitalmars-d-announce wrote: Looks very promising. One question though, why not use std.datetime.stopwatch.benchmark? I think it has some protection against optimizing compilers (I may be wrong though). Also, LDC has attributes to control optimizations on a per function basis.see : https://wiki.dlang.org/LDC-specific_language_changes Unfortunately, std.datetime.stopwatch.benchmark does not yet have such protections. It has been discussed, but there were issues with it that still need to be sorted out. In any case, what he has implemented is pretty much what's in Phobos except for the fact that he set up his to take arguments, whereas Phobos' solution just takes the function(s) to call, so anything that it does has to be self-contained. - Jonathan M Davis huh. I saw the PR and assumed it was accepted. Anyway, for DMD putting `asm` blocks seems to still disable optimizations, and for LDC, the pragma is perfect. GDC is the only unknown.
Re: Encouraging preliminary results implementing memcpy in D
On 6/14/18 9:31 AM, rikki cattermole wrote: On 15/06/2018 1:22 AM, Steven Schveighoffer wrote: It's really the impersonation that is the problem, not the anonymity. If you just would stick to one persona, and especially not impersonate people who actually post on this forum, you would not have to use Tor at all, and the forum moderators wouldn't have to worry about active moderation. Not quite. It was the attacking of the D community at large which convinced those with the power to actively ban, to ban "him". Well, for me, it wasn't attacks. There have been quite a few people in this community who were rude, insulting, and IMO, that is not worth moderation. In fact some of them have even came around and become quite good D contributors. Those problems usually work themselves out because we have a great community which does not give trolls the attention they desire. But when you start impersonating people, especially ones that post to this forum, you have crossed the line and are literally putting words into other's mouths. We've seen this before, and generally they go away, but this guy does not want to. Now he's claiming "discrimination" ;) Once they are gone, we can deactivate the extra protections that have been put in place, because until this person came along, we were quite happy moderating ourselves. It lasted nearly 20 years that peace... I agree, this will be a blip in our forum experience, and next month we'll barely remember him. -Steve
Re: Encouraging preliminary results implementing memcpy in D
On 15/06/2018 1:22 AM, Steven Schveighoffer wrote: It's really the impersonation that is the problem, not the anonymity. If you just would stick to one persona, and especially not impersonate people who actually post on this forum, you would not have to use Tor at all, and the forum moderators wouldn't have to worry about active moderation. Not quite. It was the attacking of the D community at large which convinced those with the power to actively ban, to ban "him". We have in essence decided that this person will never be willing to have positive experiences within our community and have out right decided that we do not want them here under any circumstance. A lot of work has gone into getting rid of "him". As far as I am concerned the original IP address that was used from Australia was in fact a proxy and had the single desire for this person, was to attack us. Once they are gone, we can deactivate the extra protections that have been put in place, because until this person came along, we were quite happy moderating ourselves. It lasted nearly 20 years that peace...
Re: Encouraging preliminary results implementing memcpy in D
On 6/13/18 10:32 PM, errExit wrote: On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote: I am part of the D community. I haven't discriminated against anyone. I don't know what a Tor user is. I've just searched: So Tor is an old idea of mine, implemented. :o) Ali Tor is our last line of defence against an Orson Wells future, where everyones actions are scrutinized by big brother, so that big brother can use that knowledge to put fear into, control and manipulate, those that don't conform. Sorry KingJoffrey/psychoticRabbit/etc, you literally only started using Tor when your steady IP was banned. This argument kind of falls down when your past behavior is examined. It's really the impersonation that is the problem, not the anonymity. If you just would stick to one persona, and especially not impersonate people who actually post on this forum, you would not have to use Tor at all, and the forum moderators wouldn't have to worry about active moderation. -Steve
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Mike Hello Mike, These are my results: Ubuntu 16.04.4 amd64 Linux 4.16.0-ck1+ Intel Xeon E5-2620 0 @ 2.00GHz - 12 cores 32 GB RAM dmd -O -release memcpyd.d # dmd 2.080.0 size memcpyC memcpyD 1 125349 47658 2 155014 50492 4 173099 52669 8 228236 52676 16 107897 32621 32 128039 32604 64 163644 37658 128 223840 50420 256 338769 90300 512 584772 171038 1024 878093 995813 2048 1346958 1254141 4096 2439378 2101284 8192 5631202 3554307 16384 9873090 6496635 32768 22489302 21328288 65536 50522961 45748356 size memcpyC memcpyD 1 123241 27631 1 130758 28165 1 123247 32748 2 142964 27587 2 140914 28103 4 168084 32616 4 171166 27590 8 228274 27604 8 233249 27605 4 168624 27597 8 238049 29435 16 103956 52730 ldc2 -O3 -release memcpyd.d # ldc2 1.10.0-beta1 (I think these are strange results) size memcpyC memcpyD 1 0 0 2 0 0 4 0 0 8 0 0 16 559 0 32 1003 0 64 0 0 128 0 0 256 0 0 512 0 0 1024 460182 1519048 2048 739148 1973641 4096 1533047 3168472 8192 2913463 5560106 16384 6385370 10353178 32768 20889322 21487968 65536 44920382 48339716 size memcpyC memcpyD 1 0 0 1 0 0 1 0 0 2 0 0 2 0 0 4 0 0 4 0 0 8 0 0 8 0 0 4 0 0 8 0 0 16 0 0
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 07:19:31 UTC, Nick Sabalausky (Abscissa) wrote: I'm with you on a lot of that, however, this part troubles me: "This becomes problematic for those of us who work in 'certain organisations', that insist on tracking it's employees online activities (even outside of the workplace)." If I worked in such an organization that tracked its employees activities *outside the workplace*, I'd LEAVE it ASAP, and I'd strongly suggest anyone else do the same. I mean what insane workplace is that, the 1920's mob? Apple? Honestly, if you believe strongly enough in Tor to use it, why in the world would you willfully aid and abed an organization that does that sort of thing? It doesn't make any sense at all. It's EXTREMELY self-contradictory and completely erodes your entire stance. If you're going to preach for personal freedom and privacy, at least have the basic integrity to LIVE the basic ideals you're preaching even when doing so ISN'T so trivial as installing a mere web browser. Sadly, it's increasingly 'standard practice' in HR to do just that. Monitor what employees, and potential employees have done or made available on the internet. I don't support that approach, which is exactly why I use Tor. Now, their attempts are moot, and therefore I am in no way supporting those actions. And so, your comments about 'self-contradictory' are moot also. In addition, HR data is increasingly becoming a valuable target for attack, due to the 'profiles' they keep on people. So now the situation gets even worse. Cause not only does HR have this info, so will others... eventually. The only real world option, is to prevent them from gathering data on you in the first place. btw. Some people in my team (those that use D), might have contributed to this post, but now, likely will not.
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 07:19:31 UTC, Nick Sabalausky (Abscissa) wrote: I'm with you on a lot of that, however, this part troubles me: "This becomes problematic for those of us who work in 'certain organisations', that insist on tracking it's employees online activities (even outside of the workplace)." If I worked in such an organization that tracked its employees activities *outside the workplace*, I'd LEAVE it ASAP, and I'd strongly suggest anyone else do the same. I mean what insane workplace is that, the 1920's mob? Apple? Honestly, if you believe strongly enough in Tor to use it, why in the world would you willfully aid and abed an organization that does that sort of thing? It doesn't make any sense at all. It's EXTREMELY self-contradictory and completely erodes your entire stance. If you're going to preach for personal freedom and privacy, at least have the basic integrity to LIVE the basic ideals you're preaching even when doing so ISN'T so trivial as installing a mere web browser. sorry. but what world do you live in? If all such people stopped working for such companies, what do you think the economic impact would be? Tor prevents tracking, and therefore it is not contradictory to work for such companies - because they can't track you (or at least, it become much more difficult to do so).
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 07:45:15 UTC, Joakim wrote: Tor is merely one tool used to route around those building centralized systems on top of the internet. The real solution is that as more and more decentralized tech does well, like git or cryptocurrencies, to get rid of these obsolete centralized systems altogether. The problem with the D Foundation's 'new approach' is, that there is simply no visibility as to what is being moderated, how long it takes, whom does it, and for what the filtering criteria is. We're just being told that it is too avoid 'spammers'?? If that is not a tool for abuse, then I don't know what is.
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, June 13, 2018 14:34:28 Uknown via Digitalmars-d-announce wrote: > Looks very promising. One question though, why not use > std.datetime.stopwatch.benchmark? I think it has some protection > against optimizing compilers (I may be wrong though). Also, LDC > has attributes to control optimizations on a per function > basis.see : https://wiki.dlang.org/LDC-specific_language_changes Unfortunately, std.datetime.stopwatch.benchmark does not yet have such protections. It has been discussed, but there were issues with it that still need to be sorted out. In any case, what he has implemented is pretty much what's in Phobos except for the fact that he set up his to take arguments, whereas Phobos' solution just takes the function(s) to call, so anything that it does has to be self-contained. - Jonathan M Davis
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 02:32:51 UTC, errExit wrote:
On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote:
I am part of the D community. I haven't discriminated against
anyone. I don't know what a Tor user is.
I've just searched: So Tor is an old idea of mine,
implemented. :o)
Ali
Tor is our last line of defence against an Orson Wells future,
where everyones actions are scrutinized by big brother, so that
big brother can use that knowledge to put fear into, control
and manipulate, those that don't conform.
assert("bad tor user" != "all tor users are bad");
(actually there are more bad non-tor users)
Unfortunately, it's becoming increasingly, the norm, to
discriminate against tor users (no doubt those doing that
discrimination are those that are happy to conform, of which
there will be many, sadly).
https://people.torproject.org/~lunar/20160331-CloudFlare_Fact_Sheet.pdf
Tor is merely one tool used to route around those building
centralized systems on top of the internet. The real solution is
that as more and more decentralized tech does well, like git or
cryptocurrencies, to get rid of these obsolete centralized
systems altogether.
Re: Encouraging preliminary results implementing memcpy in D
On 06/14/2018 02:34 AM, errExit wrote: The D Foundation now subjects all users having an ip originating from a tor exit node, to having their posts moderated (but by whom, when, how, criteria ?? etc). Literally millions of people could, and probably would, be using that exit node. So that is plain discrimination. It's not spammer management. Forcing people to identify themselves, is also not about spammer management either. The D Foundation IS now discimantory against those that want that believe that freedom and privacy is some to be protected. This becomes problematic for those of us who work in 'certain organisations', that insist on tracking it's employees online activities (even outside of the workplace). It's a shame the D Foundation has finally succumed to the big brother mentality - under the guise of protecting you from spam. https://blog.torproject.org/dont-let-facebook-or-any-tracker-follow-you-web I'm with you on a lot of that, however, this part troubles me: "This becomes problematic for those of us who work in 'certain organisations', that insist on tracking it's employees online activities (even outside of the workplace)." If I worked in such an organization that tracked its employees activities *outside the workplace*, I'd LEAVE it ASAP, and I'd strongly suggest anyone else do the same. I mean what insane workplace is that, the 1920's mob? Apple? Honestly, if you believe strongly enough in Tor to use it, why in the world would you willfully aid and abed an organization that does that sort of thing? It doesn't make any sense at all. It's EXTREMELY self-contradictory and completely erodes your entire stance. If you're going to preach for personal freedom and privacy, at least have the basic integrity to LIVE the basic ideals you're preaching even when doing so ISN'T so trivial as installing a mere web browser.
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 03:59:47 UTC, Cym13 wrote: Don't mistake spammer management with discrimination. I share your frustration that TOR isn't more usable than it is today with CloudFlare etc, but coming with political reasons holds no water if the reason why it was blacklisted wasn't political in the first place. It's not false, it just won't work. Hopefully once that particular user gets discouraged or we find a way to actually avoid user impersonification, then things will be able to come back to normal. The D Foundation now subjects all users having an ip originating from a tor exit node, to having their posts moderated (but by whom, when, how, criteria ?? etc). Literally millions of people could, and probably would, be using that exit node. So that is plain discrimination. It's not spammer management. Forcing people to identify themselves, is also not about spammer management either. The D Foundation IS now discimantory against those that want that believe that freedom and privacy is some to be protected. This becomes problematic for those of us who work in 'certain organisations', that insist on tracking it's employees online activities (even outside of the workplace). It's a shame the D Foundation has finally succumed to the big brother mentality - under the guise of protecting you from spam. https://blog.torproject.org/dont-let-facebook-or-any-tracker-follow-you-web
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Mike On 8 core, 16 GB Intel Skull Candy box running Ubuntu 18.04 64 bit. https://gist.githubusercontent.com/carun/f7c2c200b1be20d0a9489296d6601332/raw/db01bb8bc909c6048288fccc500bd15e5ee491b2/memcpyd-output.log Hope this helps.
Re: Encouraging preliminary results implementing memcpy in D
On Thursday, 14 June 2018 at 02:32:51 UTC, errExit wrote:
Tor is our last line of defence against an Orson Wells future,
where everyones actions are scrutinized by big brother, so that
big brother can use that knowledge to put fear into, control
and manipulate, those that don't conform.
assert("bad tor user" != "all tor users are bad");
(actually there are more bad non-tor users)
Unfortunately, it's becoming increasingly, the norm, to
discriminate against tor users (no doubt those doing that
discrimination are those that are happy to conform, of which
there will be many, sadly).
https://people.torproject.org/~lunar/20160331-CloudFlare_Fact_Sheet.pdf
Don't mistake spammer management with discrimination. I share
your frustration that TOR isn't more usable than it is today with
CloudFlare etc, but coming with political reasons holds no water
if the reason why it was blacklisted wasn't political in the
first place. It's not false, it just won't work.
Hopefully once that particular user gets discouraged or we find a
way to actually avoid user impersonification, then things will be
able to come back to normal.
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote:
I am part of the D community. I haven't discriminated against
anyone. I don't know what a Tor user is.
I've just searched: So Tor is an old idea of mine, implemented.
:o)
Ali
Tor is our last line of defence against an Orson Wells future,
where everyones actions are scrutinized by big brother, so that
big brother can use that knowledge to put fear into, control and
manipulate, those that don't conform.
assert("bad tor user" != "all tor users are bad");
(actually there are more bad non-tor users)
Unfortunately, it's becoming increasingly, the norm, to
discriminate against tor users (no doubt those doing that
discrimination are those that are happy to conform, of which
there will be many, sadly).
https://people.torproject.org/~lunar/20160331-CloudFlare_Fact_Sheet.pdf
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 16:21:53 UTC, Steven Schveighoffer wrote: I won't add much, since I'm using a Mac, and those numbers have already been posted. Reproduction is an important part of the scientific process, please post away ;) Also: memcpyD commit f5fb0fda0dbacc960ced59d7171ff76a95cc2abf on Archlinux native x86_64 4.16.13-2 with Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz and 7681MB of RAM, with dmd and ldc, no flag and optimized. DMD: size memcpyC memcpyD 1 50392 30745 2 50189 33515 4 47493 33456 8 50282 33428 16 36385 36061 32 36212 32142 64 50141 31137 128 52947 52785 256 86422 76346 512 131191 128344 1024 227875 291414 2048 444865 449210 4096 826391 823613 8192 1542429 1545051 16384 3264964 3228990 32768 11644816 11902418 65536 23658237 24026304 size memcpyC memcpyD 1 43209 37462 1 42412 38043 1 42079 38002 2 56503 39923 2 50544 38033 4 47760 37239 4 48910 36976 8 51110 38065 8 53761 36297 4 48201 35548 8 54820 38036 16 39360 35409 DMD -O: core.exception.AssertError@memcpyd.d(9): Assertion failure ??:? _d_assertp [0xc4e79cc5] ??:? pure nothrow @nogc void memcpyd.verify!(real).verify(const(real), const(real)) [0xc4e77c2c] ??:? void memcpyd.test!(real).test() [0xc4e76d60] ??:? _Dmain [0xc4e63ecd] 1 42014 26518 2 46086 26486 4 45984 29272 8 48813 26484 16 34866 18126 32 32036 18107 64 46073 20892 128 45964 27993 256 82344 50250 512 133484 94766 1024 216402 270462 2048 436465 443122 4096 815875 801596 8192 1524872 1530453 16384 3721245 3584620 32768 11776185 11739396 65536 23702074 23589480 size memcpyC memcpyD 1 37755 15424 1 40486 15319 1 40505 15352 2 47097 15653 2 46160 15319 4 43180 18110 4 46041 15321 8 46014 15342 8 46066 15341 4 43277 18105 8 45962 15321 LDC: size memcpyC memcpyD 1 56378 48873 2 59461 49025 4 50481 45299 8 53786 49517 16 46103 39122 32 48100 46139 64 83864 67401 128 83849 90426 256 122471 138309 512 169668 228472 1024 260461 276878 2048 444937 472365 4096 860962 825468 8192 1578929 1567154 16384 3429235 3284611 32768 11941494 11868947 65536 24052536 24112980 size memcpyC memcpyD 1 47853 33403 1 47623 32924 1 48190 33100 2 59752 33146 2 59574 34371 4 53928 35042 4 54131 31991 8 57929 35864 8 57372 32174 4 54901 33253 8 62537 34535 16 52487 38358 LDC -O2: core.exception.AssertError@memcpyd.d(9): Assertion failure ??:? _d_assert [0x7fba47d79b35] ??:? void memcpyd.test!(real).test() [0x5638cb483d6e] ??:? _Dmain [0x5638cb47ea30] size memcpyC memcpyD 1 0 0 2 0 0 4 0 0 8 0 0 16 385 0 32 792 0 64 1542 0 128 2981 0 256 90108 0 512 126316 0 1024 217881 391419 2048 415182 620853 4096 1244805 1240074 8192 2428417 2414095 16384 4863280 4971193 32768 12968909 12966676 65536 26393408 26395410 size memcpyC memcpyD 1 0 0 1 0 0 1 0 0 2 0 0 2 0 0 4 0 0 4 0 0 8 0 0 8 0 0 4 0 0 8 0 0
Re: Encouraging preliminary results implementing memcpy in D
On 06/13/2018 02:07 AM, ToRuSer wrote: > So well done to the D community, for discriminating against all the Tor > users out there. You've done yourself proud. I am part of the D community. I haven't discriminated against anyone. I don't know what a Tor user is. I've just searched: So Tor is an old idea of mine, implemented. :o) Ali
Re: Encouraging preliminary results implementing memcpy in D
On 6/13/18 2:46 AM, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Just FYI, Linux results on a VM are still valid -- many people (including myself) only use Linux on a VM, so even though it's not a definitive win against glibc memcpy on Linux, the end result is, it's still faster for those users :) I won't add much, since I'm using a Mac, and those numbers have already been posted. But I wanted to say don't discount those findings off-hand. -Steve
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Mike specs . (note: tried DMD v2.078.3) > dmd -O (and got strange assertion errors). (used LDC 1.8.0 instead) > ldc2 -O2 Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz 24GB Mem (Speed: 1066 MT/s) Kubuntu 18.04 LTS - 4.15.0-23-generic results size memcpyC memcpyD 1 0 0 2 0 0 4 0 0 8 0 0 16 1247 0 32 1889 0 64 3055 0 128 5040 0 256 91407 0 512 158527 0 1024 253191 870780 2048 474243 1170349 4096 932151 1933045 8192 1894059 3284901 16384 4447945 6122015 32768 18572704 20772417 65536 37470068 40211203 size memcpyC memcpyD 1 0 0 1 0 0 1 0 0 2 0 0 2 0 0 4 0 0 4 0 0 8 0 0 8 0 0 4 0 0 8 0 0 16 0 0 ..
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Mike I'm glad to see this kind of improvement! I noticed there wasn't any macOS testing though, so heres the results on my machine: size memcpyC memcpyD 1 38613 5185 2 42475 7259 4 54272 3450 8 29391 3457 16 32947 3626 32 33761 7253 64 44152 14510 128 51985 27711 256 65700 58044 512 95315 119977 1024 150726 387021 2048 265975 667858 4096 521565 1113945 8192 965322 2039064 16384 3923818 3997524 32768 15586237 15669232 65536 33195890 31640205 size memcpyC memcpyD 1 40057 5180 1 36664 5226 1 37625 5280 2 36584 5178 2 36956 5252 4 51086 3448 4 49952 3456 8 29497 3455 8 29710 3987 4 51539 3453 8 29715 3449 16 29747 23535 sys info : Intel Core i5 4258u (28w), 8GB DDR3 1600MHz RAM Looks very promising. One question though, why not use std.datetime.stopwatch.benchmark? I think it has some protection against optimizing compilers (I may be wrong though). Also, LDC has attributes to control optimizations on a per function basis.see : https://wiki.dlang.org/LDC-specific_language_changes My linux install got messed up, but I'll post benchmarks as soon as its up
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 12:45:26 UTC, Fra Mecca wrote: I get this on Linux 4.16.3-gentoo, AMD FX(tm)-6100 Six-Core Processor, 8GiB ram, using ldc2 -O3L: size memcpyC memcpyD 1 5 0 2 0 0 4 0 0 8 0 0 16 1519 0 32 1833 0 64 3816 0 128 7543 0 256 146500 0 512 194818 0 1024 329593 846142 2048 714945 1117132 4096 1596170 1803621 8192 5899818 6110026 16384 12338384 14141850 32768 24971315 26771484 65536 49806637 63260128 size memcpyC memcpyD 1 0 0 1 0 0 1 0 0 2 0 0 2 0 0 4 0 0 4 0 0 8 0 0 8 0 0 4 0 0 8 0 0 core.exception.AssertError@memcpyd.d(9): Assertion failure ??:? _d_assert [0xcaf056d5] ??:? [0xa015e7fe] ??:? [0xa0158cb0] ??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() [0xcaf29daf] ??:? _d_run_main [0xcaf29c7b] ??:? __libc_start_main [0xca160eeb] ??:? [0xa0158b29] Yes, optimizations are throwing code away. I'm working on it. Mike
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Mike I get this on Linux 4.16.3-gentoo, AMD FX(tm)-6100 Six-Core Processor, 8GiB ram, using ldc2 -O3L: size memcpyC memcpyD 1 5 0 2 0 0 4 0 0 8 0 0 16 1519 0 32 1833 0 64 3816 0 128 7543 0 256 146500 0 512 194818 0 1024 329593 846142 2048 714945 1117132 4096 1596170 1803621 8192 5899818 6110026 16384 12338384 14141850 32768 24971315 26771484 65536 49806637 63260128 size memcpyC memcpyD 1 0 0 1 0 0 1 0 0 2 0 0 2 0 0 4 0 0 4 0 0 8 0 0 8 0 0 4 0 0 8 0 0 core.exception.AssertError@memcpyd.d(9): Assertion failure ??:? _d_assert [0xcaf056d5] ??:? [0xa015e7fe] ??:? [0xa0158cb0] ??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() [0xcaf29daf] ??:? _d_run_main [0xcaf29c7b] ??:? __libc_start_main [0xca160eeb] ??:? [0xa0158b29]
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 08:55:40 UTC, drug wrote: Ubuntu 18.04 Linux 4.15.0-23-generic AMD® Fx(tm)-8350 eight-core processor × 8 size memcpyC memcpyD 1 51089 36921 2 45896 35733 4 46079 36200 8 48443 37509 16 48669 24925 32 52917 27787 64 55631 44928 128 84282 47795 256 107350 66009 512 159310 126795 1024 247683 452560 2048 440687 673211 4096 1129135 1304085 8192 4740910 4095254 16384 8389579 8874273 32768 16630336 17370310 65536 33032013 42904705 size memcpyC memcpyD 1 52354 28365 1 48407 28445 1 50264 30273 2 51312 27708 2 46138 28973 4 52753 28535 4 52150 27418 8 52220 27276 8 49625 27804 4 49356 33510 8 48529 27668 16 52662 135357 Interesting! I have an AMD 8370 running Windows 8, and I get more favorable results in Windows: size memcpyC memcpyD 1 45361 43626 2 55091 43791 4 70507 43714 8 50910 42854 16 63328 28831 32 72817 30790 64 76307 45823 128 97180 55368 256 164935 68362 512 230508 132100 1024 502189 490590 2048 892968 823070 4096 1896480 1456353 8192 4530645 4516681 16384 10886602 9921215 32768 21717080 19116839 65536 59787610 43549445 size memcpyC memcpyD 1 48770 30084 1 49169 30921 1 43370 30144 2 51404 27571 2 56002 29729 4 69588 29804 4 63743 29510 8 55492 29002 8 46752 31793 4 72673 28858 8 48989 27547 10 55527 121628 In your results, I see that for sizes 1024 and higher (that's when is dispatches to the REP MOVSB algorithm), the performance begins to degrade for Linux. I'm going to install Linux soon and see if I can fix that. Thanks for the data, Mike
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 10:17:10 UTC, Mike Franklin wrote: Well, actually, I probably should divide that time by 10,000,000 to make a more accurate representation. For rigorous benchmarking, check out the first part of Andrei's Writing Fast Code: https://www.youtube.com/watch?v=vrfYLlR8X8k One takeaway is that taking the average of many runtimes is not the best use of your dataset.
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 09:07:21 UTC, ToRuSer wrote: On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Mike All Tor users now apparently have their posts subjected to 'moderation'. (i.e. someone, will, perhaps, at some point, get around to reviewing their posts, and then, perhaps, it might reach the forum, or not.) So... maybe..you'll get those posts...or maybe not...and when.. is anyones guess. So well done to the D community, for discriminating against all the Tor users out there. You've done yourself proud. The problem is likely more that someone has used Tor to troll here and then the enpoint used was blacklisted.
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 10:13:13 UTC, Dukc wrote: On Wednesday, 13 June 2018 at 09:59:52 UTC, Mike Franklin wrote: The benchmark doesn't allocate any data; it's just copying data. Mike Ah of course. I was thinking other stuff while writing. Well, actually, I probably should divide that time by 10,000,000 to make a more accurate representation. Thanks for the feedback, Mike
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 09:59:52 UTC, Mike Franklin wrote: The benchmark doesn't allocate any data; it's just copying data. Mike Ah of course. I was thinking other stuff while writing.
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 09:40:05 UTC, Dukc wrote: If I read your benchmark graphs right, they claimed that allocating 16 kilobytes takes over 10^^6 usecs, with both mallocs. Doesn't that mean over a second, 16 kilobytes? Can't be! Are you confusing usecs with nsecs? The benchmark doesn't allocate any data; it's just copying data. Each benchmark is run 10,000,000 times to smooth out some of the entropy in the results: https://github.com/JinShil/memcpyD/blob/2e0d3c33ea876a25a04358a3ae505b2eba9f99cb/memcpyd.d#L78 The usecs in the graph is the time it takes to run the benchmark 10,000,000 times. Mike
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. If I read your benchmark graphs right, they claimed that allocating 16 kilobytes takes over 10^^6 usecs, with both mallocs. Doesn't that mean over a second, 16 kilobytes? Can't be! Are you confusing usecs with nsecs?
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Mike All Tor users now apparently have their posts subjected to 'moderation'. (i.e. someone, will, perhaps, at some point, get around to reviewing their posts, and then, perhaps, it might reach the forum, or not.) So... maybe..you'll get those posts...or maybe not...and when.. is anyones guess. So well done to the D community, for discriminating against all the Tor users out there. You've done yourself proud.
Re: Encouraging preliminary results implementing memcpy in D
Ubuntu 18.04 Linux 4.15.0-23-generic AMD® Fx(tm)-8350 eight-core processor × 8 size memcpyC memcpyD 1 51089 36921 2 45896 35733 4 46079 36200 8 48443 37509 16 48669 24925 32 52917 27787 64 55631 44928 128 84282 47795 256 107350 66009 512 159310 126795 1024 247683 452560 2048 440687 673211 4096 1129135 1304085 8192 4740910 4095254 16384 8389579 8874273 32768 16630336 17370310 65536 33032013 42904705 size memcpyC memcpyD 1 52354 28365 1 48407 28445 1 50264 30273 2 51312 27708 2 46138 28973 4 52753 28535 4 52150 27418 8 52220 27276 8 49625 27804 4 49356 33510 8 48529 27668 16 52662 135357 second run size memcpyC memcpyD 1 47248 36964 2 45624 35627 4 45535 35596 8 47920 37012 16 47960 25107 32 52798 27394 64 55444 44282 128 76819 41055 256 105852 66429 512 157629 126243 1024 253841 448974 2048 438973 667101 4096 1144280 1337549 8192 3647558 4141162 16384 8301059 8722185 32768 16413116 17506957 65536 32958933 40381270 size memcpyC memcpyD 1 48513 26288 1 46080 26842 1 48526 26989 2 48634 26419 2 43522 27150 4 48229 25737 4 52841 28117 8 49632 25913 8 46325 25487 4 40267 32343 8 45990 25220 16 46509 124042
Re: Encouraging preliminary results implementing memcpy in D
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote: I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Hi Mike, These are my results running your program under archlinux x86_64 with the zen-kernel 4.17.1, the hardware is powered by an ancient "Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz" with 4GB of ram: size memcpyC memcpyD 1 67607 56810 2 68105 57638 4 66760 58949 8 66943 61262 16 71937 43821 32 70955 48392 64 111473 54226 128 144784 77165 256 183504 113597 512 289039 180930 1024 450526 1314835 2048 782029 1890236 4096 1627622 3165319 8192 2751701 5614202 16384 6361074 11484517 32768 30931212 42805529 65536 61878379 86000892 size memcpyC memcpyD 1 66796 44745 1 66773 44343 1 66780 44157 2 66769 44370 2 66792 44529 4 66776 44298 4 66775 44412 8 66766 44409 8 70945 44359 4 66804 44367 8 71007 44432 16 75210 50656
Encouraging preliminary results implementing memcpy in D
I had a little fun today kicking the crap out of C's memcpy with a D implementation. https://github.com/JinShil/memcpyD Request for help: I don't have a Linux system running on real hardware at this time, nor do I have a wide range of platforms and machines to test with. If you'd like to help me with this potentially foolish endeavor, please run the program on your hardware and send me the results. Feedback, advise, and pull requests to improve the implementation are most welcome. Mike
