Little ping I hope an answer about IO in D and disruptor form
java world
Disruptor seem to provide a smart implementation between IO and
their buffer.
What did you think about it?
D could to provided a high level way to process efficiently a
file. (using Range, forwardrange ... will be
Java has disruptor to provide the fatest way to ring file.
website: http://lmax-exchange.github.io/disruptor/
technical information:
http://lmax-exchange.github.io/disruptor/files/Disruptor-1.0.pdf
What about hugepagesize system on LINUX ?
On 3/23/15 7:52 AM, Steven Schveighoffer wrote:
On 3/22/15 3:03 AM, Andrei Alexandrescu wrote:
* assumeSafeAppend() was unnecessarily used once per line read. Its
removal led to a whopping 35% on top of everything else. I'm not sure
what it does, but boy it does takes its sweet time. Maybe
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu
wrote:
I just took a look at making byLine faster. It took less than
one evening:
https://github.com/D-Programming-Language/phobos/pull/3089
I confess I am a bit disappointed with the leadership being
unable to delegate this task
On 3/22/15 3:03 AM, Andrei Alexandrescu wrote:
* assumeSafeAppend() was unnecessarily used once per line read. Its
removal led to a whopping 35% on top of everything else. I'm not sure
what it does, but boy it does takes its sweet time. Maybe someone should
look into it.
That's not expected.
I made the same test in C# using a 30MB plain ASCII text file.
Compared to fastest method proposed by Andrei, results are not
the best:
D:
readText.representation.count!(c = c == '\n') - 428 ms
byChunk(4096).joiner.count!(c = c == '\n') - 1160 ms
C#:
File.ReadAllLines.Length - 216 ms;
Win64,
On 3/23/15 2:13 PM, rumbu wrote:
Since this code is clearly not the best for this task, as I suspected, I
looked into jitted code and it seems that the .net runtime is smart
enough to recognize this pattern and is doing the following:
- file is mapped into memory using CreateFileMapping
- does
On Monday, 23 March 2015 at 19:25:08 UTC, Tobias Pankrath wrote:
I made the same test in C# using a 30MB plain ASCII text file.
Compared to fastest method proposed by Andrei, results are not
the best:
D:
readText.representation.count!(c = c == '\n') - 428 ms
byChunk(4096).joiner.count!(c = c
On 3/23/15 10:59 AM, Andrei Alexandrescu wrote:
On 3/23/15 7:52 AM, Steven Schveighoffer wrote:
On 3/22/15 3:03 AM, Andrei Alexandrescu wrote:
* assumeSafeAppend() was unnecessarily used once per line read. Its
removal led to a whopping 35% on top of everything else. I'm not sure
what it
On Monday, 23 March 2015 at 15:00:07 UTC, John Colvin wrote:
What would be really great would be a performance test suite
for phobos.
I'm working on it
https://github.com/D-Programming-Language/phobos/pull/2995
On 3/23/15 10:43 AM, rumbu wrote:
On Monday, 23 March 2015 at 15:00:07 UTC, John Colvin wrote:
What would be really great would be a performance test suite for
phobos. D is reaching a point where It'll probably be fast because we
did it right or I remember it being fast-ish 3 years ago when i
On Monday, 23 March 2015 at 15:00:07 UTC, John Colvin wrote:
What would be really great would be a performance test suite
for phobos. D is reaching a point where It'll probably be fast
because we did it right or I remember it being fast-ish 3
years ago when i wrote a small toy test isn't
On 3/23/15 2:42 PM, Steven Schveighoffer wrote:
On 3/23/15 10:59 AM, Andrei Alexandrescu wrote:
On 3/23/15 7:52 AM, Steven Schveighoffer wrote:
On 3/22/15 3:03 AM, Andrei Alexandrescu wrote:
* assumeSafeAppend() was unnecessarily used once per line read. Its
removal led to a whopping 35% on
On 3/23/15 7:33 PM, Andrei Alexandrescu wrote:
On 3/23/15 2:42 PM, Steven Schveighoffer wrote:
On 3/23/15 10:59 AM, Andrei Alexandrescu wrote:
On 3/23/15 7:52 AM, Steven Schveighoffer wrote:
On 3/22/15 3:03 AM, Andrei Alexandrescu wrote:
* assumeSafeAppend() was unnecessarily used once per
On 3/23/15 6:44 PM, Steven Schveighoffer wrote:
On 3/23/15 9:17 PM, Steven Schveighoffer wrote:
I re-examined and found something interesting -- assumeSafeAppend
doesn't cache the block, it only uses the cache if it's ALREADY cached.
So a large chunk of that 35% is the runtime looking up that
On 3/23/15 9:17 PM, Steven Schveighoffer wrote:
I re-examined and found something interesting -- assumeSafeAppend
doesn't cache the block, it only uses the cache if it's ALREADY cached.
So a large chunk of that 35% is the runtime looking up that block info
in the heap. On my machine, this
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu
wrote:
I confess I am a bit disappointed with the leadership being
unable to delegate this task to a trusty lieutenant in the
community. There's been a bug opened on this for a long time,
it gets regularly discussed here (with the
Am 22.03.2015 um 08:43 schrieb Sönke Ludwig:
Am 22.03.2015 um 08:18 schrieb weaselcat:
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu wrote:
I just took a look at making byLine faster. It took less than one
evening:
https://github.com/D-Programming-Language/phobos/pull/3089
I
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu
wrote:
I just took a look at making byLine faster. It took less than
one evening:
https://github.com/D-Programming-Language/phobos/pull/3089
I confess I am a bit disappointed with the leadership being
unable to delegate this task
Am 22.03.2015 um 08:18 schrieb weaselcat:
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu wrote:
I just took a look at making byLine faster. It took less than one
evening:
https://github.com/D-Programming-Language/phobos/pull/3089
I confess I am a bit disappointed with the
On 3/22/15 12:03 AM, Andrei Alexandrescu wrote:
I just took a look at making byLine faster. It took less than one evening:
https://github.com/D-Programming-Language/phobos/pull/3089
I confess I am a bit disappointed with the leadership being unable to
delegate this task to a trusty lieutenant
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu
wrote:
* For each line read there was a call to malloc() and one to
free(). I set things up that the buffer used for reading is
reused by simply making the buffer static.
What about e.g.
zip(File(a.txt).byLine, File(b.txt).byLine)
On 3/22/15 1:26 AM, Sad panda wrote:
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu wrote:
I confess I am a bit disappointed with the leadership being unable to
delegate this task to a trusty lieutenant in the community. There's
been a bug opened on this for a long time, it gets
On 3/22/15 3:10 AM, Vladimir Panteleev wrote:
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu wrote:
* For each line read there was a call to malloc() and one to free(). I
set things up that the buffer used for reading is reused by simply
making the buffer static.
What about
On 3/22/15 10:13 AM, tcak wrote:
On Sunday, 22 March 2015 at 16:03:11 UTC, Andrei Alexandrescu wrote:
On 3/22/15 3:10 AM, Vladimir Panteleev wrote:
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu wrote:
* For each line read there was a call to malloc() and one to free(). I
set
On Sunday, 22 March 2015 at 16:03:11 UTC, Andrei Alexandrescu
wrote:
On 3/22/15 3:10 AM, Vladimir Panteleev wrote:
On Sunday, 22 March 2015 at 07:03:14 UTC, Andrei Alexandrescu
wrote:
* For each line read there was a call to malloc() and one to
free(). I
set things up that the buffer used for
I just took a look at making byLine faster. It took less than one evening:
https://github.com/D-Programming-Language/phobos/pull/3089
I confess I am a bit disappointed with the leadership being unable to
delegate this task to a trusty lieutenant in the community. There's been
a bug opened on
28 matches
Mail list logo
