On Tuesday, 12 May 2015 at 20:58:16 UTC, Vladimir Panteleev wrote:
On Tuesday, 12 May 2015 at 18:14:56 UTC, Gerald Jansen wrote:
On Tuesday, 12 May 2015 at 16:35:23 UTC, Rikki Cattermole
wrote:
On 13/05/2015 4:20 a.m., Gerald Jansen wrote:
At the risk of great embarassment ... here's my program:
http://dekoppel.eu/tmp/pedupg.d
Would it be possible to give us some example data?
I might give it a go to try rewriting it tomorrow.
http://dekoppel.eu/tmp/pedupgLarge.tar.gz (89 Mb)
Contains two largish datasets in a directory structure
expected by the program.
Profiling shows that your program spends most of the time
reading the data.
I see a considerable speed boost with the following one-line
patch (plus imports):
- foreach (line; File(pednum, "r").byLine()) {
+ foreach (line; (cast(string)read(pednum)).split('\n')) {
Nice, thanks. Making that replacement in three points in the
program resulted in roughly a 30% speedup at the cost of about
30% more memory in this specific case. Unfortunately it didn't
help with the performance deteroration problem with parallel
foreach.
