On Friday, 8 November 2019 at 01:12:37 UTC, Ali Çehreli wrote:
On 11/07/2019 07:07 AM, bioinfornatics wrote:
> Dear,
>
> I try to use the async buffer describe into std.parallelism
> documentation but my test code core dump!
I admit I don't fully understand the lifetime issues but
removing the "code smell" of the modul-level File object solved
the issue for me, which requires four changes:
// (1) Comment out the module-level variable
// File file;
// ...
@system
void next(File file, ref ubyte[] buf)
{
// (2.a) Use the parameter 'file'
// (2.b) Adjust the length of the buffer
// (rawRead may read less than the requested size)
buf = file.rawRead(buf);
}
// ...
// (3) Define a local variable
auto file = File(filePath, "rb");
// ...
// (4) Use "callables" that use the local 'file':
auto asyncReader = taskPool.asyncBuf((ref ubyte[] buf) =>
next(file, buf),
() => file.eof,
bufferSize);
Ali
Thanks a lot Ali, the error message was understandable to me, and
the way you fixed differ from the documentation. Does that means
they are a bug ?
Moreover, this snippet is used to evaluate how to parse
efficiently versus the wc -l command.
fixed snippet:
https://paste.fedoraproject.org/paste/B~PjobIlVXIaJPfMjCcWSA
And I have 2 problems
1/ the executable is at least twice a time slower than wc -l
I try to increase
a) the buffer with a multiple of page size (parameter -n)
b) the number of thread (parameter -t)
2/ the result is not exactly the same as wc -l give wich imply a
bug while counting \n inside the buffer.
$ time ./build/wc_File_by_chunks_async_buffer -n 4 -t 8 -i
test_100M
1638656 test_100M
real 0m0.106s
user 0m0.101s
sys 0m0.051s
$ time wc -l test_100M
1638400 test_100M
real 0m0.067s
user 0m0.030s
sys 0m0.037s
Thanks again Ali without you It was impossible to me
