On Mon, Apr 6, 2020 at 5:22 AM Thomas Bereknyei <tombe...@gmail.com> wrote:
> Running some experiments with the sqlmaster/sqlworker system shows
> that running multiple independent workers results in many jobs being
> done multiple times. This is noted in the manual as
>
> "If you have more than one --sqlworker jobs may be run more than once."
>
> In simple testing, this is not an occasional thing, but results in
> nearly all jobs being performed N times.
So I tested:
seq 100 | parallel --sqlmaster sqlite3:///fisk/fisk 'sleep 4;echo {}'
(
parallel -uj30 --sqlworker sqlite3:///fisk/fisk &
parallel -uj30 --sqlworker sqlite3:///fisk/fisk &
parallel -uj30 --sqlworker sqlite3:///fisk/fisk &
parallel -uj30 --sqlworker sqlite3:///fisk/fisk &
wait
) | wc -l
This gives 100..103 on my machine. Thus 0..3 jobs were run twice.
So I cannot reproduce the error you see.
That is not to say that you have not found a bug, but I cannot
reproduce it on my system.
/Ole
