On Mon, Apr 6, 2020 at 5:22 AM Thomas Bereknyei <tombe...@gmail.com> wrote:
> Running some experiments with the sqlmaster/sqlworker system shows > that running multiple independent workers results in many jobs being > done multiple times. This is noted in the manual as > > "If you have more than one --sqlworker jobs may be run more than once." > > In simple testing, this is not an occasional thing, but results in > nearly all jobs being performed N times. So I tested: seq 100 | parallel --sqlmaster sqlite3:///fisk/fisk 'sleep 4;echo {}' ( parallel -uj30 --sqlworker sqlite3:///fisk/fisk & parallel -uj30 --sqlworker sqlite3:///fisk/fisk & parallel -uj30 --sqlworker sqlite3:///fisk/fisk & parallel -uj30 --sqlworker sqlite3:///fisk/fisk & wait ) | wc -l This gives 100..103 on my machine. Thus 0..3 jobs were run twice. So I cannot reproduce the error you see. That is not to say that you have not found a bug, but I cannot reproduce it on my system. /Ole