i'm running my program (bash script to be exact), that does process URL list by fetching each one with curl and then process etc..

to make it faster i run 30-40 of them in parallel to maximize speed, as often there are network stalls and doing many transfers in parallel speed it up. all is run with nice -n 20 to not interfere too much with other tasks.

it runs on Core2Duo SMP system with 6.2p11/amd64

the script takes some CPU time so system it's quite loaded.

when i first start it it takes about 25% user time and 20% system time.

after half a day doing exactly the same thing, network having same load, after processing few millions URL, system time jumps to over 50%, idle gets close to zero, system load goes up.

that's strange but it goes on, after about next half a day system crashes with "sleeping thread owns a non-sleepable lock".

no core dump, after panic kernel hangs.

is there a fix for that? unix is for running multiple parallel tasks, so it should not cause crashes!

freebsd-questions@freebsd.org mailing list
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Reply via email to