Hello,
First time using GNU parallel. I hope it is the right tool for the job,
and that you might give some advice :)
Here is the job:
- 350 independent php data processing scripts, run every 8 hours. Uses
up to 100MB RAM per process, but most will use only 10-20MB. Scripts run
1-90 minutes.
example: php crons.php -run_job=allstarhealth > allstarhealth.txt 2>&1
- Amazon EC2 Linux Fedora instance. VM with one core, one CPU, and 1G
memory
- GOAL: launch the php scripts in parallel on a single core, and remain
below 10% CPU (AWS Free Tier) and below 1GB RAM utilization.
My question is, what are the appropriate options to suit that goal?
Based on the docs, this is what I'm guessing. Please give your opinion!:
# parallel \
--max-procs N \
--semaphore \
--load 0.1 \ # not sure if this is for CPU or RAM,
assume CPU. Need to "wait until CPU <= 9%" before launching next process
--memfree size \ # useful?
--noswap \ # we do have a swap file, but performance
is very slow
-a file_list.sh
thanks in advance
