Re: Fwd: Pilerimport and cron

[Philip Deubner]

Thank you for your support and your suggestion. In the meantime I was able to get the script running. Silly me - I simply forgot that cron needs an absolute path - so it simply had to be

instead of



One day I'll be able to see these errors in beforehand...
Thank you for your help, problem solved.
Br,

Philip

Am 02.02.2015 um 22:30 schrieb Janos SUTO:

*/15 * * * * cd /home/piler && */usr/local/bin/*pilerimport -i host -u user -p verysecretandstrongpassword */15 * * * * cd /home/piler && pilerimport -i host -u user -p verysecretandstrongpassword

well, it's true that pilerimport has no pid file, locking, and
other fancy features. The reason behind is that you can run multiple
pilerimport processes speeding up the import - provided that you
have the required horse power.

However you may emulate lockfile support, see util/indexer.delta.sh
for an example:

#!/bin/bash

MAINTMPFILE=/var/run/piler/import.tmp



date > $MAINTMPFILE

function finish {
   rm -f $MAINTMPFILE
}

trap finish EXIT

cd /var/piler/imap
/usr/local/bin/pilerimport -i ... -u ... -p .....


Thus if pilerimport is running for a long time, the next iteration will
quit, and syslogs the problem.


Janos



On 2015-02-02 18:43, Philip Deubner wrote:

if [ -f $MAINTMPFILE ]; then echo "import is already running. It started at "`cat $MAINTMPFILE` | logger -p mail.error ; exit 1; fi

Thank you for your feedback, just some more questions:
The most important question would be how to get the last two lines
into a periodical pull via cron.

* * * * * cd /var/run/pilerimport && su piler -c "pilerimport -i
'host' -u 'user' -p 'password' -P 143 -x '' "

This looks just like my initial setup:

*/15 * * * * cd /home/piler && pilerimport -i host -u user -p
verysecretandstrongpassword

Where is the difference?

run the following from terminal

mkdir -P /var/run/pilerimport
chmod 777 /var/run/pilerimport
cd /var/run/pilerimport






On 2/2/2015 6:21 PM, Philip Deubner wrote:

su piler -c "pilerimport -i 'host' -u 'user' -p 'password' -P 143 -x '' " ^^note there are single ' and " double quotes.. the -x '' <= are 2x single quotes BTW, contact us if you need a proper admin/company to manage your piler server.

I disagree with the approach from experience.

Rather write a simple shell script that does a few things.











Make sure piler import is run as piler user

*** su piler -c "pilerimport"

*** set a pid in /var/run/ to prevent the script from spawning multiple times.. Your idea is to spawn a new process every 15mins.. however the pilerimport could still be running... this will eventually lead to a lock of either the imap server or your pilerimport host. Never use the harddrive as a store for temporary files, you will lag your disk i/o and cause excessive wear. *** set the tempory directory to /var/shm this is a "ram drive" available on all linux systems, so the temp files will be stored in memory reducing disk i/o to zero and reducing lag. *** in the script create a temp dir with mkdir -p /var/shm/pilerimport and set the permission to 777, to avoid any permission errors.

Well, that would definetely be a fancy script... But honestly, I currently have no idea on how to do such (setting the pid and setting the temporary directory). So if you have an idea on where I can search to solve these issues, please tell me. Apart from that, the problem would likely be the same, wouldn't it? Finally, I would have to call the script using cron ans telling cron to have the commands run in a certain directory - which already currently does not work. So, from my point of view, this should be the first problem to deal with - or am I wrong?

bg,

Philip

smime.p7s
Description: S/MIME Cryptographic Signature