Hi, Thanks ... I will give it a shot tomorrow.
SEM support is indeed needed ... or else everything blows right up in your face. In freebsd < 7.2 you needed to compile your kernel to make it work, but now its default in the kernel. Think we need Andrew to make a section about this in the wiki/documentation. mvh On Mon, Jul 18, 2011 at 10:53 PM, Johan Hendriks <[email protected]> wrote: > I have read somewhere that python on FreeBSD need SEM support. > So go to /usr/ports/lang/python2x where x is your version of perl and > do a make config and select SEM. > > # cd /usr/ports/lang/python27 > # make config > [X] SEM Use POSIX semaphores (experimental) > > I did not test without SEM! > So maybe it work without SEM , but i do not know, so to be sure > recompile python! > > I created the directory /usr/local/etc/baruwa > within this dir i create the file baruwa with the following content. > > # mkdir /usr/local/etc/baruwa > # cd /usr/local/etc/baruwa/ > # vi baruwa > > # Baruwa Celery worker daemon configuration > # > CELERYD_NODES=$(hostname -s) > CELERYD_CHDIR="/usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/" > CELERYD_MULTI="$CELERYD_CHDIR/manage.py celeryd_multi" > CELERYD_OPTS="--time-limit=300 --concurrency=8" > CELERY_CONFIG_MODULE="celeryconfig" > CELERYD_LOG_FILE=/var/log/celery/celeryd.log > CELERYD_PID_FILE="/var/run/celery/celeryd.pid" > CELERYD_USER="celery" > CELERYD_GROUP="celery" > export DJANGO_SETTINGS_MODULE="settings" > > Ok rabbitMQ > > I use my setup in a jail. > So i needed to make sure rabbitmq did not listen on localhost (127.0.0.1) > I did the following > # vi /usr/local/etc/rabbitmq/rabbitmq.conf > > NODE_IP_ADDRESS=192.168.1.21 > > then start rabbitmq > # /usr/local/etc/rc.d/rabbitmq start > > As per baruwa install documentation do the following. > > # rabbitmqctl add_user baruwa yourbrokerpassword > # rabbitmqctl add_vhost baruwa > # rabbitmqctl set_permissions -p baruwa baruwa ".*" ".*" ".*" > # rabbitmqctl delete_user guest > > This is my > /usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/settings.py > (celery part) > #celery > CELERY_CONCURRENCY = 20 > CELERY_DISABLE_RATE_LIMITS = True > BROKER_HOST = "192.168.1.21" > BROKER_PORT = 5672 > BROKER_USER = "baruwa" > BROKER_PASSWORD = "yourbrokerpassword" > BROKER_VHOST = "baruwa" > > Where BROWKER_PASSWORD is set by the rabbitmq add_user command > To make things easy i created a link to > /usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/settings.py > in the /usr/local/etc/baruwa dir > > > Then i added the user celery > # adduser > Make sure the shell is /usr/sbin/nologin > > After the user is created you should see something like this in your > passwd file > # vipw > celery:*:1001:1001::0:0:Celery deamon:/home/celery:/usr/sbin/nologin > > And in your /etc/group file > # vi /etc/group > celery:*:1001: > > then i created the celery folder in /var/log > # cd /var/log > # mkdir celery > # chown celery:celery celery > > also change the group of the mailscanner quarantine folder. > > # chown -R postfix:celery quarantine > > Create the startup file for the deamon > The content of this file is one line !!!!! > > # vi /usr/local/etc/rc.d/celeryd.sh > > /usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa//manage.py > celeryd_multi start celery --uid=celery --gid=celery > '--workdir="/usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/"' > --pidfile=/var/run/celery/celeryd.pid > --logfile=/var/log/celery/celeryd.log --loglevel=INFO > --cmd=/usr/local/bin/celeryd --time-limit=300 --concurrency=8 > > > THE CELERY DEAMON DOES NOT START AFTER A REBOOT. > so you have to manually start it > # /usr/local/etc/rc.d/celeryd.sh start > > I did not find time to find a way to make it startable at boot. > > But as far as i know celery is only used for releasing messages, it does > not disrupt the working of your mailscanner if it should reboot by accident. > You find out when you try to release a message and it does not work. > But he!, we are on FreeBSD so that would not happen much :D > > check your /var/log/celery/celeryd.log file !! > > Hope this helps. > be sure to edit files to your setup like the python path and so on. > And i could have made a typo! > > regards, > Johan Hendriks > Double L Automatisering > > > > Mikael Syska schreef: >> Hi list and especially Johan Hendriks(since you are also a FreeBSD user), >> >> I read that you talked about making a special "FreeBSD howto" ... I >> have the exact same problems and I'm also 100% lost. >> >> Havent tried this after I found the post in my mail archive, but just >> wanted to make sure you havent already created a wonderful wiki for >> the FreeBSD users. >> >> Everything runs fine ... except the devil most users are having >> problems with ATM. Cerleryd .... >> >> Can't really figure out how its all connected, init files, config >> files, module configuration, manage.py, you name it .... completely >> lost and now I need to rest before I give it a try again tomorrow. >> >> mvh >> >> On Tue, Jun 28, 2011 at 10:45 PM, Johan Hendriks >> <[email protected]> wrote: >>> On Tue, 28 Jun 2011 21:44:28 +0200, Andrew Colin Kissa wrote: >>>> On 28 Jun 2011, at 9:05 PM, Johan Hendriks wrote: >>>> >>>>> /usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/manage.py: >>>>> error: no such option: --uid >>>> >>>> you may have to run celeryd from /etc/rc until am able to provide you >>>> with a proper >>>> init script for freebsd, the generic script does not seem to do it. >>>> >>>> -- >>>> Baruwa - www.baruwa.org >>>> >>>> _______________________________________________ >>>> Baruwa mailing list >>>> [email protected] >>>> http://lists.baruwa.org/mailman/listinfo/baruwa >>> Ok i am a little further now. >>> >>> if i run the following, the celeryd is starting ok and works. (as far >>> as i can see that is) >>> >>> /usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa//manage.py >>> celeryd_multi start beasty --uid=celery --gid=celery >>> '--workdir="/usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/"' >>> --pidfile=/var/run/celery/celeryd.pid >>> --logfile=/var/log/celery/celeryd.log --loglevel=INFO >>> --cmd=/usr/local/bin/celeryd --time-limit=300 --concurrency=8 >>> >>> >>> The strange thing is that i got that line from doing a sh -x >>> /usr/local/etc/rc.d/celery start >>> >>> it gave me this output >>> beasty # sh -x /usr/local/etc/rc.d/celery start >>> + set -e >>> + DEFAULT_PID_FILE=/var/run/celery/celeryd.pid >>> + DEFAULT_LOG_FILE=/var/log/celery/celeryd.log >>> + DEFAULT_CELERYD=/usr/local/bin/celeryd >>> + DEFAULT_LOG_LEVEL=INFO >>> + DEFAULT_NODES=celery >>> + test -f /usr/local/etc/baruwa/baruwa >>> + . /usr/local/etc/baruwa/baruwa >>> + hostname -s >>> + CELERYD_NODES=beasty >>> + >>> CELERYD_CHDIR=/usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/ >>> + >>> CELERYD_MULTI='/usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa//manage.py >>> celeryd_multi' >>> + CELERYD_OPTS='--time-limit=300 --concurrency=8' >>> + CELERY_CONFIG_MODULE=celeryconfig >>> + CELERYD_LOG_FILE=/var/log/celery/celeryd.log >>> + CELERYD_PID_FILE=/var/run/celery/celeryd.pid >>> + CELERYD_USER=celery >>> + CELERYD_GROUP=celery >>> + export DJANGO_SETTINGS_MODULE=settings >>> + CELERYD_PID_FILE=/var/run/celery/celeryd.pid >>> + CELERYD_LOG_FILE=/var/log/celery/celeryd.log >>> + CELERYD_LOG_LEVEL=INFO >>> + >>> CELERYD_MULTI='/usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa//manage.py >>> celeryd_multi' >>> + CELERYD=/usr/local/bin/celeryd >>> + CELERYD_NODES=beasty >>> + export CELERY_LOADER >>> + [ -n '' ] >>> + [ -n celery ] >>> + DAEMON_OPTS=' --uid=celery' >>> + [ -n celery ] >>> + DAEMON_OPTS=' --uid=celery --gid=celery' >>> + [ -n >>> /usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/ ] >>> + DAEMON_OPTS=' --uid=celery --gid=celery >>> --workdir="/usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/"' >>> + export >>> PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/root/bin:/usr/local/sbin:/sbin:/usr/sbin:/usr/bin:/usr/local/bin >>> + check_dev_null >>> + [ ! -c /dev/null ] >>> + start_workers >>> + >>> /usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa//manage.py >>> celeryd_multi start beasty --uid=celery --gid=celery >>> '--workdir="/usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/"' >>> --pidfile=/var/run/celery/celeryd.pid >>> --logfile=/var/log/celery/celeryd.log --loglevel=INFO >>> --cmd=/usr/local/bin/celeryd --time-limit=300 --concurrency=8 >>> celeryd-multi v2.2.6 >>>> Starting nodes... >>> > beasty.beasty.double-l.local: OK >>> + exit 0 >>> >>> If i run the file /usr/local/etc/rc.d/celery it does not work, if i use >>> the same command that the init script uses, it runs fine? >>> I do not know the difference between the 2!!! >>> >>> So to sum up >>> >>> using the script >>> beasty # /usr/local/etc/rc.d/celery start >>> celeryd-multi v2.2.6 >>>> Starting nodes... >>> > beasty.beasty.double-l.local: OK >>> beasty # >>> >>> I see this in the log /var/log/celery/celeryd.log >>> >>> [2011-06-28 22:16:47,302: WARNING/MainProcess] -------------- >>> [email protected] >>> ty.double-l.local v2.2.6 >>> ---- **** ----- >>> --- * *** * -- [Configuration] >>> -- * - **** --- . broker: amqplib://guest@localhost:5672/ >>> - ** ---------- . loader: celery.loaders.default.Loader >>> - ** ---------- . logfile: /var/log/celery/celeryd.log@INFO >>> - ** ---------- . concurrency: 8 >>> - ** ---------- . events: OFF >>> - *** --- * --- . beat: OFF >>> -- ******* ---- >>> --- ***** ----- [Queues] >>> -------------- . celery: exchange:celery (direct) >>> binding:celery >>> >>> >>> [Tasks] >>> [2011-06-28 22:16:47,315: INFO/PoolWorker-1] child process calling >>> self.run() >>> [2011-06-28 22:16:47,318: INFO/PoolWorker-2] child process calling >>> self.run() >>> [2011-06-28 22:16:47,325: INFO/PoolWorker-4] child process calling >>> self.run() >>> [2011-06-28 22:16:47,327: INFO/PoolWorker-3] child process calling >>> self.run() >>> [2011-06-28 22:16:47,328: INFO/PoolWorker-5] child process calling >>> self.run() >>> [2011-06-28 22:16:47,329: INFO/PoolWorker-6] child process calling >>> self.run() >>> [2011-06-28 22:16:47,335: INFO/PoolWorker-7] child process calling >>> self.run() >>> [2011-06-28 22:16:47,338: INFO/PoolWorker-8] child process calling >>> self.run() >>> [2011-06-28 22:16:47,340: WARNING/MainProcess] >>> [email protected] >>> al has started. >>> [2011-06-28 22:16:50,346: ERROR/MainProcess] Consumer: Connection >>> Error: Socket >>> closed. Trying again in 2 seconds... >>> And so on >>> >>> For some reason it is guest@ >>> >>> However >>> if i run the following: >>> beasty # >>> /usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa//manage.py >>> celeryd_multi start beasty --uid=celery --gid=celery >>> '--workdir="/usr/local/lib/python2.7/site-packages/baruwa-1.1.0-py2.7.egg/baruwa/"' >>> --pidfile=/var/run/celery/celeryd.pid >>> --logfile=/var/log/celery/celeryd.log --loglevel=INFO >>> --cmd=/usr/local/bin/celeryd --time-limit=300 --concurrency=8 >>> celeryd-multi v2.2.6 >>>> Starting nodes... >>> > beasty.beasty.double-l.local: OK >>> beasty # >>> >>> >>> Then i see this in the log. >>> >>> [2011-06-28 22:22:49,961: WARNING/MainProcess] -------------- >>> [email protected] >>> ty.double-l.local v2.2.6 >>> ---- **** ----- >>> --- * *** * -- [Configuration] >>> -- * - **** --- . broker: >>> amqplib://[email protected]:5672/baruwa >>> - ** ---------- . loader: djcelery.loaders.DjangoLoader >>> - ** ---------- . logfile: /var/log/celery/celeryd.log@INFO >>> - ** ---------- . concurrency: 8 >>> - ** ---------- . events: OFF >>> - *** --- * --- . beat: OFF >>> -- ******* ---- >>> --- ***** ----- [Queues] >>> -------------- . beasty.double-l.local: exchange:default (direct) >>> binding:bea >>> sty.double-l.local >>> . default: exchange:default (direct) >>> binding:default >>> >>> [Tasks] >>> . preview-message >>> . process-quarantine >>> . process-quarantined-msg >>> . release-message >>> . test-smtp-server >>> [2011-06-28 22:22:49,978: INFO/PoolWorker-1] child process calling >>> self.run() >>> [2011-06-28 22:22:49,982: INFO/PoolWorker-2] child process calling >>> self.run() >>> [2011-06-28 22:22:49,986: INFO/PoolWorker-3] child process calling >>> self.run() >>> [2011-06-28 22:22:49,994: INFO/PoolWorker-4] child process calling >>> self.run() >>> [2011-06-28 22:22:49,995: INFO/PoolWorker-5] child process calling >>> self.run() >>> [2011-06-28 22:22:49,999: INFO/PoolWorker-8] child process calling >>> self.run() >>> [2011-06-28 22:22:49,999: INFO/PoolWorker-7] child process calling >>> self.run() >>> [2011-06-28 22:22:49,999: INFO/PoolWorker-6] child process calling >>> self.run() >>> [2011-06-28 22:22:50,002: WARNING/MainProcess] >>> [email protected] >>> al has started. >>> >>> It runs fine as far i can see. >>> >>> Thanks again >>> regards, >>> Johan Hendriks >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Baruwa mailing list >>> [email protected] >>> http://lists.baruwa.org/mailman/listinfo/baruwa >>> >> _______________________________________________ >> Baruwa mailing list >> [email protected] >> http://lists.baruwa.org/mailman/listinfo/baruwa >> Keep Baruwa FREE - http://pledgie.com/campaigns/12056 >> > > _______________________________________________ > Baruwa mailing list > [email protected] > http://lists.baruwa.org/mailman/listinfo/baruwa > Keep Baruwa FREE - http://pledgie.com/campaigns/12056 > _______________________________________________ Baruwa mailing list [email protected] http://lists.baruwa.org/mailman/listinfo/baruwa Keep Baruwa FREE - http://pledgie.com/campaigns/12056
