ts:start only controls the search daemon (searchd) process. Delta indexes are completely separate, and run by the rails app - which in your case, is done within Passenger (running as root).
-- Pat On 24/05/2009, at 5:55 PM, Elad Meidar wrote: > > yes, you are right. > > i did 'ps aux | grep index' to see who is running 'indexer' calls > > r...@socialninjaz:~# ps aux | grep index > web 29873 0.0 6.5 150724 68792 pts/2 S+ 00:13 0:02 /opt/ > ruby-enterprise-1.8.6-20090201/bin/ruby /usr/bin/rake > RAILS_ENV=production ts:index > root 31432 0.6 0.2 30016 2336 ? S 00:53 0:00 /usr/ > local/bin/indexer --config /var/www/statussearch2/releases/ > 20090523013634/config/production.sphinx.conf status_update_delta > root 31434 0.0 0.0 3944 604 pts/0 R+ 00:53 0:00 grep > index > > > so root is running the delta updates... any idea why? ts:start was > made with 'web' as you can see here.... > > > > On May 24, 7:47 pm, Pat Allan <[email protected]> wrote: >> Looks like passenger is being run as root - I'm not sure why the >> first >> apache is root and the rest not, but it's probably passenger that >> involves talking to Rails. >> >> Something I'm not sure was made clear in previous emails (so sorry if >> I'm explaining what's already obvious): delta indexing is invoked by >> the rails stack, not a rake task, so that means passenger in this >> case, and so it's using passenger's permissions to run indexer. I >> think that's the source of the problem. >> >> -- >> Pat >> >> On 24/05/2009, at 4:40 PM, Elad Meidar wrote: >> >> >> >>> Running on passenger, simple install, nothing fancy. i can post >>> whatever you want :) >> >>> i am confused too, probably cause' i don't know what exactly the >>> 'searchd' and 'indexer' commands are doing with these files... >> >>> i did 'ps aux | grep apache' (thinking that would be the right >>> command >>> to invoke in a passenger installation case). >> >>> root 3964 0.0 0.7 156428 8104 ? Ss May19 0:44 / >>> usr/ >>> sbin/apache2 -k start >>> root 3985 0.0 0.3 152440 3712 ? Sl May19 0:01 / >>> opt/ >>> ruby-enterprise-1.8.6-20090201/lib/ruby/gems/1.8/gems/ >>> passenger-2.2.2/ >>> ext/apache2/ApplicationPoolServerExecutable 0 /opt/ruby- >>> enterprise-1.8.6-20090201/lib/ruby/gems/1.8/gems/passenger-2.2.2/ >>> bin/ >>> passenger-spawn-server /opt/ruby-enterprise-1.8.6-20090201/bin/ >>> ruby / >>> tmp/passenger.3964 >>> www-data 4057 0.0 0.5 156428 5572 ? S 17:20 0:00 / >>> usr/ >>> sbin/apache2 -k start >>> www-data 6055 0.0 0.5 156428 5596 ? S 09:34 0:00 / >>> usr/ >>> sbin/apache2 -k start >>> www-data 19524 0.0 0.5 156428 5580 ? S 12:44 0:00 / >>> usr/ >>> sbin/apache2 -k start >>> www-data 22550 0.0 0.5 156432 5728 ? S May23 0:00 / >>> usr/ >>> sbin/apache2 -k start >>> www-data 22557 0.0 0.5 156564 5736 ? S May23 0:00 / >>> usr/ >>> sbin/apache2 -k start >>> www-data 22820 0.0 0.5 156428 5572 ? S 14:28 0:00 / >>> usr/ >>> sbin/apache2 -k start >>> www-data 22946 0.0 0.5 156560 5700 ? S May23 0:00 / >>> usr/ >>> sbin/apache2 -k start >>> www-data 24681 0.0 0.5 156564 5736 ? S May21 0:00 / >>> usr/ >>> sbin/apache2 -k start >>> www-data 25864 0.0 0.5 156428 5588 ? S 14:56 0:00 / >>> usr/ >>> sbin/apache2 -k start >>> web 28277 0.0 0.0 3948 648 pts/2 S+ 23:38 0:00 >>> grep >>> apache >>> www-data 29885 0.0 0.5 156428 5576 ? S 16:09 0:00 / >>> usr/ >>> sbin/apache2 -k start >> >>> On May 24, 6:59 pm, Pat Allan <[email protected]> wrote: >>>> Getting closer... >> >>>> The question we need to solve: Why are the delta files owned by >>>> root? >>>> Are you running the site via mongrels or passenger? >> >>>> What's the output of `ps aux | grep mongrel`? (if mongrels are what >>>> you're using, of course) >> >>>> -- >>>> Pat >> >>>> On 24/05/2009, at 3:31 PM, Elad Meidar wrote: >> >>>>> Ok, >> >>>>> i re-did my entire deployment all over again, making sure that the >>>>> 'web' user is responsible for all actions taken in the deployment >>>>> process, including thinking sphinx related tasks. >> >>>>> Now, deltas *DO* Appear on search, but i can't re-index: >> >>>>> w...@socialninjaz:/var/www/statussearch2/current$ rake >>>>> RAILS_ENV=production ts:index >>>>> (in /var/www/statussearch2/releases/20090523013634) >>>>> Generating Configuration to /var/www/statussearch2/releases/ >>>>> 20090523013634/config/production.sphinx.conf >>>>> /usr/local/bin/indexer --config /var/www/statussearch2/releases/ >>>>> 20090523013634/config/production.sphinx.conf --all --rotate >>>>> Sphinx 0.9.8.1-release (r1533) >>>>> Copyright (c) 2001-2008, Andrew Aksyonoff >> >>>>> using config file '/var/www/statussearch2/releases/20090523013634/ >>>>> config/production.sphinx.conf'... >>>>> indexing index 'status_update_core'... >>>>> collected 62039 docs, 5.7 MB >>>>> collected 0 attr values >>>>> sorted 0.1 Mvalues, 100.0% done >>>>> sorted 22.3 Mhits, 97.9% done >>>>> total 62039 docs, 5703116 bytes >>>>> total 3146.338 sec, 1812.62 bytes/sec, 19.72 docs/sec >>>>> indexing index 'status_update_delta'... >>>>> FATAL: failed to open /var/www/statussearch2/releases/ >>>>> 20090523013634/ >>>>> db/sphinx/production/status_update_delta.tmp.spl: Permission >>>>> denied, >>>>> will not index. Try --rotate option. >> >>>>> The file exists but under root ownership again. >> >>>>> w...@socialninjaz:/var/www/statussearch2/current$ ls -l db/sphinx/ >>>>> production/ >>>>> total 133328 >>>>> -rw-r--r-- 1 web web 2479160 May 24 20:25 >>>>> status_update_core.spa >>>>> -rw-r--r-- 1 web web 83895816 May 24 20:25 >>>>> status_update_core.spd >>>>> -rw-r--r-- 1 web web 367 May 24 20:25 >>>>> status_update_core.sph >>>>> -rw-r--r-- 1 web web 6302754 May 24 20:25 >>>>> status_update_core.spi >>>>> -rw------- 1 web web 0 May 24 20:26 >>>>> status_update_core.spl >>>>> -rw-r--r-- 1 web web 1266960 May 24 20:25 >>>>> status_update_core.spm >>>>> -rw-r--r-- 1 web web 40304468 May 24 20:25 >>>>> status_update_core.spp >>>>> -rw-r--r-- 1 root root 30960 May 24 22:30 >>>>> status_update_delta.spa >>>>> -rw-r--r-- 1 root root 1165980 May 24 22:30 >>>>> status_update_delta.spd >>>>> -rw-r--r-- 1 root root 367 May 24 22:30 >>>>> status_update_delta.sph >>>>> -rw-r--r-- 1 root root 364375 May 24 22:30 >>>>> status_update_delta.spi >>>>> -rw------- 1 web web 0 May 24 22:30 >>>>> status_update_delta.spl >>>>> -rw-r--r-- 1 root root 15476 May 24 22:30 >>>>> status_update_delta.spm >>>>> -rw-r--r-- 1 root root 514466 May 24 22:30 >>>>> status_update_delta.spp >>>>> -rw-r--r-- 1 root root 0 May 24 22:30 >>>>> status_update_delta.tmp.spl >> >>>>> i made sure that all capistrano activity and cron jobs are >>>>> operated by >>>>> the 'web' user... i don't really know what is going on really... >> >>>>> On May 23, 9:25 pm, Pat Allan <[email protected]> wrote: >>>>>> I guess what I was wondering is whether you were using the 'run' >>>>>> command or the 'sudo' command in your capistrano tasks - I know >>>>>> I've >>>>>> made the mistake of using the latter when 'run' would have been >>>>>> the >>>>>> better choice. >> >>>>>> -- >>>>>> Pat >> >>>>>> On 23/05/2009, at 5:59 PM, Elad Meidar wrote: >> >>>>>>> now SSH. i thought about testing the configuration and running >>>>>>> process >>>>>>> manually before deploying with it. >> >>>>>>> On May 23, 6:34 pm, Pat Allan <[email protected]> wrote: >>>>>>>> How are you running the rake task? Via capistrano? Or ssh'd >>>>>>>> into >>>>>>>> your >>>>>>>> production machine? >> >>>>>>>> -- >>>>>>>> Pat >> >>>>>>>> On 23/05/2009, at 3:23 PM, Elad Meidar wrote: >> >>>>>>>>> i'm running passenger on the default apache user www-data, i >>>>>>>>> didn't >>>>>>>>> change nothing from the default apache/passenger >>>>>>>>> installations. >> >>>>>>>>> i tried a little test.... >> >>>>>>>>> i chown'ed the *detla* files to web:web, just like the *core* >>>>>>>>> files >>>>>>>>> and checked that it really happened. >>>>>>>>> then, i ran "rake RAILS_ENV=production ts:index --rotate" and >>>>>>>>> listed >>>>>>>>> the files again. >> >>>>>>>>> owner was again root. >> >>>>>>>>> On May 23, 4:37 pm, Pat Allan <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> Are your mongrels running as root? Or passenger? This is the >>>>>>>>>> process >>>>>>>>>> that will invoke delta indexing, and thus overwrite the >>>>>>>>>> existing >>>>>>>>>> files >>>>>>>>>> to new ones with root access only. >> >>>>>>>>>> -- >>>>>>>>>> Pat >> >>>>>>>>>> On 23/05/2009, at 1:34 PM, Elad Meidar wrote: >> >>>>>>>>>>> Well, i moved everything to web >>>>>>>>>>> (ts:stop, ts:index, :ts:start after clearing all the db/ >>>>>>>>>>> sphinx >>>>>>>>>>> folder) >> >>>>>>>>>>> but still all the delta files are created under the root >>>>>>>>>>> ownership, i >>>>>>>>>>> really don't know why.. i am sure that only the web user is >>>>>>>>>>> doing >>>>>>>>>>> any >>>>>>>>>>> kind of thinking_sphinx related actions. >>>>>>>>>>> when i manually chown the files to be under the "web" user, >>>>>>>>>>> deltas >>>>>>>>>>> appear on search and everything is awesome. >> >>>>>>>>>>> this is my crontab for the web user... any idea how or who >>>>>>>>>>> is >>>>>>>>>>> changing >>>>>>>>>>> those files ownerships? >> >>>>>>>>>>> */2 * * * * cd /var/www/statussearch2/current/ && rake >>>>>>>>>>> RAILS_ENV=production ts:index --rotate >>>>>>>>>>> * */5 * * * cd /var/www/statussearch2/current/ && rake >>>>>>>>>>> RAILS_ENV=production ts:index >> >>>>>>>>>>> On May 23, 10:20 am, Elad Meidar <[email protected]> wrote: >>>>>>>>>>>> well, the rake tasks are run by the deploying user, which >>>>>>>>>>>> is >>>>>>>>>>>> 'web' >> >>>>>>>>>>>> but i think that there are some cron tasks (--rotate for >>>>>>>>>>>> example) >>>>>>>>>>>> that >>>>>>>>>>>> are run by 'root' >> >>>>>>>>>>>> i'll move everything to 'web' and i'll see where it's >>>>>>>>>>>> heading. >> >>>>>>>>>>>> Thnx. >> >>>>>>>>>>>> On May 23, 2:19 am, James Healy <[email protected]> wrote: >> >>>>>>>>>>>>> Pat Allan wrote: >>>>>>>>>>>>>> You need the web server and the rake tasks to be run by >>>>>>>>>>>>>> the >>>>>>>>>>>>>> same >>>>>>>>>>>>>> user >>>>>>>>>>>>>> - either both by root, or some other user of your choice. >>>>>>>>>>>>>> This >>>>>>>>>>>>>> should >>>>>>>>>>>>>> avoid any permissions issues. >> >>>>>>>>>>>>>> The *easiest* way is probably to run the rake tasks with >>>>>>>>>>>>>> sudo - >>>>>>>>>>>>>> not >>>>>>>>>>>>>> convinced that's the *best* way though. Others may know >>>>>>>>>>>>>> better :) >> >>>>>>>>>>>>> As a general rule you really don't want to run internet >>>>>>>>>>>>> accessible >>>>>>>>>>>>> daemons as root. >> >>>>>>>>>>>>> I personally use the Debian convention of www-data user >>>>>>>>>>>>> and >>>>>>>>>>>>> group >>>>>>>>>>>>> for my >>>>>>>>>>>>> webserver, mongrels and cron triggered rake tasks. It >>>>>>>>>>>>> doesn't >>>>>>>>>>>>> matter too >>>>>>>>>>>>> much which user you use, just pick or create one with >>>>>>>>>>>>> reduced >>>>>>>>>>>>> privileges. You want to minimise the impact of a malicious >>>>>>>>>>>>> user >>>>>>>>>>>>> finding >>>>>>>>>>>>> an exploitable bug in the prcess. >> >>>>>>>>>>>>> -- James Healy <jimmy-at-deefa-dot-com> Sat, >> >> ... >> >> read more ยป > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
