Can you log the output of the cron job, and see if the indexer is getting the same segmentation fault in that scenario as well? If so, it doesn't seem to be an issue with Thinking Sphinx, but Sphinx itself, and not one I'm familiar with. Perhaps the folks on the Sphinx forums may have better suggestions?
http://sphinxsearch.com/forum/ Though if the cron job hits a different error (or none at all), I'm certainly curious as to what it could be. Cheers -- Pat On 01/02/2013, at 4:51 AM, [email protected] wrote: > Engine Yard Env. > Preloaded with Sphinx 0.9.8.1-release (r1533). (for now can't change > versions) > Rails gem: thinking-sphinx (2.0.12) > > Situation. Index on a single main table. This table has joins to at least 3 > others. One of those is for indexable content the others are used for > attributes. > On app deploy > - bundle exec rake thinking_sphinx:configure > - bundle exec rake thinking_sphinx:index > - monit reload (to my limited understanding, this stops the service, reloads > the config, and then restarts it) > > At this point if new content had been added to the indexed table since the > last time the index was rebuilt and before the deploy we do see the index > file grow in size and the new content is searchable both through the search > command line and through the thinking_sphinx gem in the console. > > Where we run into issues is getting new data in the index while the service > is running. We had a cron running every 30 mins that would execute "cd > /data/ep_dev/current && RAILS_ENV=alpha bundle exec rake > thinking_sphinx:index" but no new data was ever picked up. On a whim this > was changed to reindex, for that we see the file timestamp changing but no > change in size or new data. Looking at the soruce code for the reindex rake > tasks the last think that reindex does after running index is touch the index > files. > > Lastly, we tried bundle exec rake thinking_sphinx:rebuild. Which from > looking at the code executes a stop, index, and then a start. When we do > this we DO see the size of the index grow and we get new searchable data. So > it appears that the indexer doesn't like to run while sphinx is running. > > At this point I tried running the spinx indexder command in the hopes for > more info. Got some, kinda. > > app_master:yellow i-e60de596 current $ indexer -c > /data/ep_dev/current/config/sphinx/alpha.sphinx.conf --all --rotate > Sphinx 0.9.8.1-release (r1533) > Copyright (c) 2001-2008, Andrew Aksyonoff > > using config file '/data/ep_dev/current/config/sphinx/alpha.sphinx.conf'... > indexing index 'learning_content_article_core'... > collected 77 docs, 0.5 MB > Segmentation fault > > Unfortunatly the segmentation fault does not produce a core. > > At this point we have a 'working' solution by issuing a nightly rebuild but > that's not really acceptable. Any insight or thoughts of configs to tweak > would be greatly appreciated. Below is our conf file(s). > > app_master:yellow i-e60de596 config $ cat sphinx.yml > alpha: > searchd_log_file: /var/log/engineyard/sphinx/ep_dev/searchd.log > query_log_file: /var/log/engineyard/sphinx/ep_dev/query.log > pid_file: /var/run/sphinx/ep_dev.pid > address: localhost > port: 9312 > mem_limit: 32M > config_file: /data/ep_dev/current/config/sphinx/alpha.sphinx.conf > searchd_file_path: /var/log/engineyard/sphinx/ep_dev/indexes > > app_master:yellow i-e60de596 config $ cat alpha.sphinx.conf > indexer > { > mem_limit = 32M > } > > searchd > { > address = localhost > port = 9312 > log = /var/log/engineyard/sphinx/ep_dev/searchd.log > query_log = /var/log/engineyard/sphinx/ep_dev/query.log > pid_file = /var/run/sphinx/ep_dev.pid > } > > source learning_content_article_core_0 > { > type = mysql > sql_host = XXXXX > sql_user = XXXXX > sql_pass = XXXXX > sql_db = ep_dev > sql_query_pre = SET NAMES utf8 > sql_query_pre = SET TIME_ZONE = '+0:00' > sql_query = XXXXX > sql_query_range = SELECT IFNULL(MIN(`id`), 1), IFNULL(MAX(`id`), 1) FROM > `learning_content_articles` > sql_attr_uint = sphinx_internal_id > sql_attr_uint = sphinx_deleted > sql_attr_uint = class_crc > sql_attr_bool = is_public > sql_attr_bool = featured > sql_attr_timestamp = created_at > sql_attr_timestamp = updated_at > sql_attr_str2ordinal = category > sql_attr_str2ordinal = topic > sql_query_info = SELECT * FROM `learning_content_articles` WHERE `id` = > (($id - 0) / 1) > } > > index learning_content_article_core > { > path = > /var/log/engineyard/sphinx/ep_dev/indexes/learning_content_article_core > charset_type = utf-8 > source = learning_content_article_core_0 > } > > index learning_content_article > { > type = distributed > local = learning_content_article_core > } > > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/thinking-sphinx?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/thinking-sphinx?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
