Engine Yard Env.
Preloaded with Sphinx 0.9.8.1-release (r1533). (for now can't change
versions)
Rails gem: thinking-sphinx (2.0.12)
Situation. Index on a single main table. This table has joins to at least
3 others. One of those is for indexable content the others are used for
attributes.
On app deploy
- bundle exec rake thinking_sphinx:configure
- bundle exec rake thinking_sphinx:index
- monit reload (to my limited understanding, this stops the service,
reloads the config, and then restarts it)
At this point if new content had been added to the indexed table since the
last time the index was rebuilt and before the deploy we do see the index
file grow in size and the new content is searchable both through the search
command line and through the thinking_sphinx gem in the console.
Where we run into issues is getting new data in the index while the service
is running. We had a cron running every 30 mins that would execute "cd
/data/ep_dev/current && RAILS_ENV=alpha bundle exec rake
thinking_sphinx:index" but no new data was ever picked up. On a whim this
was changed to reindex, for that we see the file timestamp changing but no
change in size or new data. Looking at the soruce code for the reindex
rake tasks the last think that reindex does after running index is touch
the index files.
Lastly, we tried bundle exec rake thinking_sphinx:rebuild. Which from
looking at the code executes a stop, index, and then a start. When we do
this we DO see the size of the index grow and we get new searchable data.
So it appears that the indexer doesn't like to run while sphinx is running.
At this point I tried running the spinx indexder command in the hopes for
more info. Got some, kinda.
app_master:yellow i-e60de596 current $ indexer -c
/data/ep_dev/current/config/sphinx/alpha.sphinx.conf --all --rotate
Sphinx 0.9.8.1-release (r1533)
Copyright (c) 2001-2008, Andrew Aksyonoff
using config file '/data/ep_dev/current/config/sphinx/alpha.sphinx.conf'...
indexing index 'learning_content_article_core'...
collected 77 docs, 0.5 MB
Segmentation fault
Unfortunatly the segmentation fault does not produce a core.
At this point we have a 'working' solution by issuing a nightly rebuild but
that's not really acceptable. Any insight or thoughts of configs to tweak
would be greatly appreciated. Below is our conf file(s).
app_master:yellow i-e60de596 config $ cat sphinx.yml
alpha:
searchd_log_file: /var/log/engineyard/sphinx/ep_dev/searchd.log
query_log_file: /var/log/engineyard/sphinx/ep_dev/query.log
pid_file: /var/run/sphinx/ep_dev.pid
address: localhost
port: 9312
mem_limit: 32M
config_file: /data/ep_dev/current/config/sphinx/alpha.sphinx.conf
searchd_file_path: /var/log/engineyard/sphinx/ep_dev/indexes
app_master:yellow i-e60de596 config $ cat alpha.sphinx.conf
indexer
{
mem_limit = 32M
}
searchd
{
address = localhost
port = 9312
log = /var/log/engineyard/sphinx/ep_dev/searchd.log
query_log = /var/log/engineyard/sphinx/ep_dev/query.log
pid_file = /var/run/sphinx/ep_dev.pid
}
source learning_content_article_core_0
{
type = mysql
sql_host = XXXXX
sql_user = XXXXX
sql_pass = XXXXX
sql_db = ep_dev
sql_query_pre = SET NAMES utf8
sql_query_pre = SET TIME_ZONE = '+0:00'
sql_query = XXXXX
sql_query_range = SELECT IFNULL(MIN(`id`), 1), IFNULL(MAX(`id`), 1) FROM
`learning_content_articles`
sql_attr_uint = sphinx_internal_id
sql_attr_uint = sphinx_deleted
sql_attr_uint = class_crc
sql_attr_bool = is_public
sql_attr_bool = featured
sql_attr_timestamp = created_at
sql_attr_timestamp = updated_at
sql_attr_str2ordinal = category
sql_attr_str2ordinal = topic
sql_query_info = SELECT * FROM `learning_content_articles` WHERE `id` =
(($id - 0) / 1)
}
index learning_content_article_core
{
path =
/var/log/engineyard/sphinx/ep_dev/indexes/learning_content_article_core
charset_type = utf-8
source = learning_content_article_core_0
}
index learning_content_article
{
type = distributed
local = learning_content_article_core
}
--
You received this message because you are subscribed to the Google Groups
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
