So, I've realised this is actually possible - albeit with a little bit of
duplication - with Thinking Sphinx v3 (a release candidate is out, and I should
have the full release out today too, with a bit of luck):
# app/indices/article_index.rb
# Whatever you normally use with :delta is the argument here.
processor = ThinkingSphinx::Deltas.processor_for(true)
options = {:with => :active_record, :delta_processor => processor, :delta?
=> false}
ThinkingSphinx::Index.define_for :article, options do
# core index definition
indexes title, body
has created_at, user_id
end
ThinkingSphinx::Index.define_for :article, options.merge(:delta? => true) do
# delta index definition
indexes title, body
has created_at, user_id
set_property :sql_range_step => 10_000_000
end
What are your thoughts on this?
--
Pat
On 02/01/2013, at 8:41 AM, Johann Tagle wrote:
> Thanks Pat and sorry for late reply - got caught up with the holidays (Happy
> New Year!). For the new support maybe something like:
>
> define_index do
> .
> .
> .
> enable_delta do
> #put delta-specific settings here
> end
> end
>
> Thanks again!
>
> Johann
>
> On Sunday, December 23, 2012 9:11:22 AM UTC+8, Pat Allan wrote:
> Hi Johann
> Yeah, currently there's no way to have settings just for the delta index.
> TS's DSL doesn't allow for it… I'm open to suggestions on how it could, but
> this would very likely be something that I add to TS 3, not earlier versions.
>
> You could just set the sql_range_step to be quite large generally? May not
> have too much of an impact for full indexing (indeed, in some situations can
> speed things up).
>
> Cheers
>
> --
> Pat
>
> On 19/12/2012, at 3:38 PM, Johann Tagle wrote:
>
> > I guess the solution for me is to add an sql_range_step with a high value
> > in the source definition of the delta. Hmmm maybe it would be good if TS
> > can allow index configuration settings for main and delta.
> >
> > On Monday, December 17, 2012 5:58:48 PM UTC+8, Johann Tagle wrote:
> > Our setup:
> >
> > - multiple app servers, no sphinx installed.
> > - one dedicated server running sphinx
> > - TS is still used to generated the sphinx config file, then loaded to the
> > sphinx server
> > - TS will set delta=1 for new/updated records, but we let cron jobs on the
> > sphinx server do the full and delta index updating
> >
> > Everything is working, except I see an inefficiency with the way the delta
> > index is updated for one model. This model has around 5M records, with
> > about 10k being updated daily. Just before writing this, I ran the
> > statement defined by sql_query_range and saw that the values for the min
> > and max ids are at the opposite extremes, meaning the delta reindexing will
> > practically scan through the whole table when there are just 5k records
> > with delta=1. We indexed the delta column and so doing a simple "select *
> > from table where delta=1" is quite fast.
> >
> > Is there anything we can put within define_index or somewhere so that it
> > will generate the config for the delta source to not use ranged queries?
> >
> > Thanks
> >
> > Johann
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Thinking Sphinx" group.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msg/thinking-sphinx/-/w4FxEspJkoUJ.
> > To post to this group, send email to [email protected].
> > To unsubscribe from this group, send email to
> > [email protected].
> > For more options, visit this group at
> > http://groups.google.com/group/thinking-sphinx?hl=en.
>
>
>
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Thinking Sphinx" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/thinking-sphinx/-/M9dpgxcEZY4J.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/thinking-sphinx?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Thinking Sphinx" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/thinking-sphinx?hl=en.
