Ah, I never knew that it only changed the values in memory... that's
good to know! And I really should get the kill list stuff into
Thinking Sphinx proper.
Thanks for that info, muchly appreciated.
Cheers
--
Pat
On 28/11/2009, at 12:37 PM, Canvas wrote:
> Hi Pat,
>
> It turns out that "client.update('buy_sell_file_core',
> ['sphinx_deleted'], { 4013 => [1] }) " updates index in RAM only. And
> the original core index stays unchanged. I am now trying sphinx 0.9.9-
> rc2, which includes a new feature sql_query_killlist to deal with this
> issue. I'll let you know when I try it out.
>
> http://www.sphinxsearch.com/forum/view.html?id=2552
>
> Thanks a lot.
>
> Canvas
>
> On Nov 26, 6:25 pm, Pat Allan <[email protected]> wrote:
>> Hmm, I wonder if Sphinx doesn't delete documents that appear in both
>> indexes...
>>
>> Scenarios to try:
>> - delete a record, add a different record, merge and see if the
>> deleted record is kept around or not
>> - flag the core copy as deleted, merge with the empty delta, re-index
>> delta, merge again, see if only the updates are kept.
>>
>> --
>> Pat
>>
>> On 27/11/2009, at 12:48 PM, Canvas wrote:
>>
>>
>>
>>> Hi Pat,
>>
>>> Thanks for your timely reply. client.update() does work. But I
>>> still
>>> have a wierd problem. Following are the steps for me to reproduce
>>> the
>>> problem.
>>
>>> Step 1: build full index
>>> # rake thinking_sphinx:index RAILS_ENV=development
>>
>>> Step 2: make some changes to file 4013
>>
>>> Step 3: build delta index ==> the file 4013 is searchable with both
>>> old and new data
>>> # rake thinking_sphinx:index:delta RAILS_ENV=development
>>
>>> Step 4: flag the file in core index as "sphinx_deleted" ==> the file
>>> is searchable only by new data, so far so good.
>>>>> client.update('buy_sell_file_core', ['sphinx_deleted'], { 4013 =>
>>>>> [1] })
>>
>>> Step 5: merge delta to core
>>> # /usr/local/bin/indexer --config '/workspace/CA/BETA_2/
>>> EconveyancePro/
>>> config/development.sphinx.conf' --rotate --merge buy_sell_file_core
>>> buy_sell_file_delta --merge-dst-range sphinx_deleted 0 0
>>
>>> And now the problem occurs, I can search by both new and old data
>>> again. It doesn't make any sense to me. Why can I search by the old
>>> data again? Isn't the old index supposed to be deleted in Step 4?
>>> Did
>>> I do anything wrong in step 5?
>>
>>> Thank you very much for your help.
>>
>>> Best wishes,
>>
>>> Canvas
>>
>>> On Nov 25, 12:26 am, Pat Allan <[email protected]> wrote:
>>>> Hi Canvas
>>
>>>> Are you using Thinking Sphinx? It adds an internal attribute called
>>>> sphinx_deleted, and sets records' values to 1 when they are deleted
>>>> in
>>>> Ruby code. Then, if you're using the datetime deltas, the merge
>>>> automatically uses the --merge-dst-range option to remove deleted
>>>> items from the index.
>>
>>>> However, if you're using Riddle, then the equivalent call is
>>>> client.update, not UpdateAttributes. I recommend looking at the
>>>> source
>>>> code to get a good understanding of how it all works. Riddle
>>>> documentation is pretty thin on the ground, but I'd like to improve
>>>> it
>>>> over time.
>>
>>>> Hope this helps.
>>
>>>> --
>>>> Pat
>>
>>>> On 25/11/2009, at 8:00 AM, Canvas wrote:
>>
>>>>> Hi there guys,
>>
>>>>> I am currently using sphinx 9.8.1. The following is from sphinx
>>>>> 9.8.1
>>>>> document:
>>
>>>>> " The basic command syntax is as follows:
>>
>>>>> indexer --merge DSTINDEX SRCINDEX [--rotate]
>>
>>>>> Only the DSTINDEX index will be affected: the contents of SRCINDEX
>>>>> will be merged into it. --rotate switch will be required if
>>>>> DSTINDEX
>>>>> is already being served by searchd. The initially devised usage
>>>>> pattern is to merge a smaller update from SRCINDEX into DSTINDEX.
>>>>> Thus, when merging the attributes, values from SRCINDEX will win
>>>>> if
>>>>> duplicate document IDs are encountered. Note, however, that the
>>>>> "old"
>>>>> keywords will not be automatically removed in such cases. For
>>>>> example,
>>>>> if there's a keyword "old" associated with document 123 in
>>>>> DSTINDEX,
>>>>> and a keyword "new" associated with it in SRCINDEX, document 123
>>>>> will
>>>>> be found by both keywords after the merge. You can supply an
>>>>> explicit
>>>>> condition to remove documents from DSTINDEX to mitigate that; the
>>>>> relevant switch is --merge-dst-range:
>>
>>>>> indexer --merge main delta --merge-dst-range deleted 0 0
>>
>>>>> This switch lets you apply filters to the destination index along
>>>>> with
>>>>> merging. There can be several filters; all of their conditions
>>>>> must be
>>>>> met in order to include the document in the resulting mergid
>>>>> index. In
>>>>> the example above, the filter passes only those records where
>>>>> 'deleted' is 0, eliminating all records that were flagged as
>>>>> deleted
>>>>> (for instance, using UpdateAttributes() call). "
>>
>>>>> It seems that I need to use "UpdateAttributs()" call to update
>>>>> full
>>>>> index before merging. My question here is how to call
>>>>> "UpdateAttributes
>>>>> ()" to update the full index to mark the records in the delta
>>>>> index as
>>>>> deleted?
>>
>>>>> By the way, I am using a view which contains a "update_at" column,
>>>>> which is used as the timestamp to catch data in delta index.
>>
>>>>> Any suggestion is appreciated. Thanks.
>>
>>>>> Best wishes,
>>
>>>>> Canvas
>>
>>>>> --
>>
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Thinking Sphinx" group.
>>>>> To post to this group, send email to [email protected]
>>>>> .
>>>>> To unsubscribe from this group, send email to
>>>>> [email protected]
>>>>> .
>>>>> For more options, visit this group
>>>>> athttp://groups.google.com/group/thinking-sphinx?hl=en
>>>>> .- Hide quoted text -
>>
>>>> - Show quoted text -
>>
>>> --
>>
>>> You received this message because you are subscribed to the Google
>>> Groups "Thinking Sphinx" group.
>>> To post to this group, send email to [email protected]
>>> .
>>> To unsubscribe from this group, send email to
>>> [email protected]
>>> .
>>> For more options, visit this group
>>> athttp://groups.google.com/group/thinking-sphinx?hl=en
>>> .- Hide quoted text -
>>
>> - Show quoted text -
>
> --
>
> You received this message because you are subscribed to the Google
> Groups "Thinking Sphinx" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected]
> .
> For more options, visit this group at
> http://groups.google.com/group/thinking-sphinx?hl=en
> .
>
>
--
You received this message because you are subscribed to the Google Groups
"Thinking Sphinx" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/thinking-sphinx?hl=en.
