Hi Pat, I am confused too. It doesn't make much sense to me the way it is now. I will construct a sample app in the weekend for you to reproduce the issue.
Best wishes, Canvas On Jan 5, 12:47 am, Pat Allan <[email protected]> wrote: > This is confusing - if the setting's in the configuration file, then it's up > to Sphinx to use it - it's not managed by Thinking Sphinx at all. I'm not > quite sure what the next step here is... > > If you want to construct a demo rails app which can reproduce the issue, > that'd be great, then I can have a look on my machine. > > -- > Pat > > On 05/01/2010, at 5:29 AM, Canvas wrote: > > > Hi Pat, > > > I just came back from a long vocation. Sorry for the long delay. > > > sqlquery_killlist does appear in development.sphinx.conf after I run > > "rake thinking_sphinx:configure". So the problem I have now is that > > the configuration is there but has no effect. > > > Best wishes, > > > Canvas > > > On Dec 25 2009, 8:00 pm, Pat Allan <[email protected]> wrote: > >> Hi Canvas > > >> Sorry for the delay in getting back to you... > > >> When you're using 1.3.12 (and I recommend switching to 1.3.14 anyway), > >> does the kill list appear in the config file? Is it that the setting is > >> there and not having any effect, or that it's not getting into the conf > >> file? > > >> As for the speed changes, that's interesting to know... there's been a > >> *lot* of changes since Ed's fork appeared, so I guess that means there's > >> plenty of potential reasons why that's now the case. I'll try to > >> investigate when I have some time. > > >> -- > >> Pat > > >> On 18/12/2009, at 7:56 AM, Canvas wrote: > > >>> Hi Pat, > > >>> I just tried rails 2.3.4 + thinking-sphinx 0.9.9 (Ed's fork) + sphinx > >>> 0.9.9 final release. Everything works fine. Indexing is fast, > >>> searching is fast, and sql_query_killlist works.. It's really weird > >>> that thinking-sphinx > >>> 1.3.12 is slow in searching and sql_query_killlist does not work in my > >>> case. > > >>> Best wishes, > > >>> Canvas > > >>> On Dec 17, 11:48 am, Canvas <[email protected]> wrote: > >>>> Hi Pat, > > >>>> I still can not figure out why sql_query_killlist does not work in > >>>> thinking-sphinx 1.3.12. Any advice is appreciated. I am now using > >>>> rails 2.3.4, sphinx 0.9.9 final release, thinking-sphinx 1.3.12. > > >>>> It's quite interesting that sql_quwery_killlist works quite well with > >>>> rails 2.0.2, sphinx 0.9.9 final release and thinking-sphinx 0.9.9 > >>>> (Ed's fork). > > >>>> I am stuck here now. Any advice will be appreciated. > > >>>> Best wishes, > > >>>> Canvas > > >>>> On Dec 16, 5:29 pm, Canvas <[email protected]> wrote: > > >>>>> Hi Pat, > > >>>>> I just found out that sql_query_killlist works perfectly with sphinx > >>>>> 0.9.9 + thinking-sphinx 0.9.9, but it does not work at all with sphinx > >>>>> 0.9.9 + thinking-sphinx 1.3.12. This proves that sql_query_killlist > >>>>> works as far as sphinx 0.9.9 is concerned, which means something might > >>>>> be wrong with thinking sphinx 1.3.12. I couldn't figure it out yet. > >>>>> Any help will be appreciated. > > >>>>> I also noticed that when doing full index, thinking-sphinx 1.3.12 is > >>>>> much faster, but searching is much slower , than thinking-sphinx > >>>>> 0.9.9. Is it possible to improve the searching performance for > >>>>> thinking-sphinx 1.3.12? > > >>>>> Thank you very much. > > >>>>> Best wishes, > > >>>>> Canvas > > >>>>> On Dec 16, 2:31 pm, Canvas <[email protected]> wrote: > > >>>>>> Hi Pat, > > >>>>>> I customized thinking-sphinx. Threshold is not in use at all in my > >>>>>> case. > > >>>>>> I created a table sphinx_delta_index_start_points. The table holds one > >>>>>> and only one row. The db migrate is as following: > > >>>>>> class CreateTableSphinxDeltaIndexStartPoints < ActiveRecord::Migration > >>>>>> def self.up > >>>>>> create_table :sphinx_delta_index_start_points do |t| > >>>>>> t.datetime :delta_index_start_at, :null => false > >>>>>> end > >>>>>> end > > >>>>>> def self.down > >>>>>> drop_table :sphinx_delta_index_start_points > >>>>>> end > >>>>>> end > > >>>>>> Every time a full index or merge index is executed, the start-time > >>>>>> will be updated in the only row in the table above. And the three key > >>>>>> item in configuration file for delta index will be as following. # > >>>>>> {[email protected]_table_name} is used to represent whatever table name > >>>>>> your model represents in your application. > > >>>>>> sql_query = SELECT ... FROM "#[email protected]_table_name}" WHERE # > >>>>>> {[email protected]_table_name}.id >= $start AND # > >>>>>> {[email protected]_table_name}.id <= $end AND # > >>>>>> {[email protected]_table_name}.`updated_at` >= ( SELECT MIN > >>>>>> (delta_index_start_at) FROM sphinx_delta_index_start_points ) GROUP BY > >>>>>> #[email protected]_table_name}.id ORDER BY NULL > > >>>>>> sql_query_range = SELECT IFNULL(MIN(`id`), 1), IFNULL(MAX(`id`), 1) > >>>>>> FROM #[email protected]_table_name} WHERE updated_at >= ( SELECT MIN > >>>>>> (delta_index_start_at) FROM sphinx_delta_index_start_points ). > > >>>>>> sql_query_killlist = "SELECT id FROM #[email protected]_table_name} > >>>>>> WHERE updated_at >= (SELECT MIN(delta_index_start_at) FROM > >>>>>> sphinx_delta_index_start_points)". > > >>>>>> Hope this helps. > > >>>>>> Best wishes, > > >>>>>> Canvas > > >>>>>> On Dec 9, 11:11 pm, Pat Allan <[email protected]> wrote: > > >>>>>>> Quick question: how did you write a select that returns deleted > >>>>>>> values? :) > > >>>>>>> -- > >>>>>>> Pat > > >>>>>>> On 01/12/2009, at 7:36 AM, Canvas wrote: > > >>>>>>>> Hi Pat, > > >>>>>>>> I just tried it. It works! > > >>>>>>>> Please refer to the following url for further information. > > >>>>>>>>http://www.sphinxsearch.com/docs/manual-0.9.9.html#conf-sql-query-kil... > > >>>>>>>> On Nov 27, 6:04 pm, Pat Allan <[email protected]> wrote: > >>>>>>>>> Ah, I never knew that it only changed the values in memory... > >>>>>>>>> that's > >>>>>>>>> good to know! And I really should get the kill list stuff into > >>>>>>>>> Thinking Sphinx proper. > > >>>>>>>>> Thanks for that info, muchly appreciated. > > >>>>>>>>> Cheers > > >>>>>>>>> -- > >>>>>>>>> Pat > > >>>>>>>>> On 28/11/2009, at 12:37 PM, Canvas wrote: > > >>>>>>>>>> Hi Pat, > > >>>>>>>>>> It turns out that "client.update('buy_sell_file_core', > >>>>>>>>>> ['sphinx_deleted'], { 4013 => [1] }) " updates index in RAM only. > >>>>>>>>>> And > >>>>>>>>>> the original core index stays unchanged. I am now trying sphinx > >>>>>>>>>> 0.9.9- > >>>>>>>>>> rc2, which includes a new feature sql_query_killlist to deal with > >>>>>>>>>> this > >>>>>>>>>> issue. I'll let you know when I try it out. > > >>>>>>>>>>http://www.sphinxsearch.com/forum/view.html?id=2552 > > >>>>>>>>>> Thanks a lot. > > >>>>>>>>>> Canvas > > >>>>>>>>>> On Nov 26, 6:25 pm, Pat Allan <[email protected]> wrote: > >>>>>>>>>>> Hmm, I wonder if Sphinx doesn't delete documents that appear in > >>>>>>>>>>> both > >>>>>>>>>>> indexes... > > >>>>>>>>>>> Scenarios to try: > >>>>>>>>>>> - delete a record, add a different record, merge and see if the > >>>>>>>>>>> deleted record is kept around or not > >>>>>>>>>>> - flag the core copy as deleted, merge with the empty delta, > >>>>>>>>>>> re-index > >>>>>>>>>>> delta, merge again, see if only the updates are kept. > > >>>>>>>>>>> -- > >>>>>>>>>>> Pat > > >>>>>>>>>>> On 27/11/2009, at 12:48 PM, Canvas wrote: > > >>>>>>>>>>>> Hi Pat, > > >>>>>>>>>>>> Thanks for your timely reply. client.update() does work. But I > >>>>>>>>>>>> still > >>>>>>>>>>>> have a wierd problem. Following are the steps for me to > >>>>>>>>>>>> reproduce > >>>>>>>>>>>> the > >>>>>>>>>>>> problem. > > >>>>>>>>>>>> Step 1: build full index > >>>>>>>>>>>> # rake thinking_sphinx:index RAILS_ENV=development > > >>>>>>>>>>>> Step 2: make some changes to file 4013 > > >>>>>>>>>>>> Step 3: build delta index ==> the file 4013 is searchable with > >>>>>>>>>>>> both > >>>>>>>>>>>> old and new data > >>>>>>>>>>>> # rake thinking_sphinx:index:delta RAILS_ENV=development > > >>>>>>>>>>>> Step 4: flag the file in core index as "sphinx_deleted" ==> the > >>>>>>>>>>>> file > >>>>>>>>>>>> is searchable only by new data, so far so good. > >>>>>>>>>>>>>> client.update('buy_sell_file_core', ['sphinx_deleted'], { 4013 > >>>>>>>>>>>>>> => > >>>>>>>>>>>>>> [1] }) > > >>>>>>>>>>>> Step 5: merge delta to core > >>>>>>>>>>>> # /usr/local/bin/indexer --config '/workspace/CA/BETA_2/ > >>>>>>>>>>>> EconveyancePro/ > >>>>>>>>>>>> config/development.sphinx.conf' --rotate --merge > >>>>>>>>>>>> buy_sell_file_core > >>>>>>>>>>>> buy_sell_file_delta --merge-dst-range sphinx_deleted 0 0 > > >>>>>>>>>>>> And now the problem occurs, I can search by both new and old data > >>>>>>>>>>>> again. It doesn't make any sense to me. Why can I search by the > >>>>>>>>>>>> old > >>>>>>>>>>>> data again? Isn't the old index supposed to be deleted in Step > >>>>>>>>>>>> 4? > >>>>>>>>>>>> Did > >>>>>>>>>>>> I do anything wrong in step 5? > > >>>>>>>>>>>> Thank you very much for your help. > > >>>>>>>>>>>> Best wishes, > > >>>>>>>>>>>> Canvas > > >>>>>>>>>>>> On Nov 25, 12:26 am, Pat Allan <[email protected]> wrote: > >>>>>>>>>>>>> Hi Canvas > > >>>>>>>>>>>>> Are you using Thinking Sphinx? It adds an internal attribute > >>>>>>>>>>>>> called > >>>>>>>>>>>>> sphinx_deleted, and sets records' values to 1 when they are > >>>>>>>>>>>>> deleted > >>>>>>>>>>>>> in > >>>>>>>>>>>>> Ruby code. Then, if you're using the datetime deltas, the merge > >>>>>>>>>>>>> automatically uses the --merge-dst-range option to remove > >>>>>>>>>>>>> deleted > >>>>>>>>>>>>> items from the index. > > >>>>>>>>>>>>> However, if you're using Riddle, then the equivalent call is > >>>>>>>>>>>>> client.update, not UpdateAttributes. I recommend looking at the > >>>>>>>>>>>>> source > >>>>>>>>>>>>> code to get a good understanding of how it all works. Riddle > >>>>>>>>>>>>> documentation is pretty thin on the ground, but I'd like to > >>>>>>>>>>>>> improve > >>>>>>>>>>>>> it > >>>>>>>>>>>>> over time. > > >>>>>>>>>>>>> Hope this helps. > > >>>>>>>>>>>>> -- > >>>>>>>>>>>>> Pat > > >>>>>>>>>>>>> On 25/11/2009, at 8:00 AM, Canvas wrote: > > >>>>>>>>>>>>>> Hi there guys, > > >>>>>>>>>>>>>> I am currently using sphinx 9.8.1. The following is from sphinx > >>>>>>>>>>>>>> 9.8.1 > >>>>>>>>>>>>>> document: > > >>>>>>>>>>>>>> " The basic command syntax is as follows: > > >>>>>>>>>>>>>> indexer --merge DSTINDEX SRCINDEX [--rotate] > > >>>>>>>>>>>>>> Only the DSTINDEX index will be affected: the contents of > >>>>>>>>>>>>>> SRCINDEX > >>>>>>>>>>>>>> will be merged into it. --rotate > > ... > > read more »
-- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
