What versions of Rails and Thinking Sphinx are you using? Also, you can also use search_for_ids instead of search(:ids_only => true).
In staging mode, try invoking User and Contact before the search call in your rake task (you don't need to do anything with them, just a line like 'User; Contact' will do the job). I think it's a class loading issue. -- Pat On 28/08/2009, at 8:33 PM, gobigdave wrote: > > I'm pretty sure there is a bug here. I changed my configs around so I > can run on my dev box with the staging environment. This allows me to > reproduce the problem on my local machine. Given that I only need ids, > I added :ids_only => true. This prevents any need for accessing the > database and simply gives me back an array of ids. > > matching_people_ids = Person.search(people_search_options(:ids_only => > true, :max_matches => MAX_MATCHES, :per_page => MAX_MATCHES)) > > Again, this works from the console, but returns 0 people with rake > (slightly different data set on my dev box.) Change Person.search to > Contact.search and I get back 115 ids. The more I look, the more it > seems like there is bug in environments other than development. The > search engine does not seem to be querying all the child types. > > I'm happy to help track this down, but I'd love some hints on where to > start. Is it the Riddle plugin? > > On Aug 28, 9:46 am, gobigdave <[email protected]> wrote: >> I went through this a couple more times. In your first reply, you >> said >> "the searches aren't actually happening." What's strange is that from >> the console, I get 118 records, but when run in rake I get 1 record. >> Doesn't that mean that some part of the search is executing? It's >> always seems to be the User half and not the Contact half. That's why >> I was wondering if there was something not right with the loading by >> type. >> >> Right now, I'm going through the TS code, but I haven't found >> anything >> yet. >> >> On Aug 28, 8:31 am, gobigdave <[email protected]> wrote: >> >> >> >>> As an experiment, I changed: >> >>> matching_people_ids = Person.search(people_search_options(:select => >>> "people.id", :max_matches => MAX_MATCHES, :per_page => >>> MAX_MATCHES)).map { |p| p.id } >> >>> to: >> >>> matching_people = Person.search(people_search_options(:select => >>> "people.id", :max_matches => MAX_MATCHES, :per_page => MAX_MATCHES)) >>> tmp = matching_people.first >>> matching_people_ids = matching_people.map { |p| p.id } >> >>> I still get the same issue. >> >>> Also, is this behavior different for each environment? I notice that >>> everything works in development, but it doesn't work on my staging >>> environment. Since it doesn't work, I haven't pushed it to >>> production >>> yet, so I can't comment about that. >> >>> On Aug 28, 7:25 am, gobigdave <[email protected]> wrote: >> >>>> I'm confused. I left out some code to keep things simple in my >>>> example. Build_matches! actually tracks results and determines new >>>> matches and removes old matches. >> >>>> def build_matches! >>>> self.search! # changes the state >> >>>> existing_people_ids = self.matches.find(:all, :select => >>>> "matches.person_id").map { |r| r.person_id } >>>> matching_people_ids = >>>> Person.search(people_search_options(:select => >>>> "people.id", :max_matches => MAX_MATCHES, :per_page => >>>> MAX_MATCHES)).map { |p| p.id } >> >>>> new_matches = matching_people_ids - existing_people_ids >>>> no_longer_matching = existing_people_ids - matching_people_ids >> >>>> # process new, no longer matching, etc. >> >>>> return self.complete! # returns true >>>> end >> >>>> Wouldn't the call to map be the same as first since it requires >>>> going >>>> through the entire result set? I may not be rendering the result >>>> set, >>>> but I am definitely looping through the entire thing. Plus, since >>>> build_matches! returns true, the console never actually 'renders' >>>> the >>>> search results. >> >>>> -Dave >> >>>> On Aug 28, 5:22 am, Pat Allan <[email protected]> wrote: >> >>>>> Issue 1 is because Sphinx stores the class names - built off the >>>>> type >>>>> column if there is one - as integers, and then interprets that >>>>> back to >>>>> class names when searching. Hence the split between User and >>>>> Contact. >> >>>>> It's not related to issue 2, I'm pretty certain. This one is >>>>> because >>>>> Sphinx lazily loads search results - it works in console, >>>>> because to >>>>> 'render' each result, it needs to know the contents of the >>>>> array, and >>>>> so makes the query to Sphinx to find out. In larger amounts of >>>>> code, >>>>> and without the need to 'render' results to screen, the searches >>>>> aren't actually happening. >> >>>>> To get around this, you could do the following >> >>>>> def self.build_all_matches >>>>> Request.all.each do |request| >>>>> request.build_matches!.first >>>>> end >>>>> end >> >>>>> The call to first *needs* to know about the search results, and so >>>>> each will get populated from Sphinx. >> >>>>> Hope this makes sense - it's not the easiest thing to describe. >> >>>>> -- >>>>> Pat >> >>>>> On 28/08/2009, at 3:59 AM, gobigdave wrote: >> >>>>>> My application allows users to save searches. These saved >>>>>> searches are >>>>>> planned to be executed each night with a rake task, but I'm >>>>>> getting >>>>>> inconsistent results. It's very strange. >> >>>>>> There are three classes of people: >> >>>>>> class Person < ActiveRecord::Base >>>>>> define_index do >>>>>> # My index definition is here >>>>>> end >>>>>> end >> >>>>>> class User < Person; end >>>>>> class Contact < Person; end >> >>>>>> My saved search class is Request. >> >>>>>> class Request < ActiveRecord::Base >>>>>> def self.build_all_matches >>>>>> Request.all.each do |request| >>>>>> request.build_matches! >>>>>> end >>>>>> end >>>>>> def build_matches! >>>>>> # All the searching takes place here >>>>>> Person.search(people_search_options) >>>>>> end >>>>>> end >> >>>>>> Finally, there is a rake task: >> >>>>>> task :build_all_matches => :environment do >>>>>> Request.build_all_matches >>>>>> end >> >>>>>> Debugging this, I found several confusing issues. >> >>>>>> 1. From the console or from my application, executing >>>>>> Request.build_all_matches or Request.first.build_matches! >>>>>> produces the >>>>>> expected results. However, I noticed something strange in the >>>>>> logs. >>>>>> Even though the build_matches! method calls Person.search, the >>>>>> logs >>>>>> have: >> >>>>>> SELECT people.id FROM `people` WHERE (`people`.`id` IN >>>>>> (963,926,1008,984,931,929,36,1009,1006,993,947,33,930,62,187,98,407,1010,49 >>>>>> >>>>>> , >>>>>> 202,1005,156,954,248,177,55,256,179,266,183,271,274,288,292,196,31,126,301 >>>>>> >>>>>> , >>>>>> 133,318,320,331,16,338,341,345,240,165,352,355,356,364,371,374,375,378,401 >>>>>> >>>>>> , >>>>>> 258,408,409,414,548,549,560,575,580,584,591,602,611,616,293,631,297,657,19 >>>>>> >>>>>> >>>>>> 9,662,664,694,695,697,698,723,730,741,751,209,773,790,796,798,801,826,834,8 >>>>>> >>>>>> >>>>>> 46,858,348,879,882,905,912,918,269,871,347,254,251,769,336,306,224,658,637 >>>>>> >>>>>> , 619,299,201,192)) >>>>>> AND ( (`people`.`type` = 'Contact' ) ) >>>>>> SELECT people.id FROM `people` WHERE (`people`.`id` IN (1336)) >>>>>> AND >>>>>> (`people`.`activated` = 1) AND ( (`people`.`type` = 'User') ) >> >>>>>> Why is TS loading User and Contact people separately? I called >>>>>> Person.search. Shouldn't it only do a >>>>>> Person.find(:all, :conditions => >>>>>> "people.id in (...)") and not bother trying to separate records >>>>>> by >>>>>> type? >> >>>>>> I bring this up because I think this is where the problem is. See >>>>>> number 2. >> >>>>>> 2. Now, when I run the build_all_matches rake task and look >>>>>> through >>>>>> the logs for the same instance of Request executing, I only see >>>>>> this >>>>>> in the logs: >> >>>>>> SELECT people.id FROM `people` WHERE (`people`.`id` IN (1336)) >>>>>> AND >>>>>> (`people`.`activated` = 1) AND ( (`people`.`type` = 'User') ) >> >>>>>> The select with (`people`.`type` = 'Contact' ) is not executed, >>>>>> and >>>>>> the results of build_matches! is incorrect. There is only 1 >>>>>> person >>>>>> returned from Person.search instead of the 118 I expected. I >>>>>> verified >>>>>> with several log statements that the same inputs are happening. >>>>>> It's >>>>>> always the load of people.type = 'User' that is skipped on all >>>>>> Request >>>>>> objects, but ONLY when I run it from the rake task. Same thing >>>>>> happens if I make rake task that only calls build_matches! on a >>>>>> single >>>>>> Request object. >> >>>>>> Rake, console, searchd, and web app are all running as the same >>>>>> user. >>>>>> I am using version 1.2.7 of the gem on Rails 2.3.3, Ruby 1.8.6, >>>>>> and >>>>>> rake 0.8.7. >> >>>>>> Help! I've been staring at this for hours, and it doesn't make >>>>>> any >>>>>> sense. I'm guessing it's something about the way rake loads >>>>>> Rails that >>>>>> is causing TS to get confused, but I haven't found where or how >>>>>> yet. > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
