Turns out the problem was with STI. I was expecting all rows to be indexed and searchable via the root classes #search method. However only rows with a blank or null type are searchable via the root classes search method.
I just assumed that since the primary search count was significantly lower then the full count that not all of the rows were being indexed. On May 20, 2:24 pm, Jacob <[email protected]> wrote: > Well everything looks to be exactly where it was when Sphinx stopped > indexing the data, but everything works now. Not sure what was going > on but it works now :) > > I appreciate everyones help! > Best > Jake > > On May 19, 6:18 pm, Pat Allan <[email protected]> wrote: > > > Hmm, wouldn't think a mysql index would be the cause - but what you're > > facing is not a common problem, so anything could be the cause :) > > > -- > > Pat > > > On 19/05/2009, at 3:19 PM, Jacob wrote: > > > > Hi Pat, > > > > Thanks for taking a gander. I took everything out of the object except > > > for a single index which still erred, however once I removed all of > > > the other fields from the table everything worked. Now I'm adding the > > > fields back one by one to see if I can pin point which one caused any > > > trouble. Would it be possible a mysql index could have caused a > > > conflict? > > > > Thanks > > > > On May 19, 5:06 pm, Pat Allan <[email protected]> wrote: > > >> Hi Jake > > > >> Just been looking over your original config - there's a lot there! > > > >> I'd try add some fields and attributes back in, bit by bit. See if > > >> there's some point where it doesn't work anymore. I know it's a bit > > >> fiddly, but I had to do this a week or so ago on someone else's data. > > >> Get two terminals going, one in script/console for testing, the other > > >> with all the rake tasks lined up: > > >> rake ts:stop ts:in ts:start > > > >> That way, you make the change to the index, save the file, run the > > >> rake task line, then run the test query in script/console. > > > >> The symbols or method calls should both work fine, provided it's not > > >> an existing method on Object (and there's nothing in the config that > > >> stood out to me). > > > >> Cheers > > > >> -- > > >> Pat > > > >> On 19/05/2009, at 12:02 PM, Jacob wrote: > > > >>> Well I just ripped everything out of my model and table leaving me > > >>> with nothing but a single column. > > > >>> class Property < ActiveRecord::Base > > >>> define_index do > > >>> indexes :listing_number > > >>> end > > >>> end > > > >>> And suddenly the indexes start working again. Are their specific > > >>> column names that I should be avoiding, or would the type/default/ > > >>> nullable combo be the culprit? > > > >>> Best > > >>> Jake > > > >>> On May 19, 12:34 pm, Jacob <[email protected]> wrote: > > >>>> I was a bit wary of the tutorial using variables rather then > > >>>> symbols, > > >>>> but I just went with it. > > >>>> I did go back and double check the single index on my users table > > >>>> and > > >>>> it turns out it is indexing and I can search. > > > >>>> As for the table that is not indexing, I removed everything from > > >>>> define_index except a single index, using a symbol for the field. I > > >>>> tried stopping sphinx, deleting db/sphinx, indexing and starting, > > >>>> again with no luck. Here is the output and a pastie of the > > >>>> schema. I > > >>>> cannot share the model, but is there something I should be looking > > >>>> for? I'll take gander to see if my model is redefining any sphinx > > >>>> methods. > > > >>>> Table:http://pastie.org/private/v8yfsooycac2jxxx4lz7cg > > >>>> Updated Sphinx conf:http://pastie.org/483000 > > > >>>> Thanks > > > >>>> jacobbas...@dude:~/src/smart$ rake ts:stop > > >>>> (in /home/jacobbasham/src/smart) > > >>>> Sphinx 0.9.8.1-release (r1533) > > >>>> Copyright (c) 2001-2008, Andrew Aksyonoff > > > >>>> using config file '/home/jacobbasham/src/smart/config/ > > >>>> development.sphinx.conf'... > > >>>> stop: succesfully sent SIGTERM to pid 32212 > > >>>> Stopped search daemon (pid 32212). > > > >>>> jacobbas...@dude:~/src/smart$ rm -rf db/sphinx/ > > > >>>> jacobbas...@dude:~/src/smart$ rake ts:index > > >>>> (in /home/jacobbasham/src/smart) > > >>>> Generating Configuration to /home/jacobbasham/src/smart/config/ > > >>>> development.sphinx.conf > > >>>> indexer --config /home/jacobbasham/src/smart/config/ > > >>>> development.sphinx.conf --all > > >>>> Sphinx 0.9.8.1-release (r1533) > > >>>> Copyright (c) 2001-2008, Andrew Aksyonoff > > > >>>> using config file '/home/jacobbasham/src/smart/config/ > > >>>> development.sphinx.conf'... > > >>>> indexing index 'property_core'... > > >>>> collected 700 docs, 0.0 MB > > >>>> collected 0 attr values > > >>>> sorted 0.0 Mvalues, 100.0% done > > >>>> sorted 0.0 Mhits, 100.0% done > > >>>> total 700 docs, 5600 bytes > > >>>> total 0.031 sec, 182898.94 bytes/sec, 22862.37 docs/sec > > >>>> distributed index 'property' can not be directly indexed; skipping. > > >>>> indexing index 'user_core'... > > >>>> collected 45204 docs, 0.0 MB > > >>>> collected 0 attr values > > >>>> sorted 0.0 Mvalues, 100.0% done > > >>>> sorted 0.0 Mhits, 100.0% done > > >>>> total 45204 docs, 423 bytes > > >>>> total 0.152 sec, 2782.16 bytes/sec, 297316.49 docs/sec > > >>>> distributed index 'user' can not be directly indexed; skipping. > > > >>>> jacobbas...@dude:~/src/smart$ rake ts:start > > >>>> (in /home/jacobbasham/src/smart) > > >>>> searchd --pidfile --config /home/jacobbasham/src/smart/config/ > > >>>> development.sphinx.conf > > >>>> Sphinx 0.9.8.1-release (r1533) > > >>>> Copyright (c) 2001-2008, Andrew Aksyonoff > > > >>>> using config file '/home/jacobbasham/src/smart/config/ > > >>>> development.sphinx.conf'... > > >>>> creating server socket on 127.0.0.1:3312 > > >>>> Started successfully (pid 705). > > > >>>> jacobbas...@dude:~/src/smart$ script/console > > >>>> Loading development environment (Rails 2.1.0)>> > > >>>> User.search("[email protected]").first.email > > > >>>> => "[email protected]" > > > >>>>>> Property.first(:conditions => "listing_number is not > > >>>>>> null").listing_number > > >>>> => "06430044" > > >>>>>> Property.search("06430044") > > > >>>> => [] > > > >>>> On May 19, 10:44 am, aitrus <[email protected]> wrote: > > > >>>>> My first inclination would be to try referencing the column > > >>>>> names as > > >>>>> symbols. IE, > > > >>>>> indexes :listing_number > > > >>>>> rather than, > > > >>>>> indexes listing_number > > > >>>>> If that still doesn't do the trick, I'd remove the "bitmask" > > >>>>> portion > > >>>>> and do a rake ts:stop/index/start, and post the output plus your > > >>>>> table > > >>>>> structure and the model and I'll peek at it some more. > > > >>>>> Scott > > > >>>>> On May 19, 7:22 am, Jacob <[email protected]> wrote: > > > >>>>>> Scott, > > > >>>>>> My original define_index has grown to have some fancy other > > >>>>>> options I > > >>>>>> had to add in (so I can easily index a bitmask) so it will look a > > >>>>>> bit > > >>>>>> funny. Although I have tried ripping everything out but a single > > >>>>>> indexed field, as well as indexing a single field on a different > > >>>>>> model. > > > >>>>>> fancy index:http://pastie.org/482782 > > >>>>>> generated config:http://pastie.org/482784 > > > >>>>>> The second 'test' index is a basic > > > >>>>>> define_index do > > >>>>>> indexes email > > >>>>>> end > > > >>>>>> Jake > > > >>>>>> On May 19, 12:48 am, aitrus <[email protected]> wrote: > > > >>>>>>> Can you pastie your define_index statements, and your table > > >>>>>>> schema? > > >>>>>>> Can ;you also pastie your sphinx.conf that's generated? > > > >>>>>>> Be sure to check the name of your columns. Some may conflict > > >>>>>>> with > > >>>>>>> reserved words in Ruby / Sphinx / whatever. > > > >>>>>>> Also, test your define_index first, with one or two columns > > >>>>>>> and no > > >>>>>>> other fancy options. > > > >>>>>>> Scott > > > >>>>>>> On May 18, 10:21 am, Jacob <[email protected]> wrote: > > > >>>>>>>> This is probably more a question for the Sphinx forum, but I'll > > >>>>>>>> ask > > >>>>>>>> here first. I have defined indexes for two of my models and was > > >>>>>>>> able > > >>>>>>>> to index a couple weeks back. Now when I try and index it looks > > >>>>>>>> like > > >>>>>>>> Sphinx is finding the documents, but isn't indexing anything. > > >>>>>>>> The > > >>>>>>>> physical size of the collected documents also looks to be very > > >>>>>>>> off. > > >>>>>>>> I've tried Googling for days and haven't been able to find any > > >>>>>>>> leads. > > >>>>>>>> Does this mean the indexing query is not returning any rows? > > > >>>>>>>> Thanks > > >>>>>>>> Jake > > > >>>>>>>> indexing index 'property_core'... > > >>>>>>>> collected 700 docs, 0.2 MB > > >>>>>>>> collected 0 attr values > > >>>>>>>> sorted 0.0 Mvalues, 100.0% done > > >>>>>>>> sorted 0.0 Mhits, 100.0% done > > >>>>>>>> total 700 docs, 212855 bytes > > >>>>>>>> total 0.144 sec, 1482617.91 bytes/sec, 4875.77 docs/sec > > >>>>>>>> distributed index 'property' can not be directly indexed; > > >>>>>>>> skipping. > > >>>>>>>> indexing index 'user_core'... > > >>>>>>>> collected 45204 docs, 0.0 MB > > >>>>>>>> collected 0 attr values > > >>>>>>>> sorted 0.0 Mvalues, 100.0% done > > >>>>>>>> sorted 0.0 Mhits, 100.0% done > > >>>>>>>> total 45204 docs, 423 bytes > > >>>>>>>> total 0.151 sec, 2795.51 bytes/sec, 298743.00 docs/sec > > >>>>>>>> distributed index 'user' can not be directly indexed; skipping. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
