I appreciate you taking the time to write this up. Evil "has_many facets" is the bread & butter of what I need to accomplish for this app. Too bad it's not set up to do that. I'd love to hear from Pat if he has any thoughts on this. Even though faceting is a goal for version 1.2, some folks might just jump in and get started on initial versions of that. Seems to me like there's a need for a real alternative to solr that is fast, robust, and easy. My opinion is that Ferret has index corruption horror stories, solr is performance bottleneck when it comes to indexing, and Ultrasphinx has a cumbersome syntax for index definition. I really like TS's clean design, ease of use and performance, and I appreciate the effort you folks are putting into it.
Till next time, -H -----Original Message----- From: Dan Pickett <[email protected]> Reply-to: [email protected] To: [email protected] Subject: [ts] Re: from acts_as_solr to thinking_sphinx, first thoughts Date: Tue, 17 Feb 2009 15:34:24 -0500 Ya sure, so basically as I understand it (and Pat can correct all that is wrong with this :-) )- the need for the translate method is due to crc'd attributes. A few different datatypes get casted in the query so they can't be easily extracted. So what I propose is we establish a convention around "translated" columns for attributes that are crc'ed. If they're in the result set that comes back from the query we can match the crc'd attr to the actual field value. This would eliminate the need for the send call and would open up the opportunity for expressions to be faceted. This also makes it easier to aggregate facets across models. The example you're talking about I think it's a little more complicated. Right now there's a single entry per instance of the model with the define_index in it. I'm not immediately sure how we would get the proper facet results and retain the 1-1 correspondence we have going. Short answer is, I'm not sure how to make has_many type facets work. I would have to dig a bit deeper. Maybe Pat has something in mind for this. On Tue, Feb 17, 2009 at 3:02 PM, Harold A. Gimenez <[email protected]> wrote: Does anyone have any ideas on how to create these facets? I'm having trouble when dealing with a has_many associations and faceting. For example, an item has_many categories. Indexing works fine, but when I try to get facet counts, I find the error described below by both Mickael and myself (the call to translate on facet.rb). Creating facets on has_one associations works as expected. Dan, could you explain how the translation field your proposing for CRC'ed attributes would work, and if it would solve this issue? Thanks again, -H -----Original Message----- From: Mickaël <[email protected]> Reply-to: [email protected] To: Thinking Sphinx <[email protected]> Subject: [ts] Re: from acts_as_solr to thinking_sphinx, first thoughts Date: Tue, 17 Feb 2009 06:56:28 -0800 (PST) Hi all, I've read this topic, and, I've the same problem. Below, Model relations UserProfile <---> Trainings -------------------------------------------------------------------------------- UserProfile Model define_index do indexes trainings.diploma_level_id, :sortable => true, :facet => true # Attribute indexes :last_name, :sortable => true, :facet => true indexes :first_name, :sortable => true, :facet => true # Properties set_property :enable_star => 1 set_property :min_prefix_len => 1 end has_many :trainings -------------------------------------------------------------------------------- Training Model belongs_to :diploma_level belongs_to :user_profile [...] -------------------------------------------------------------------------------- In my Test Controller, i do that def test @facets = UserProfile.facets("Marc") end -------------------------------------------------------------------------------- With my favorite browser, i get http://localhost:3000/index/test and i'm showing this NoMethodError in IndexController#test undefined method `diploma_level_id' for []:Array For more information, Application Trace http://pastebin.com/f23a48369 On 17 fév, 09:44, Pat Allan <[email protected]> wrote: > Hi Harold > > The timestamp issue crops up on MySQL as well - I'm not sure if Sphinx > finds it a limitation either - I get the feeling it might only like > unsigned integers. Either way, we need to figure out a workaround > from a database perspective. I guess a function could be written for > PG to do a smart timestamp? > > -- > Pat > > On 17/02/2009, at 1:58 AM, Harold A. Gimenez wrote: > > > Thanks to both for looking into this. > > > Dan, you are right. The issue is on the send call in translate. This > > is a one-to-many relationship, and so by the time it reaches > > object.send (... ...), it tries to call the method on an array of AR > > objects, not on the AR object itself. This causes the no method > > found issue. I've pasted the stack trace here:http://gist.github.com/65198 > > > Regarding the date-of-birth issue, I am still not sure what's > > causing it. However, I do see that it is using cast(extract(epoch > > from people.dob) as int) on dob when it is configured as an > > attribute. Most of these DOBs are pre unix timestamp = 0 > > (1970-01-01), and so when this is run directly on the database, I > > get integer out of range error. That said, the number of docs > > created is much less than the number of people with a DOB < > > 1970-01-01, so there's something else going on as well. I am using > > postgreSQL 8.2 for these trials... > > > Can sphinx create attributes with dates older than 1970-01-01? I > > understand that it converts them to integers (unix timestamps), but > > is there a way around it? Can it take negative integers representing > > dates pre 1970? > > > Thanks again, > > -H > > > -----Original Message----- > > From: Dan Pickett <[email protected]> > > Reply-to: [email protected] > > To: [email protected] > > Subject: [ts] Re: from acts_as_solr to thinking_sphinx, first thoughts > > Date: Mon, 16 Feb 2009 06:48:38 -0500 > > > awesome, Pat. > > > I think this is probably due to the send call in translate of > >facet.rb - I proposed and am still considering creating a > > translation field for crc'd attributes like dates and strings in the > > generated configuration. The indexed fields wouldn't be a problem > > because they're already in the query. > > > What do you think? > > > On Mon, Feb 16, 2009 at 3:05 AM, Pat Allan <p...@freelancing- > > gods.com> wrote: > > > Hi Harold > > > Can you post the stack-trace of thatfaceterror? > > > The date-of-bith issue is odd - it's nice to know it works as a field, > > but that means you won't be able to filter by it. I've seen this issue > > occasionally, but not reliably, and would be awesome to track it down. > > What happens if you switch it back to being an attribute, run ts:in, > > and then check the sql_query value for that index in config/ > > development.sphinx.conf - remove the parts of the WHERE clause that > > handle id paging, and see how many records it returns... > > > Dan: I've started to look through your fork, and I've been refactoring > > both your and my code. Haven't got it ready for release just yet > > though. > > > Cheers > > > -- > > Pat > > > On 16/02/2009, at 3:47 AM, Harold A. Giménez Ch. wrote: > > > > Hi Dan. Thanks for your response. > > > > I did see thefacetsupport. However, it is only working for me when > > > thefacetis on the model being indexed, not on one of its > > > associations. If I try adding :facet=> true to one of the > > > attributes that are associations, it indexes without issues, but > > > when calling Model.facets('something') I get an exception along the > > > lines of: > > > > NoMethodError: undefined method `facet_name' for #<Class: > > > 0x7f6a4d321a68> > > > > Regarding the question below about person.dob and why it would not > > > index all of my records, turns out that I tried indexes person.dob > > > (instead of has person.dob) and it did index the full set. > > > > Thanks again for the help, > > > > -Harold > > > > On Sun, Feb 15, 2009 at 11:39 AM, Dan Pickett > > > <[email protected]> wrote: > > > Hey Harold, > > > > The current version of Thinking Sphinx hasfacetsupport (add :facet > > > => true to your attribute ). There's some additional functionality > > > in my fork for class based facets, but it definitely could use some > > > more love from contributors - > > > >http://github.com/dpickett/thinking-sphinx/tree/master > > > > I'm hoping to continue to work on it, but I haven't had a lot of > > > free cycles. > > > > Regards, > > > Dan > > > > On Sat, Feb 14, 2009 at 10:44 PM, Harold A. Gimenez <[email protected] > > > > wrote: > > > Hi, > > > > I started looking at sphinx today, with the intent to port an > > > application's search engine from solr to sphinx. The solr > > > implementation is solid, but doesn't meet my business needs - we > > > have to bulk load the database and reindex new data more often than > > > indexing solr takes! It is a huge bottleneck in our ops. I have a > > > few comments which I'd like to share, as well as a couple of > > > questions I hope I can get some guidance with. > > > > Sphinx (and TS) installation was a breeze. First step for me was to > > > try out the indexing speed. I added a few fields and attributes to a > > > Patient model and kicked off indexing. I was discouraged by how long > > > it took. Turns out that the (fake) data I was indexing came from > > > fixtures. Switching to a production-like database, with sequential > > > primary keys, indexed 500K+ records in less than a minute. Wow! > > > > Now I have a problem. I've added a DOB attribute (has > > > person.dob, :as => :date_of_birth). Now, when I run the index, it is > > > only picking up 45885 docs (I'm expecting over 500K+). The dob > > > field is of type timestamp. As a trial, I switched person.dob to > > > person.created_at, and all of the 500K+ documents do get generated. > > > Any thoughts on why this is happening? > > > > The next challenge isfacetsupport, specifically on associations. > > > Is there a fork somewhere where this has been implemented? Any > > > pointers would be appreciated. > > > > Thanks, > > > -H > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
