and no, it didn't make much speed difference on the server. my feeling at the moment is that the database design is causing serious cpu crunching. I have noticed it other than when indexing though.
On Mar 12, 8:39 am, Nick Sellen <[email protected]> wrote: > it's generated normally - the ids range from 1 to 13966 with ~4000 > items - although for the user index which the stats above are from > it's just 1 to 1538 with 1538 items. > > the other two factors which I'm looking at are: > 1. whilst the db box is fast and powerful, dual quad core processors / > 15k SAS disks. the query is bottlenecking on being constrained to one > CPU core - the individual cores aren't THAT powerful and so could > quite conceivably be outdone by my laptop > 2. database design, theres quite a lot of rails polymorphic columns, > which had the default 255 length and utf-8 collation = quite a lot of > bytes. I've changed this to varchar(14) with latin1_general_ci > collation. I didn't have a massive boost on my laptop - a few seconds > quicker, just running it on the server now... > > cheers, > > On Mar 12, 4:54 am, Pat Allan <[email protected]> wrote: > > > > > Hi Nick > > > How was your production data generated - normally, or by fixtures? (Or > > even: what's the min and max ids in your user table?) If there's a really > > massive window, then Sphinx is making a lot of queries to get through all > > the data, but you can change > > that:http://freelancing-god.github.com/ts/en/common_issues.html#slow_indexing > > > If that doesn't help matters, let us know. > > > Cheers > > > -- > > Pat > > > On 12/03/2010, at 9:09 AM, Nick Sellen wrote: > > > > I've got a case of very slow indexing - 111 seconds to index 9 > > > documents. It's not really thinking sphinx that is responsible for > > > this part anyway - but just looking for any tips or ideas. > > > > - on my laptop it runs fine, e.g.: > > > > indexing index 'user_delta'... > > > total 2 docs, 68 bytes > > > total 0.048 sec, 1425.43 bytes/sec, 41.92 docs/sec > > > > - on the production server one time it ran like this: > > > > indexing index 'user_delta'.. > > > total 9 docs, 320 bytes > > > total 111.667 sec, 2.87 bytes/sec, 0.08 docs/sec > > > > - and another like this: > > > > indexing index 'user_delta'... > > > total 14 docs, 533 bytes > > > total 8.589 sec, 62.06 bytes/sec, 1.63 docs/sec > > > > the indexer is running on seperate box to the db - the db server is a > > > very fast database server (SAS disks etc) so the hardware should be > > > fine, but the load on it is surprisingly high when running the > > > indexing. > > > > I ran a MySQL EXPLAIN command on the generated SQL configuration and > > > it shows all indexes being used. > > > > so, yeah, any experiences people might have that could help would be > > > much appreciated. > > > > cheers! > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Thinking Sphinx" group. > > > To post to this group, send email to [email protected]. > > > To unsubscribe from this group, send email to > > > [email protected]. > > > For more options, visit this group > > > athttp://groups.google.com/group/thinking-sphinx?hl=en. -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
