it's generated normally - the ids range from 1 to 13966 with ~4000 items - although for the user index which the stats above are from it's just 1 to 1538 with 1538 items.
the other two factors which I'm looking at are: 1. whilst the db box is fast and powerful, dual quad core processors / 15k SAS disks. the query is bottlenecking on being constrained to one CPU core - the individual cores aren't THAT powerful and so could quite conceivably be outdone by my laptop 2. database design, theres quite a lot of rails polymorphic columns, which had the default 255 length and utf-8 collation = quite a lot of bytes. I've changed this to varchar(14) with latin1_general_ci collation. I didn't have a massive boost on my laptop - a few seconds quicker, just running it on the server now... cheers, On Mar 12, 4:54 am, Pat Allan <[email protected]> wrote: > Hi Nick > > How was your production data generated - normally, or by fixtures? (Or even: > what's the min and max ids in your user table?) If there's a really massive > window, then Sphinx is making a lot of queries to get through all the data, > but you can change > that:http://freelancing-god.github.com/ts/en/common_issues.html#slow_indexing > > If that doesn't help matters, let us know. > > Cheers > > -- > Pat > > On 12/03/2010, at 9:09 AM, Nick Sellen wrote: > > > > > I've got a case of very slow indexing - 111 seconds to index 9 > > documents. It's not really thinking sphinx that is responsible for > > this part anyway - but just looking for any tips or ideas. > > > - on my laptop it runs fine, e.g.: > > > indexing index 'user_delta'... > > total 2 docs, 68 bytes > > total 0.048 sec, 1425.43 bytes/sec, 41.92 docs/sec > > > - on the production server one time it ran like this: > > > indexing index 'user_delta'.. > > total 9 docs, 320 bytes > > total 111.667 sec, 2.87 bytes/sec, 0.08 docs/sec > > > - and another like this: > > > indexing index 'user_delta'... > > total 14 docs, 533 bytes > > total 8.589 sec, 62.06 bytes/sec, 1.63 docs/sec > > > the indexer is running on seperate box to the db - the db server is a > > very fast database server (SAS disks etc) so the hardware should be > > fine, but the load on it is surprisingly high when running the > > indexing. > > > I ran a MySQL EXPLAIN command on the generated SQL configuration and > > it shows all indexes being used. > > > so, yeah, any experiences people might have that could help would be > > much appreciated. > > > cheers! > > > -- > > You received this message because you are subscribed to the Google Groups > > "Thinking Sphinx" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/thinking-sphinx?hl=en. -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
