Hi, Thanks for the reply. Not sure what you mean about the columns . They are columns containing ids for other tables, that I am using to limit my actual search queries.
Changing the sql_range_step has not seemed to make any noticeable difference in the amount of time it takes. I have tried at the default value, in 100,000 blocks, and with the huge value to try and get all the values at once. I thought it seemed pretty slow too, considering there are no joins or anything like that happening. Thanks again, Simon On Dec 19, 2:33 am, Pat Allan <[email protected]> wrote: > Hi Simon > > Is x_id and y_id the actual columns you're referencing? If not, can you > provide exactly what your define_index block looks like? It will give me a > better picture of whether your indexing is slow or not. > > Has changing the sql_range_step value made any difference? What happens if > you put it back to the default of 1000? 30 minutes for 800,000 values does > sound slow, for what appears to be quite a simple index definition. > > Cheers > > -- > Pat > > On 18/12/2010, at 6:50 AM, Simon wrote: > > > Hi there, > > > I have a table I'm indexing that has roughly 800,000 rows. From > > reading around online and in this group I feel like it's taking a long > > time for my index to get generated. > > > I have the following in my model: > > define_index do > > indexes title, body > > has x_id > > has locked > > has created_at, y_id > > > set_property :delta => true > > end > > > In my sphinx.yml file, I have the following: > > max_matches: 1000 > > html_strip: 1 > > sql_range_step: 10000000 > > min_word_len: 3 > > mem_limit: 256M > > > And here is sample output from running rake ts:index: > > indexing index 'entry_core'... > > collected 841492 docs, 1783.4 MB > > collected 0 attr values > > sorted 0.8 Mvalues, 100.0% done > > sorted 205.5 Mhits, 100.0% done > > total 841492 docs, 1783446653 bytes > > total 1343.154 sec, 1327804.99 bytes/sec, 626.50 docs/sec > > indexing index 'entry_delta'... > > collected 0 docs, 0.0 MB > > collected 0 attr values > > sorted 0.0 Mvalues, nan% done > > total 0 docs, 0 bytes > > total 250.385 sec, 0.00 bytes/sec, 0.00 docs/sec > > distributed index 'entry' can not be directly indexed; skipping. > > > It's taking around 25-30 minutes to run (without having any delta > > indexes). That seems like quite a while compared to what I've seen as > > sample times from other people. Does anybody have any suggestions for > > what I could do to improve the performance, or any comments on the > > speed of the indexing compared to what they have seen? > > > Thanks, > > > Simon > > > -- > > You received this message because you are subscribed to the Google Groups > > "Thinking Sphinx" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/thinking-sphinx?hl=en. -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
