if you use the mysql command-line tool to connect to the test database, how quickly does the UPDATE statement (the first sql_query_pre) run?
-- Pat On 16/10/2009, at 8:32 AM, rejeep wrote: > > Under *source klass_core_0* > sql_query_pre = UPDATE `java_classes` SET `delta` = 0 WHERE `delta` = > 1 > sql_query_pre = SET NAMES utf8 > > And under *source klass_delta_0 : klass_core_0* > sql_query_pre = > sql_query_pre = SET NAMES utf8 > > On Oct 15, 9:20 pm, Pat Allan <[email protected]> wrote: >> Sorry, I should have been more clear... in your test.sphinx.conf >> file, >> what are the sql_query_pre values in your klass_core_i source? >> >> -- >> Pat >> >> On 15/10/2009, at 11:16 PM, Pat Allan wrote: >> >> >> >>> Wow, something's really going wrong there with the sql_query_pre SQL >>> statements... can you copy and paste the ones from the klass_core_1 >>> source? >> >>> -- >>> Pat >> >>> On 15/10/2009, at 7:51 PM, rejeep wrote: >> >>>> Sorry for the late reply. >> >>>> Here's the log: >>>> Sphinx 0.9.9-rc2 (r1785) >>>> Copyright (c) 2001-2009, Andrew Aksyonoff >> >>>> using config file '/home/rejeep/dev/closing/config/ >>>> test.sphinx.conf'... >>>> indexing index 'klass_core'... >>>> ERROR: index 'klass_core': sql_query_pre[0]: Lock wait timeout >>>> exceeded; try restarting transaction (DSN=mysql:// >>>> closing:*...@localhost:3306/closing_test). >>>> total 0 docs, 0 bytes >>>> total 51.825 sec, 0 bytes/sec, 0.00 docs/sec >>>> indexing index 'klass_delta'... >>>> collected 0 docs, 0.0 MB >>>> collected 0 attr values >>>> sorted 0.0 Mvalues, nan% done >>>> total 0 docs, 0 bytes >>>> total 0.005 sec, 0 bytes/sec, 0.00 docs/sec >>>> distributed index 'klass' can not be directly indexed; skipping. >>>> indexing index 'interface_core'... >>>> ERROR: index 'interface_core': sql_query_pre[0]: Lock wait timeout >>>> exceeded; try restarting transaction (DSN=mysql:// >>>> closing:*...@localhost:3306/closing_test). >>>> total 0 docs, 0 bytes >>>> total 50.052 sec, 0 bytes/sec, 0.00 docs/sec >>>> indexing index 'interface_delta'... >>>> collected 0 docs, 0.0 MB >>>> collected 0 attr values >>>> sorted 0.0 Mvalues, nan% done >>>> total 0 docs, 0 bytes >>>> total 0.005 sec, 0 bytes/sec, 0.00 docs/sec >>>> distributed index 'interface' can not be directly indexed; >>>> skipping. >>>> indexing index 'enum_core'... >>>> ERROR: index 'enum_core': sql_query_pre[0]: Lock wait timeout >>>> exceeded; try restarting transaction (DSN=mysql:// >>>> closing:*...@localhost:3306/closing_test). >>>> total 0 docs, 0 bytes >>>> total 51.055 sec, 0 bytes/sec, 0.00 docs/sec >>>> indexing index 'enum_delta'... >>>> collected 0 docs, 0.0 MB >>>> collected 0 attr values >>>> sorted 0.0 Mvalues, nan% done >>>> total 0 docs, 0 bytes >>>> total 0.005 sec, 0 bytes/sec, 0.00 docs/sec >>>> distributed index 'enum' can not be directly indexed; skipping. >>>> indexing index 'annotation_core'... >>>> ERROR: index 'annotation_core': sql_query_pre[0]: Lock wait timeout >>>> exceeded; try restarting transaction (DSN=mysql:// >>>> closing:*...@localhost:3306/closing_test). >>>> total 0 docs, 0 bytes >>>> total 51.058 sec, 0 bytes/sec, 0.00 docs/sec >>>> indexing index 'annotation_delta'... >>>> collected 0 docs, 0.0 MB >>>> collected 0 attr values >>>> sorted 0.0 Mvalues, nan% done >>>> total 0 docs, 0 bytes >>>> total 0.005 sec, 0 bytes/sec, 0.00 docs/sec >>>> distributed index 'annotation' can not be directly indexed; >>>> skipping. >>>> total 4 reads, 0.000 sec, 32.0 kb/call avg, 0.0 msec/call avg >>>> total 16 writes, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg >>>> rotating indices: succesfully sent SIGHUP to searchd (pid=11305). >>>> Loaded suite /usr/bin/rake >> >>>> On Oct 11, 8:44 pm, Pat Allan <[email protected]> wrote: >>>>> Right. Can you show me the output when it's running slowly? >> >>>>> -- >>>>> Pat >> >>>>> On 11/10/2009, at 7:09 PM, rejeep wrote: >> >>>>>> That is not slow at all. But that's because the database does not >>>>>> contain any records. But if I create a record from a factory in >>>>>> the >>>>>> test and then run the rake task, also from the test, it will take >>>>>> forever. >> >>>>>> On Oct 11, 1:36 pm, Pat Allan <[email protected]> wrote: >>>>>>> Either I'm missing it, or there isn't actually any data being >>>>>>> indexed >>>>>>> (nor does it seem to be running slowly). What's the output (and >>>>>>> how >>>>>>> are you running it) when it *is* running slowly? >> >>>>>>> -- >>>>>>> Pat >> >>>>>>> On 10/10/2009, at 1:28 PM, rejeep wrote: >> >>>>>>>> The test database is exactly the same as the development. I >>>>>>>> don't >>>>>>>> know >>>>>>>> if it matters, but it's a cucumber test, so it's actually the >>>>>>>> cucumber >>>>>>>> environment. But that environment is only linking to test, so >>>>>>>> it >>>>>>>> should be exactly the same. >> >>>>>>>> $ rake ts:in RAILS_ENV=test >>>>>>>> (in /home/rejeep/dev/closing) >>>>>>>> Generating Configuration to /home/rejeep/dev/closing/config/ >>>>>>>> test.sphinx.conf >>>>>>>> Sphinx 0.9.9-rc2 (r1785) >>>>>>>> Copyright (c) 2001-2009, Andrew Aksyonoff >> >>>>>>>> using config file '/home/rejeep/dev/closing/config/ >>>>>>>> test.sphinx.conf'... >>>>>>>> indexing index 'klass_core'... >>>>>>>> collected 0 docs, 0.0 MB >>>>>>>> collected 0 attr values >>>>>>>> sorted 0.0 Mvalues, nan% done >>>>>>>> total 0 docs, 0 bytes >>>>>>>> total 0.005 sec, 0 bytes/sec, 0.00 docs/sec >>>>>>>> indexing index 'klass_delta'... >>>>>>>> collected 0 docs, 0.0 MB >>>>>>>> collected 0 attr values >>>>>>>> sorted 0.0 Mvalues, nan% done >>>>>>>> total 0 docs, 0 bytes >>>>>>>> total 0.001 sec, 0 bytes/sec, 0.00 docs/sec >>>>>>>> distributed index 'klass' can not be directly indexed; >>>>>>>> skipping. >>>>>>>> indexing index 'interface_core'... >>>>>>>> collected 0 docs, 0.0 MB >>>>>>>> collected 0 attr values >>>>>>>> sorted 0.0 Mvalues, nan% done >>>>>>>> total 0 docs, 0 bytes >>>>>>>> total 0.002 sec, 0 bytes/sec, 0.00 docs/sec >>>>>>>> indexing index 'interface_delta'... >>>>>>>> collected 0 docs, 0.0 MB >>>>>>>> collected 0 attr values >>>>>>>> sorted 0.0 Mvalues, nan% done >>>>>>>> total 0 docs, 0 bytes >>>>>>>> total 0.001 sec, 0 bytes/sec, 0.00 docs/sec >>>>>>>> distributed index 'interface' can not be directly indexed; >>>>>>>> skipping. >>>>>>>> indexing index 'enum_core'... >>>>>>>> collected 0 docs, 0.0 MB >>>>>>>> collected 0 attr values >>>>>>>> sorted 0.0 Mvalues, nan% done >>>>>>>> total 0 docs, 0 bytes >>>>>>>> total 0.001 sec, 0 bytes/sec, 0.00 docs/sec >>>>>>>> indexing index 'enum_delta'... >>>>>>>> collected 0 docs, 0.0 MB >>>>>>>> collected 0 attr values >>>>>>>> sorted 0.0 Mvalues, nan% done >>>>>>>> total 0 docs, 0 bytes >>>>>>>> total 0.001 sec, 0 bytes/sec, 0.00 docs/sec >>>>>>>> distributed index 'enum' can not be directly indexed; skipping. >>>>>>>> indexing index 'annotation_core'... >>>>>>>> collected 0 docs, 0.0 MB >>>>>>>> collected 0 attr values >>>>>>>> sorted 0.0 Mvalues, nan% done >>>>>>>> total 0 docs, 0 bytes >>>>>>>> total 0.001 sec, 0 bytes/sec, 0.00 docs/sec >>>>>>>> indexing index 'annotation_delta'... >>>>>>>> collected 0 docs, 0.0 MB >>>>>>>> collected 0 attr values >>>>>>>> sorted 0.0 Mvalues, nan% done >>>>>>>> total 0 docs, 0 bytes >>>>>>>> total 0.002 sec, 0 bytes/sec, 0.00 docs/sec >>>>>>>> distributed index 'annotation' can not be directly indexed; >>>>>>>> skipping. >>>>>>>> total 8 reads, 0.000 sec, 32.0 kb/call avg, 0.0 msec/call avg >>>>>>>> total 32 writes, 0.001 sec, 0.0 kb/call avg, 0.0 msec/call avg >>>>>>>> Loaded suite /usr/bin/rake >>>>>>>> Started >> >>>>>>>> Finished in 0.000242 seconds. >> >>>>>>>> 0 tests, 0 assertions, 0 failures, 0 errors >> >>>>>>>> On Oct 10, 10:42 am, Pat Allan <[email protected]> >>>>>>>> wrote: >>>>>>>>> If there's only one record, then it shouldn't matter what your >>>>>>>>> range >>>>>>>>> step is, because Sphinx finds the smallest and largest ids >>>>>>>>> available - >>>>>>>>> which in this case, are the same number. >> >>>>>>>>> Do you have all the same database indexes on your test db and >>>>>>>>> development db? What's the output of rake ts:in >>>>>>>>> RAILS_ENV=test? >> >>>>>>>>> -- >>>>>>>>> Pat >> >>>>>>>>> On 09/10/2009, at 8:00 PM, rejeep wrote: >> >>>>>>>>>> I didn't mean that I was trying to index a single record. >>>>>>>>>> What I >>>>>>>>>> meant >>>>>>>>>> was that I only had one record in the database for that test. >>>>>>>>>> So >>>>>>>>>> even >>>>>>>>>> if there would be some high index, I guess it wouldn't take >>>>>>>>>> to >>>>>>>>>> long >>>>>>>>>> anyway. >> >>>>>>>>>> If I try with this conf: >>>>>>>>>> development: >>>>>>>>>> sql_range_step: 1000 >>>>>>>>>> production: >>>>>>>>>> sql_range_step: 1000 >>>>>>>>>> test: >>>>>>>>>> sql_range_step: 1000 >> >>>>>>>>>> It's fast in development, but equally slow in test. >> >>>>>>>>>> It doesn't matter if I use 1000 or 10000000 for >>>>>>>>>> sql_range_step. >> >>>>>>>>>> On Oct 9, 5:43 pm, Pat Allan <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>>> Hi Rejeep >> >>>>>>>>>>> Sphinx doesn't allow you to process a single record - only a >>>>>>>>>>> full >>>>>>>>>>> index. Also, do you have sql_range_step set for the test >>>>>>>>>>> environment, >>>>>>>>>>> as well as the development environment? >> >>>>>>>>>>> -- >>>>>>>>>>> Pat >> >>>>>>>>>>> On 09/10/2009, at 11:53 AM, rejeep wrote: >> >>>>>>>>>>>> Hi! >> >>>>>>>>>>>> I have a site where I want to test the search. From the >>>>>>>>>>>> test I >>>>>>>>>>>> create >>>>>>>>>>>> the records and then create the index. But it's so slow >>>>>>>>>>>> that >>>>>>>>>>>> it >>>>>>>>>>>> times >>>>>>>>>>>> out. Yes, I'm using factories and I am aware of the id >>>>>>>>>>>> problem. >>>>>>>>>>>> But >>>>>>>>>>>> first of all. In the test I only want to index 1 record. >>>>>>>>>>>> And >>>>>>>>>>>> the >>>>>>>>>>>> id of >>>>>>>>>>>> that usually is between 1000 - 5000. So that should not be >>>>>>>>>>>> such a >>>>>>>>>>>> big >>>>>>>>>>>> problem, right? And even thought I set sql_range_step, it >>>>>>>>>>>> is >>>>>>>>>>>> still >>>>>>>>>>>> slow. >> >>>>>>>>>>>> Since it is super fast in development the only thing I >>>>>>>>>>>> could >>>>>>>>>>>> think >>>>>>>>>>>> be >>>>>>>>>>>> the problem was the id's. But since it's not. Does someone >>>>>>>>>>>> has >>>>>>>>>>>> any >>>>>>>>>>>> other suggestion? >> >>>>>>>>>>>> If I skip creating any records before the indexing in the >>>>>>>>>>>> test. >>>>>>>>>>>> Then >>>>>>>>>>>> the indexing is fast. >> >> > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
