In this particular case, there shouldn't be any other indexing going on. Our 5 indexable caches:
"default" - Datastore "logs" -- logging "search" -- text search "prospective_search" -- some weird search :-) "tasks" -- JMS-like stuff We disable logging for this test, and there is no Search, PSearch or Queue taking place. Hence the only ops should be those 4 ops to "default" cache. We did see huge perf drop when trace logging is enabled, and that wouldn't surprise me then, but that's not the case here. But 4 ops, for 50sec ... On Apr 12, 2013, at 2:41 PM, Emmanuel Bernard <[email protected]> wrote: > The index master is essentially a queue + the queue processor. > So if many people as the index master to index data (even as async). > the sync call will have to wait for all the other ones to be done before > carrying on. > But are the other put involving indexed entities? If not, then that's > not what is going on. > > I can't explain one index operation taking 50s if nothing is happening > in parallel. > As Sanne said, use the blackhole backend on your index master node. We > will know if the index is really taking all that time. > > Emmanuel > > On Fri 2013-04-12 14:19, Ales Justin wrote: >> To our "default" cache - which is where we store Datastore entries, >> the test only does 1 put, 1 get, 1 delete, 1 query. >> >> https://github.com/capedwarf/capedwarf-blue/blob/master/cluster-tests/src/test/java/org/jboss/test/capedwarf/cluster/test/QueryTest.java >> >> There are other indexes that participate in this test as well, "logging" >> cache, etc. >> But only "default" is configured with new "sync" stuff, >> others all have this: >> >> <indexing index="LOCAL"> >> <property >> name="hibernate.search.default.worker.execution">async</property> >> >> So I fail to see how this is "bunch of them". >> >> -Ales >> >> On Apr 12, 2013, at 2:12 PM, Emmanuel Bernard <[email protected]> wrote: >> >>> No, it is one RPC call for every put or delete in the grid. I'm sure the >>> test makes a bunch of them. >>> >>> On Fri 2013-04-12 12:40, Bela Ban wrote: >>>> I might have misunderstood something: if this is *one* sync RPC, then >>>> enabling or disabling of bundling won't have an impact; as 60ms added to >>>> 50 secs doesn't make a diff. >>>> >>>> On 4/12/13 12:20 PM, Sanne Grinovero wrote: >>>>> +1 >>>>> I'm going to verify that. Some index tuning options won't hurt either, >>>>> but first I'll set the blackhole to confirm that indexing is the cause. >>>>> >>>>> On Apr 12, 2013 11:12 AM, "Bela Ban" <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> OK, then it isn't JGroups related. Probably some indexing work done by >>>>> HS is on the critical path now, and wasn't before. The test probably >>>>> returned before that work was done. >>>>> >>>>> On 4/12/13 11:21 AM, Ales Justin wrote: >>>>>> Still the same -- after changing this to false. >>>>>> >>>>>> 11:20:04,013 INFO >>>>> [org.hibernate.search.indexes.impl.DirectoryBasedIndexManager] >>>>> (http-/192.168.30.235:8080-2) HSEARCH000168: Serialization service >>>>> Avro SerializationProvider v1.0 being used for index >>>>> 'default_capedwarf-test__com.google.appengine.api.datastore.Entity' >>>>>> 11:20:54,730 INFO [org.jboss.web] (ServerService Thread Pool -- >>>>> 59) JBAS018224: Unregister web context: /capedwarf-tests >>>>>> >>>>>> >>>>>> On Apr 12, 2013, at 9:43 AM, Bela Ban <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>>> You need to set enable_bundling to *false*, not true ! >>>>>>> >>>>>>> On 4/11/13 9:13 PM, Ales Justin wrote: >>>>>>>> Although this change fixes query lookup, >>>>>>>> it adds horrible performance: >>>>>>>> >>>>>>>> Running CapeDwarf cluster QueryTest: >>>>>>>> >>>>>>>> with HSEARCH-1296 >>>>>>>> >>>>>>>> 21:00:27,188 INFO >>>>> [org.hibernate.search.indexes.impl.DirectoryBasedIndexManager] >>>>> (http-/192.168.1.102:8080-1) HSEARCH000168: Serialization service >>>>> Avro SerializationProvider v1.0 being used for index >>>>> 'default_capedwarf-test__com.google.appengine.api.datastore.Entity' >>>>>>>> 21:01:17,911 INFO [org.jboss.web] (ServerService Thread Pool >>>>> -- 49) JBAS018224: Unregister web context: /capedwarf-tests >>>>>>>> >>>>>>>> 50sec >>>>>>>> >>>>>>>> old 4.2.0.Final HS >>>>>>>> >>>>>>>> 21:08:19,988 INFO >>>>> [org.hibernate.search.indexes.impl.DirectoryBasedIndexManager] >>>>> (http-/192.168.1.102:8080-2) HSEARCH000168: Serialization service >>>>> Avro SerializationProvider v1.0 being used for index >>>>> 'default_capedwarf-test__com.google.appengine.api.datastore.Entity' >>>>>>>> 21:08:20,829 INFO [org.jboss.web] (ServerService Thread Pool >>>>> -- 49) JBAS018224: Unregister web context: /capedwarf-tests >>>>>>>> >>>>>>>> 841ms >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> I added >>>>>>>> >>>>>>>> <property >>>>> name="enable_bundling">true</property> >>>>>>>> >>>>>>>> to AS jgroups transport config, but no improvement. >>>>>>>> >>>>>>>> Any (other) idea? >>>>>>>> >>>>>>>> -Ales >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> infinispan-dev mailing list >>>>>>>> [email protected] >>>>> <mailto:[email protected]> >>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Bela Ban, JGroups lead (http://www.jgroups.org) >>>>>>> _______________________________________________ >>>>>>> infinispan-dev mailing list >>>>>>> [email protected] >>>>> <mailto:[email protected]> >>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> infinispan-dev mailing list >>>>>> [email protected] >>>>> <mailto:[email protected]> >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> >>>>> >>>>> -- >>>>> Bela Ban, JGroups lead (http://www.jgroups.org) >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> [email protected] <mailto:[email protected]> >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> [email protected] >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>> >>>> -- >>>> Bela Ban, JGroups lead (http://www.jgroups.org) >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> [email protected] >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> _______________________________________________ >>> infinispan-dev mailing list >>> [email protected] >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> _______________________________________________ >> infinispan-dev mailing list >> [email protected] >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
