And I'll try to pinpoint the actual change causing this (this makes that much easier)
________________________________ From: lars hofhansl <[email protected]> To: "[email protected]" <[email protected]> Sent: Thursday, November 29, 2012 11:51 AM Subject: Re: small perf degradation in 0.94 trunk vs. older versions Thanks N! (I didn't know it was so easy to reproduce) ________________________________ From: Nicolas Liochon <[email protected]> To: lars hofhansl <[email protected]> Cc: [email protected] Sent: Thursday, November 29, 2012 8:01 AM Subject: Re: small perf degradation in 0.94 trunk vs. older versions Here are some tests on a standalone hbase. I just create a table with 3000 regions from the shell: create 't1', 'f1', {NUMREGIONS => 3000, SPLITALGO => 'HexStringSplit'} It seems that's it's between august 20th and september 1st, but there could be some small other stuff along the line as well... commit eb36a3e410998e72d76edcc5c181dfa54bba1c39 Date: Tue Oct 2 20:39:14 2012 +0000 0 row(s) in 184.4040 seconds 0 row(s) in 191.4830 seconds 0 row(s) in 189.6370 seconds 0 row(s) in 198.9080 seconds 0 row(s) in 202.8180 seconds commit 8c93fcfca2eb12162b99b8e1e327bab872bba6b7 Date: Wed May 30 17:06:52 2012 +0000 create 't1', 'f1', {NUMREGIONS => 3000, SPLITALGO => 'HexStringSplit'} 0 row(s) in 165.2160 seconds 0 row(s) in 142.7890 seconds 0 row(s) in 137.7260 seconds commit 01f0b3c77ea30b1c22a0e96929e97bb9f5faab2bgit Date: Thu Jul 19 22:19:49 2012 +0000 0 row(s) in 173.9160 seconds 0 row(s) in 155.0360 seconds 0 row(s) in 141.5210 seconds 0 row(s) in 140.6480 seconds commit bc51a7267d2647630224f646b62b80458591414a Date: Sat Sep 1 04:35:29 2012 +0000 0 row(s) in 161.1990 seconds 0 row(s) in 171.0010 seconds 0 row(s) in 167.2430 seconds 0 row(s) in 189.9960 seconds 0 row(s) in 205.0460 seconds 0 row(s) in 194.4270 seconds 0 row(s) in 191.9020 seconds commit 5ea73b7992819ec6e24ba7c2c5beee306f7d12da Date: Thu Aug 9 21:52:21 2012 +0000 0 row(s) in 164.1430 seconds 0 row(s) in 154.3410 seconds 0 row(s) in 154.3370 seconds 0 row(s) in 142.6060 seconds 0 row(s) in 138.8110 seconds 0 row(s) in 144.4190 seconds commit 7179970b4a00ce630004d72e8e45e30fa9f4881b Date: Mon Aug 20 23:38:50 2012 +0000 0 row(s) in 146.4550 seconds 0 row(s) in 150.3020 seconds 0 row(s) in 143.6050 seconds 0 row(s) in 148.4000 seconds 0 row(s) in 164.8320 seconds commit b4e5c3ae45935a37e1ab8651998c6a8131180eec Date: Mon Aug 13 19:32:19 2012 +0000 0 row(s) in 151.0200 seconds 0 row(s) in 186.2860 seconds 0 row(s) in 141.6130 seconds 0 row(s) in 141.1880 seconds 0 row(s) in 140.9320 seconds 0 row(s) in 162.9540 seconds 0 row(s) in 139.5530 seconds On Thu, Nov 29, 2012 at 10:07 AM, Nicolas Liochon <[email protected]> wrote: > I'm going to try with some random commits. Hopefully I will reproduce it a > on standalone instance as well... > Stay tuned :-) > > > > On Wed, Nov 28, 2012 at 11:02 PM, lars hofhansl <[email protected]>wrote: > >> Did a quick scan through the changes committed since 0.94.1: >> HBASE-6608 >> HBASE-6364 >> HBASE-6587 >> HBASE-6537 >> HBASE-6713 >> HBASE-6438 >> HBASE-6299 >> HBASE-7018 >> HBASE-7038 >> HBASE-7060 >> >> Any chance you can do this again with 0.94.2 (or even do a binary search >> through the commits to pinpoint the change)? >> >> -- Lars >> >> ------------------------------ >> *From:* Nicolas Liochon <[email protected]> >> *To:* [email protected]; lars hofhansl <[email protected]> >> *Sent:* Tuesday, November 27, 2012 10:42 AM >> *Subject:* Re: small perf degradation in 0.94 trunk vs. older versions >> >> It was Version 0.94.3, r38dbd22c99debd9010e9e5f4fbabeeaf3c4e1ddd >> >> >> On Tue, Nov 27, 2012 at 7:41 PM, lars hofhansl <[email protected]>wrote: >> >> Interesting. Thanks N. >> >> I'll look through the 0.94 commits since 0.94.1 to see what's causing >> this. >> With 0.94 trunk you mean the 0.94.3RC? >> >> >> -- Lars >> >> >> >> ________________________________ >> From: Nicolas Liochon <[email protected]> >> To: [email protected] >> Sent: Tuesday, November 27, 2012 5:07 AM >> Subject: small perf degradation in 0.94 trunk vs. older versions >> >> Hi, >> >> On a create table / reassignment, I feel we lost some performances >> recently >> on 0.94. It's not huge (5% to 10%), but I would prefer them to be the >> other >> way around. >> Here are some tests I've done on test cluster, all the time with Hadoop >> 1.1.0; >> >> scenario: 2 RS. Create a table with 3000 regions. >> Start a third RS. Kill -15 the second one: there are 1500 regions to >> reassign. >> >> I've made multiple measures, there are all there: >> It's in seconds. >> >> Creating a table with 3000 regions: >> 0.92: 261s; 260s >> 0.94.0: 260s; 260s >> 0.94.1: 261s; 260s >> 0.94 trunk: 292s; 281s; 282s; >> 0.96 trunk: 173s; 178s >> >> Reassign after the kill >> 0.92: 107s, 110s >> 0.94.0: 105s; 105s >> 0.94.1: 107s; 107s >> 0.94 trunk: 122s; 105s; 116s >> 0.96 trunk: 50s; 50s; >> >> I don't know if there is a reason the the 0.94 trunk results... >> The results for 0.96s are actually not that great, my tests (on a single >> machine) were much better a few months ago. I will look at that. >> >> Cheers, >> >> N. >> >> >> >> >> >
