[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2017-01-22 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15833400#comment-15833400 ] Yu Li commented on HBASE-16698: --- The JIRA here introduced mvcc preassign feature, but only implement for

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2017-01-22 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15833383#comment-15833383 ] Yu Li commented on HBASE-16698: --- HBASE-17482 is a fix for master branch with the same way as stated in

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2017-01-22 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15833381#comment-15833381 ] Yu Li commented on HBASE-16698: --- Will do below actions soon: 1. Close this JIRA since all codes already

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2017-01-22 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15833378#comment-15833378 ] Yu Li commented on HBASE-16698: --- After some review of the mechanism, let me further explain about the

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2017-01-15 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823179#comment-15823179 ] Hadoop QA commented on HBASE-16698: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15600905#comment-15600905 ] stack commented on HBASE-16698: --- On HBASE-3899, I remembered it as an experiment that was never finished

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-23 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15599149#comment-15599149 ] Yu Li commented on HBASE-16698: --- bq. Lets not backport to 1.2 until it in 1.3. Thats how we generally do

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-22 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598586#comment-15598586 ] stack commented on HBASE-16698: --- Reinstitute hbase-3899? > Performance issue: handlers stuck waiting for

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-22 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598402#comment-15598402 ] stack commented on HBASE-16698: --- [~enis] you think we could park 'context' in the log, let the handler go

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-22 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598266#comment-15598266 ] stack commented on HBASE-16698: --- bq. On master, when I do a jstack with some load, almost all the handlers

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-21 Thread Enis Soztutar (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596488#comment-15596488 ] Enis Soztutar commented on HBASE-16698: --- On master, when I do a jstack with some load, almost all

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-21 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595796#comment-15595796 ] stack commented on HBASE-16698: --- Lets not backport to 1.2 until it in 1.3. Thats how we generally do it.

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-20 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15593787#comment-15593787 ] Yu Li commented on HBASE-16698: --- Thanks for the confirmation [~mantonov] > Performance issue: handlers

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-20 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15593766#comment-15593766 ] Yu Li commented on HBASE-16698: --- Thanks for the question [~enis], the above numbers are against branch-1

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-20 Thread Enis Soztutar (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15593072#comment-15593072 ] Enis Soztutar commented on HBASE-16698: --- Time to resolve this? [~carp84] thanks for the

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-20 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592549#comment-15592549 ] Hudson commented on HBASE-16698: SUCCESS: Integrated in Jenkins build HBase-1.4 #482 (See

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-20 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592458#comment-15592458 ] stack commented on HBASE-16698: --- Makes sense [~mantonov] > Performance issue: handlers stuck waiting for

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-20 Thread Mikhail Antonov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592412#comment-15592412 ] Mikhail Antonov commented on HBASE-16698: - [~stack] [~carp84] after testing on my side I think

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-20 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591092#comment-15591092 ] Yu Li commented on HBASE-16698: --- Thank you sir [~stack], I've just pushed it into branch-1 :-) >

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-19 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590613#comment-15590613 ] stack commented on HBASE-16698: --- +1 on push to branch-1. Let me know if you want me to do it for you

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-19 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590555#comment-15590555 ] Yu Li commented on HBASE-16698: --- [~stack] mind if I push this to branch-1(branch-1.4) first to avoid

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-19 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588348#comment-15588348 ] Yu Li commented on HBASE-16698: --- Perf data for one single region: Test environment (nothing changed but no

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586840#comment-15586840 ] stack commented on HBASE-16698: --- [~carp84] points out above that WALPE doesn't exercise this patch AT ALL

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586638#comment-15586638 ] Hudson commented on HBASE-16698: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1810 (See

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586213#comment-15586213 ] Yu Li commented on HBASE-16698: --- I also have no idea why we introduce the findbugs issue, will also take a

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586211#comment-15586211 ] Yu Li commented on HBASE-16698: --- Thanks [~stack] for opening the sub issue. Regarding WALPE test, mind

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586070#comment-15586070 ] Hadoop QA commented on HBASE-16698: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585979#comment-15585979 ] stack commented on HBASE-16698: --- Ok on my suggestion actually making for a larger patch when I thought it

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585776#comment-15585776 ] stack commented on HBASE-16698: --- I revert the revert of this change from master (so the master patch and

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585766#comment-15585766 ] stack commented on HBASE-16698: --- Numbers looks good. > Performance issue: handlers stuck waiting for

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585433#comment-15585433 ] Yu Li commented on HBASE-16698: --- Ok, here are more performance number with YCSB. First the testing

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-18 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585090#comment-15585090 ] Yu Li commented on HBASE-16698: --- Thanks for coming back sir [~stack] :-) bq. what if you added a new

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15584509#comment-15584509 ] stack commented on HBASE-16698: --- [~allan163] Thanks for the questions [~carp84] So, story is clearer now

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15584453#comment-15584453 ] stack commented on HBASE-16698: --- Let me answer [~enis] I just went through this issue again and the patch.

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-17 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15581685#comment-15581685 ] Yu Li commented on HBASE-16698: --- Thanks for the quick feedback [~chenheng]. The previous decision is for

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-17 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15581674#comment-15581674 ] Yu Li commented on HBASE-16698: --- Current process of {{doMiniBatchMutation}} for branch-1 is like: 1.

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-16 Thread Heng Chen (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15581074#comment-15581074 ] Heng Chen commented on HBASE-16698: --- patch for branch-1 LGTM. +1 We will open it by default on

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-16 Thread Heng Chen (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15581067#comment-15581067 ] Heng Chen commented on HBASE-16698: --- {quote} so in my analysis, waiting for sync and waiting for latch

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-16 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15580983#comment-15580983 ] Allan Yang commented on HBASE-16698: Yes, I know sync operation will batch as many as possible. When

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-16 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15580977#comment-15580977 ] Yu Li commented on HBASE-16698: --- {quote} Update patch for branch-1 after code rebase. Also make it able to

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-16 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15580967#comment-15580967 ] Yu Li commented on HBASE-16698: --- Actually we won't sync one by one, but sync a bunch at a time, or say we

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-16 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15580945#comment-15580945 ] Allan Yang commented on HBASE-16698: After, carefully reviewed the code of branch-1.2, I understand

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-16 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15579655#comment-15579655 ] Hadoop QA commented on HBASE-16698: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-16 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15579535#comment-15579535 ] Yu Li commented on HBASE-16698: --- [~allan163] and [~enis], please let me know whether my explanation address

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576919#comment-15576919 ] Hudson commented on HBASE-16698: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #1787 (See

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread Enis Soztutar (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576798#comment-15576798 ] Enis Soztutar commented on HBASE-16698: --- The other thing is that we cannot maintain two different

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread Enis Soztutar (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576766#comment-15576766 ] Enis Soztutar commented on HBASE-16698: --- I also have a hard time understanding the patch. We are

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576730#comment-15576730 ] Yu Li commented on HBASE-16698: --- And I'd say the previously existed {{cell.getSequenceId() == 0}} check in

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576097#comment-15576097 ] stack commented on HBASE-16698: --- Pardon me. Was looking in master branch. Let me revert this patch from

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread binlijin (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574596#comment-15574596 ] binlijin commented on HBASE-16698: -- I find master branch-1.2 branch-1.1 they are all different. >

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread binlijin (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574588#comment-15574588 ] binlijin commented on HBASE-16698: -- There is no problem for master version, only some of the branch-1

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574578#comment-15574578 ] Allan Yang commented on HBASE-16698: The CountDownLatch can't be moved since we need to wait until

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread binlijin (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574563#comment-15574563 ] binlijin commented on HBASE-16698: -- a forked 1.1.2 version. > Performance issue: handlers stuck

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread Heng Chen (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574517#comment-15574517 ] Heng Chen commented on HBASE-16698: --- I think [~allan163] you are right. It is different between

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-14 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574381#comment-15574381 ] Allan Yang commented on HBASE-16698: {code} //

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-13 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574336#comment-15574336 ] stack commented on HBASE-16698: --- bq. Why handlers stuck at CountDownLatch? They are waiting on sync

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-13 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574087#comment-15574087 ] Hudson commented on HBASE-16698: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1782 (See

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-13 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15573654#comment-15573654 ] stack commented on HBASE-16698: --- Committed below addendum to address this FindBugs complaint: Code

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-13 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572906#comment-15572906 ] Hudson commented on HBASE-16698: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1780 (See

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-13 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571587#comment-15571587 ] Yu Li commented on HBASE-16698: --- Thanks for review [~chenheng], sure will upload numbers for

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-13 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571584#comment-15571584 ] Yu Li commented on HBASE-16698: --- Thanks for revisiting this [~stack] Yes sir, we're running w/ this in

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-12 Thread Heng Chen (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570840#comment-15570840 ] Heng Chen commented on HBASE-16698: --- Numbers seems 20 regions on one RS. If you have time, please

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-12 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570145#comment-15570145 ] stack commented on HBASE-16698: --- So, [~carp84], you running w/ this in production? I should apply this to

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15540760#comment-15540760 ] Hadoop QA commented on HBASE-16698: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-02 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15540746#comment-15540746 ] Yu Li commented on HBASE-16698: --- btw, this testing is against latest code of branch-1, not master >

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-02 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15540744#comment-15540744 ] Yu Li commented on HBASE-16698: --- This is a common workload and nothing special, I hope this simple result

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-10-02 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15540740#comment-15540740 ] Yu Li commented on HBASE-16698: --- Ok, here is the result with a single RS and PE tool, with below command:

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-28 Thread Heng Chen (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528581#comment-15528581 ] Heng Chen commented on HBASE-16698: --- {quote} so the main problem is sequential appends and the logic

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-27 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526894#comment-15526894 ] Yu Li commented on HBASE-16698: --- I don't have much benchmarking number at hand, but from our online cluster

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-26 Thread Heng Chen (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524987#comment-15524987 ] Heng Chen commented on HBASE-16698: --- How much the performance will be downgrade when ops are just for

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-26 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523055#comment-15523055 ] Yu Li commented on HBASE-16698: --- bq. Why not? If a false positive and you can't clean it up... Because

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-25 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15521978#comment-15521978 ] stack commented on HBASE-16698: --- On the patch, I'd be good w/ it going in as off by default in branch-1 and

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-25 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15521975#comment-15521975 ] stack commented on HBASE-16698: --- bq. but I don't think it's a good idea doing the same thing for

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-25 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15521114#comment-15521114 ] Yu Li commented on HBASE-16698: --- Checked below failed UT cases in HadoopQA report and confirmed all could

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15518312#comment-15518312 ] Yu Li commented on HBASE-16698: --- ok, so firstly set per-table is an option not necessity, it's just like

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15518215#comment-15518215 ] Hadoop QA commented on HBASE-16698: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15518057#comment-15518057 ] Hadoop QA commented on HBASE-16698: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread binlijin (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517980#comment-15517980 ] binlijin commented on HBASE-16698: -- I think set per-table is not a good idea, because all regions in a

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517773#comment-15517773 ] Yu Li commented on HBASE-16698: --- btw, will try to get some perf comparison data through YCSB benchmark

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517763#comment-15517763 ] Yu Li commented on HBASE-16698: --- bq. So no speed up if one region only on a regionserver. Ok. I buy it. Yes

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517748#comment-15517748 ] stack commented on HBASE-16698: --- bq. So all CountDownLatch are released in sequential, no parallelism...

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517721#comment-15517721 ] Yu Li commented on HBASE-16698: --- bq. So, why is this patch faster? In current implementation, contention is

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517689#comment-15517689 ] stack commented on HBASE-16698: --- bq. Previously the CountDownLatch will be released one by one due to

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Elliott Clark (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517679#comment-15517679 ] Elliott Clark commented on HBASE-16698: --- If mvcc isn't the same order as the wal log order then

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517572#comment-15517572 ] Yu Li commented on HBASE-16698: --- grep "CountDownLatch.await" in the jstack and we could see 98 handlers

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517563#comment-15517563 ] Yu Li commented on HBASE-16698: --- Thanks for chiming in boss [~stack] :-) bq. Yes, intentionally one-by-one

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517531#comment-15517531 ] stack commented on HBASE-16698: --- Oh. Saw the patch. Why is this faster? I see an added reentrant lock.

[jira] [Commented] (HBASE-16698) Performance issue: handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517490#comment-15517490 ] stack commented on HBASE-16698: --- Thank you for digging in here [~carp84]. This bit I don't follow bq.