[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882220#comment-13882220 ] Hudson commented on HBASE-8755: --- FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #65 (See

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882215#comment-13882215 ] Hudson commented on HBASE-8755: --- SUCCESS: Integrated in HBase-TRUNK #4856 (See

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13867544#comment-13867544 ] ramkrishna.s.vasudevan commented on HBASE-8755: --- Nice one. A new write

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847690#comment-13847690 ] stack commented on HBASE-8755: -- I did a compare over three hours and throughput flips in

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847698#comment-13847698 ] Andrew Purtell commented on HBASE-8755: --- Yes [~stack], let's get this out into a

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847958#comment-13847958 ] Hudson commented on HBASE-8755: --- SUCCESS: Integrated in HBase-TRUNK #4722 (See

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848055#comment-13848055 ] Hudson commented on HBASE-8755: --- SUCCESS: Integrated in HBase-0.98 #11 (See

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848092#comment-13848092 ] Hudson commented on HBASE-8755: --- SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #8 (See

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848207#comment-13848207 ] Hudson commented on HBASE-8755: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #5 (See

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846412#comment-13846412 ] stack commented on HBASE-8755: -- All the Async* are just hanging out waiting. Missed

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846583#comment-13846583 ] stack commented on HBASE-8755: -- I notice a syncer exited (it is missing from the thread

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846657#comment-13846657 ] stack commented on HBASE-8755: -- Catching all exceptions, I got a NPE. 2013-12-12

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846984#comment-13846984 ] Hadoop QA commented on HBASE-8755: -- {color:red}-1 overall{color}. Here are the results

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846994#comment-13846994 ] stack commented on HBASE-8755: -- This is what I'll commit. I've been running it on small

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846997#comment-13846997 ] stack commented on HBASE-8755: -- 32 threads on one node writing a cluster of 4 nodes (~8

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847069#comment-13847069 ] Hadoop QA commented on HBASE-8755: -- {color:red}-1 overall{color}. Here are the results

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847194#comment-13847194 ] Feng Honghua commented on HBASE-8755: - [~stack] thanks. seems no further blocking

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847195#comment-13847195 ] Feng Honghua commented on HBASE-8755: - bq.It does 10% less throughput after

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846090#comment-13846090 ] stack commented on HBASE-8755: -- Pardon me. This is taking a while; hardware issues and now I

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846107#comment-13846107 ] stack commented on HBASE-8755: -- Hmm... Happens when this patch is in place. Stuck here:

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846110#comment-13846110 ] stack commented on HBASE-8755: -- Will provide more on this tomorrow. A new write thread

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846130#comment-13846130 ] Feng Honghua commented on HBASE-8755: - The stack-traces are ok: when log-roll occurs,

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13844108#comment-13844108 ] Hadoop QA commented on HBASE-8755: -- {color:red}-1 overall{color}. Here are the results

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13844827#comment-13844827 ] stack commented on HBASE-8755: -- All sounds good above [~fenghh] I'm running a few tests here

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13844838#comment-13844838 ] Ted Yu commented on HBASE-8755: --- +1 from me. A new write thread model for HLog to improve

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13844839#comment-13844839 ] Jonathan Hsieh commented on HBASE-8755: --- I did most of a review of v4 last week --

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13844859#comment-13844859 ] Himanshu Vashishtha commented on HBASE-8755: Thanks for the explanation

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13845035#comment-13845035 ] Feng Honghua commented on HBASE-8755: - Thanks [~jmhsieh], I've made and attached a new

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13845038#comment-13845038 ] Feng Honghua commented on HBASE-8755: - [~v.himanshu]: Looking forward to your

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13845103#comment-13845103 ] Hadoop QA commented on HBASE-8755: -- {color:red}-1 overall{color}. Here are the results

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13845106#comment-13845106 ] stack commented on HBASE-8755: -- I'm running some tests local just to make sure. Will report

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13843224#comment-13843224 ] Feng Honghua commented on HBASE-8755: - [~stack] : thanks for the comment, below are

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13843227#comment-13843227 ] Feng Honghua commented on HBASE-8755: - I'll read and update according to

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13843231#comment-13843231 ] Jean-Marc Spaggiari commented on HBASE-8755: For the random, you can also use

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13843925#comment-13843925 ] Feng Honghua commented on HBASE-8755: - [~v.himanshu]: bq.1) log rolling thread

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13843926#comment-13843926 ] Feng Honghua commented on HBASE-8755: - [~jmspaggi] bq.For the random, you can also use

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13841093#comment-13841093 ] Andrew Purtell commented on HBASE-8755: --- This work looks pretty far along. If we can

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840001#comment-13840001 ] Himanshu Vashishtha commented on HBASE-8755: This is some awesome stuff

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840074#comment-13840074 ] stack commented on HBASE-8755: -- Nice review [~himan...@cloudera.com] On 3., above, yes, that

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840663#comment-13840663 ] Jonathan Hsieh commented on HBASE-8755: --- I posted v4 for trunk on reviewboard and

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840710#comment-13840710 ] stack commented on HBASE-8755: -- Let me look into changing HLogPE after Himanshu's suggestion

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840733#comment-13840733 ] Himanshu Vashishtha commented on HBASE-8755: I agree on HLogPE comment. Yes,

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840812#comment-13840812 ] Feng Honghua commented on HBASE-8755: - Thanks very much for [~v.himanshu] and

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13839662#comment-13839662 ] stack commented on HBASE-8755: -- Here is more review on the patch. Make the changes suggested

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13838679#comment-13838679 ] stack commented on HBASE-8755: -- Sorry for the delay getting back to this [~fenghh] and thanks

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13838687#comment-13838687 ] Feng Honghua commented on HBASE-8755: - Thanks [~stack], and also the review from

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13832233#comment-13832233 ] Feng Honghua commented on HBASE-8755: - tried 4 asyncSyncer threads, below are the

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13830678#comment-13830678 ] Feng Honghua commented on HBASE-8755: - bq.Do these no longer pass? = yes, under new

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13830681#comment-13830681 ] Feng Honghua commented on HBASE-8755: - bq.= yes, under new thread model, no explicit

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13830753#comment-13830753 ] Ted Yu commented on HBASE-8755: --- bq. 2/3 have worse perf, 10 has equal perf but introduces

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13830755#comment-13830755 ] stack commented on HBASE-8755: -- bq. Do you remember how many extra threads were introduced ?

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13830356#comment-13830356 ] stack commented on HBASE-8755: -- I ran Feng's script except I left out trying 200 threads.

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13830360#comment-13830360 ] stack commented on HBASE-8755: -- We looked at using disruptor here? MPSC seems to be what we

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13830516#comment-13830516 ] Feng Honghua commented on HBASE-8755: - Thanks [~stack] A small clarification of why

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13830563#comment-13830563 ] stack commented on HBASE-8755: -- Pardon my bad math [~fenghh] Below is correction (numbers

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13830604#comment-13830604 ] stack commented on HBASE-8755: -- Looking at patch... Do these no longer pass? -

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13829631#comment-13829631 ] Feng Honghua commented on HBASE-8755: - [~stack] : I fixed the downgrade for 5 threads

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13829656#comment-13829656 ] stack commented on HBASE-8755: -- I'm starting tests running now. A new write thread model

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13820068#comment-13820068 ] Jean-Marc Spaggiari commented on HBASE-8755: [~fenghh], any chance to rebase

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13820794#comment-13820794 ] Feng Honghua commented on HBASE-8755: - [~jmspaggi] sure, I'm tuning for less than 5

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13808163#comment-13808163 ] Sergey Shelukhin commented on HBASE-8755: - nice. Thanks! A new write thread

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807214#comment-13807214 ] Sergey Shelukhin commented on HBASE-8755: - Question from above: {quote} ...is

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807618#comment-13807618 ] Hangjun Ye commented on HBASE-8755: --- [~sershe], that's a very good question. We found

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803932#comment-13803932 ] stack commented on HBASE-8755: -- Five datanodes with fusionio. hbase tip of 0.96 branch and

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804957#comment-13804957 ] Feng Honghua commented on HBASE-8755: - [~stack] thanks a lot for the detailed perf

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792389#comment-13792389 ] Feng Honghua commented on HBASE-8755: - [~stack] thanks a lot :-) btw: the HLogPE test

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792422#comment-13792422 ] stack commented on HBASE-8755: -- Does that mean that we can just use HLogPE going forward

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792430#comment-13792430 ] Feng Honghua commented on HBASE-8755: - [~stack]: Agree. Seems HLogPE is enough for

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792433#comment-13792433 ] Hangjun Ye commented on HBASE-8755: --- To clarify, a hdfs cluster is still needed to store

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792279#comment-13792279 ] Liu Shaohui commented on HBASE-8755: [~stack] [~fenghh][~yehangjun] We redo the same

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792293#comment-13792293 ] stack commented on HBASE-8755: -- [~fenghh] Our results are the same? A new write thread

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792301#comment-13792301 ] Feng Honghua commented on HBASE-8755: - [~stack]: Yes, seems this patch against trunk

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792313#comment-13792313 ] stack commented on HBASE-8755: -- [~fenghh] will do. Sorry. Meant to do it earlier. Will try

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13779581#comment-13779581 ] Liu Shaohui commented on HBASE-8755: [~stack] [~fenghh] We redo the comparision test

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13779607#comment-13779607 ] Liu Shaohui commented on HBASE-8755: update the result table and add ops diff

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13774603#comment-13774603 ] Feng Honghua commented on HBASE-8755: - bq.If I weren't in Germany right with no VPN

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13775651#comment-13775651 ] stack commented on HBASE-8755: -- bq. What was implication of not shutting this? Were tests

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13771714#comment-13771714 ] Hangjun Ye commented on HBASE-8755: --- Thanks [~stack] for doing this experiment! The

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13772189#comment-13772189 ] stack commented on HBASE-8755: -- [~fenghh] Happy holidays! I didn't profile to see where time

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13770888#comment-13770888 ] stack commented on HBASE-8755: -- Just parking the numbers here. Takes same time to write 5x

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13771587#comment-13771587 ] stack commented on HBASE-8755: -- I was wrong that HLogPE called doWrite. It calls append as

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13770455#comment-13770455 ] stack commented on HBASE-8755: -- Trying this on hdfs. It takes WAY longer. Threads stuck

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761563#comment-13761563 ] Feng Honghua commented on HBASE-8755: - Thanks [~stack]. Looking forward to your test

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761065#comment-13761065 ] stack commented on HBASE-8755: -- Staring in on taking a look at this. I tried

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13721960#comment-13721960 ] Jean-Marc Spaggiari commented on HBASE-8755: Ok. I did some other tries and

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13721973#comment-13721973 ] Feng Honghua commented on HBASE-8755: - [~jmspaggi], thanks for your test, some

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13722098#comment-13722098 ] Jean-Marc Spaggiari commented on HBASE-8755: Hi [~fenghh], I ran in

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718252#comment-13718252 ] Jean-Marc Spaggiari commented on HBASE-8755: Hi [~fenghh], I was in a process

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718270#comment-13718270 ] Feng Honghua commented on HBASE-8755: - [~jmspaggi] HBASE-8755-0.94-V1.patch is good.

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13711711#comment-13711711 ] Tianying Chang commented on HBASE-8755: --- [~zjushch] In your test, you use 300 RPC

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708433#comment-13708433 ] Jean-Marc Spaggiari commented on HBASE-8755: Hi [~fenghh], the serves

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708473#comment-13708473 ] Feng Honghua commented on HBASE-8755: - Thanks a lot [~jmspaggi], looking forward to

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708189#comment-13708189 ] Feng Honghua commented on HBASE-8755: - [~jmspaggi], what's your result of running YCSB

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13691138#comment-13691138 ] Jean-Marc Spaggiari commented on HBASE-8755: It was PerformanceEvaluation and

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13691178#comment-13691178 ] Hangjun Ye commented on HBASE-8755: --- It possibly doesn't have significant difference

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690238#comment-13690238 ] Lars Hofhansl commented on HBASE-8755: -- I ran some local tests, and even with a

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690259#comment-13690259 ] Jean-Marc Spaggiari commented on HBASE-8755: Here are my results:

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690288#comment-13690288 ] Lars Hofhansl commented on HBASE-8755: -- Interesting. For writing the WAL it should

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690375#comment-13690375 ] stack commented on HBASE-8755: -- [~jmspaggi] Thanks boss. What test is this? YCSB w/ how

[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput

[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688936#comment-13688936 ] Feng Honghua commented on HBASE-8755: - [~zjushch] We run the same tests as yours, and

  1   2   >