[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15807178#comment-15807178 ] Ted Yu commented on HBASE-17408: Created HBASE-17438 for the server side work. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch, > HBASE-17408.v2.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15807035#comment-15807035 ] ChiaPing Tsai commented on HBASE-17408: --- Can we have a separate JIRA to enhance that? > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch, > HBASE-17408.v2.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806211#comment-15806211 ] Hudson commented on HBASE-17408: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2271 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2271/]) HBASE-17408 Introduce per request limit by number of mutations (ChiaPing (tedyu: rev b2a9be02ac10908438dc84e9d483bc8785d7ce19) * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSimpleRequestController.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/SimpleRequestController.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerObserver.java > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch, > HBASE-17408.v2.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805486#comment-15805486 ] Enis Soztutar commented on HBASE-17408: --- Indeed. See the patch at HBASE-14946. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch, > HBASE-17408.v2.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803286#comment-15803286 ] ChiaPing Tsai commented on HBASE-17408: --- bq. Should we do a follow up for the server side as well Do you mean the servers process the partial rows, and then return exception for making client retry the remaining rows? > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch, > HBASE-17408.v2.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803008#comment-15803008 ] Enis Soztutar commented on HBASE-17408: --- The patch looks good for the client side. Should we do a follow up for the server side as well, similar to HBASE-14946? Because not all clients will go through AP (async client, and C++, etc). > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch, > HBASE-17408.v2.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[
https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802279#comment-15802279
]
Hadoop QA commented on HBASE-17408:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
38s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 0s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 3s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 79m 26s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 119m 21s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12845826/HBASE-17408.v2.patch |
| JIRA Issue | HBASE-17408 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux bd297f0b798c 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / df98d8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5150/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5150/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Introduce per request limit by number of mutations
>
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802009#comment-15802009 ] ChiaPing Tsai commented on HBASE-17408: --- programmers work at night :D > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch, > HBASE-17408.v2.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801993#comment-15801993 ] Ted Yu commented on HBASE-17408: +1 on v2, pending QA. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch, > HBASE-17408.v2.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801989#comment-15801989 ] Ted Yu commented on HBASE-17408: It's already 1am in Taiwan, right ? Please take your time in coming up with patch(es) :-) > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch, > HBASE-17408.v2.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801903#comment-15801903 ] ChiaPing Tsai commented on HBASE-17408: --- copy that > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801896#comment-15801896 ] Ted Yu commented on HBASE-17408: bq. The heap size of row (rowSize) is useless for RequestRowsChecker. I agree. Mind changing the parameter name to heapSizeOfRow - just to make it clearer. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801844#comment-15801844 ] ChiaPing Tsai commented on HBASE-17408: --- bq. Please move the above check immediately below where this.maxRowsPerRequest is assigned. copy that. bq. Why is 1 used in the last line above ? It means that an extra row is accepted, so we increment the row count by one. bq. rowSize has no effect ? Yes, RequestRowsChecker only consider the number of rows. The heap size of row (rowSize) is useless for RequestRowsChecker. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Ted Yu >Assignee: ChiaPing Tsai > Fix For: 2.0.0 > > Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch > > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[
https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801798#comment-15801798
]
Ted Yu commented on HBASE-17408:
Thanks for taking this JIRA.
{code}
+if (this.maxRowsPerRequest <= 0) {
+ throw new IllegalArgumentException("maxRowsPerRequest="
{code}
Please move the above check immediately below where this.maxRowsPerRequest is
assigned.
{code}
+public void notifyFinal(ReturnCode code, HRegionLocation loc, long
rowSize) {
+ if (code == ReturnCode.INCLUDE) {
+long currentRows = serverRows.containsKey(loc.getServerName())
+? serverRows.get(loc.getServerName()) : 0L;
+serverRows.put(loc.getServerName(), currentRows + 1);
{code}
Why is 1 used in the last line above ? rowSize has no effect ?
> Introduce per request limit by number of mutations
> --
>
> Key: HBASE-17408
> URL: https://issues.apache.org/jira/browse/HBASE-17408
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: ChiaPing Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-17408.v0.patch, HBASE-17408.v1.patch
>
>
> HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the
> amount of data sent from client.
> We should consider adding per request limit through the number of mutations
> in a batch.
> In recent troubleshooting sessions, customer had to do this in their
> application code to avoid OOME on the server side.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[
https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801731#comment-15801731
]
Hadoop QA commented on HBASE-17408:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 28s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 81m 33s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 123m 49s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12845797/HBASE-17408.v1.patch |
| JIRA Issue | HBASE-17408 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 8171d10e0fc7 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / dba103e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5147/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5147/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Introduce per request limit by number of
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[
https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801217#comment-15801217
]
Hadoop QA commented on HBASE-17408:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 38s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 2s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 41s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12845775/HBASE-17408.v0.patch |
| JIRA Issue | HBASE-17408 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 3159b8bd43ae 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / dba103e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5144/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5144/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Introduce per request limit by number of mutations
> --
>
> Key: HBASE-17408
> URL: https://issues.apache.org/jira/browse/HBASE-17408
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: ChiaPing Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-17408.v0.patch
>
>
> HBASE-16224 introduced
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796349#comment-15796349 ] stack commented on HBASE-17408: --- bq. Ted, giving more context definitely helps others to follow along. Yep. bq. The server would have to buffer up all the responses causing OOM on the server side. We need to chunk multiget as we do Scans? HBASE-15414 bq. On the second instance (which is the same reason Ted logged this issue), the application was doing multi requests with Increment. In this case, it was doing 15K increments in a single RPC, which would take longer than RPC timeout. Thats a good one. Thats broke. Yeah, a bound would help piecemealing in increments in batches < 15k. Thanks makes sense. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796273#comment-15796273 ] Ted Yu commented on HBASE-17408: Thanks for providing more background, [~enis] I was waiting for server log with DEBUG log showing bloated exception size in response. The operator was novice who didn't know how to deploy the hbase-server jar over the long weekend. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796238#comment-15796238 ] Enis Soztutar commented on HBASE-17408: --- Ted, giving more context definitely helps others to follow along. I have seen applications sending very big "batches" in two different settings. In the first instance, the application was sending multi-get requests, but each RPC would end up accumulating about 5K get requests (because application would call HTable.get(List) with default settings with a list of that size). The server would have to buffer up all the responses causing OOM on the server side. On the second instance (which is the same reason Ted logged this issue), the application was doing multi requests with Increment. In this case, it was doing 15K increments in a single RPC, which would take longer than RPC timeout. The following RPCs would then get ~15K exceptions due to nonce collisions (because previous RPC indeed succeeds regardless of the timeout on the client side). This RPC response is multi-GB in size causing OOMs. I have not checked the recent status for the AP, especially with HBASE-16224, but even with that, we should have both the heap size limit and number of actions-per-batch limit. With default settings, doing HTable.get(List) with a 10K gets in that list should not cause OOM on the server side (same with HTable.batch()). If we already have limits for these (sorry I did not check yet), then we should adjust down the default values at least. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796011#comment-15796011 ] stack commented on HBASE-17408: --- Suggest that you have evidence stronger than 'likely' before you go about fixes. And we'd limit the number of requests because we are handling exceptions in a dumb fashion? Why not address the latter instead? We already have size-based limit. What is the poor user to make of a new config that does count of items too? > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795755#comment-15795755 ] Ted Yu commented on HBASE-17408: See HBASE-17387 for some background. The OOME was likely caused by bloated nonce related exceptions in response. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795756#comment-15795756 ] stack commented on HBASE-17408: --- Yes, why? What was the problem. > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17408) Introduce per request limit by number of mutations
[ https://issues.apache.org/jira/browse/HBASE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795722#comment-15795722 ] ChiaPing Tsai commented on HBASE-17408: --- bq. We should consider adding per request limit through the number of mutations in a batch. Why it causes the OOME? > Introduce per request limit by number of mutations > -- > > Key: HBASE-17408 > URL: https://issues.apache.org/jira/browse/HBASE-17408 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > > HBASE-16224 introduced hbase.client.max.perrequest.heapsize to limit the > amount of data sent from client. > We should consider adding per request limit through the number of mutations > in a batch. > In recent troubleshooting sessions, customer had to do this in their > application code to avoid OOME on the server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
