[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[ https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576317#comment-15576317 ] Hudson commented on HBASE-16799: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1786 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1786/]) HBASE-16799 CP exposed Store should not expose unwanted APIs. (anoopsamjohn: rev 5f75fa0c4f660939d5eea10e3f5418386e90cea1) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWalAndCompactingMemStoreFlush.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/FlushNonSloppyStoresFirstPolicy.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestIOFencing.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java > CP exposed Store should not expose unwanted APIs > - > > Key: HBASE-16799 > URL: https://issues.apache.org/jira/browse/HBASE-16799 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16799.patch, HBASE-16799.patch, > HBASE-16799_V2.patch, HBASE-16799_V3.patch, HBASE-16799_V4.patch > > > Store is exposed to CPs. The main use cases I can think of are getting store > scanner and other getters which return different states like memstore size, > max seqId etc. Those make sense. > But we added many other APIs which changes the state of the memstore, bulk > load files etc into this interface. Even an API which expose the memstore > itself!. This allow adding mutations into memstore bypassing all steps in > region. We track the memstore size per region level as well as globally. > These only allow us to flush region at sizes and/or flush selected regions > because of global heap pressure. Now if a CP get hold of store and/or > memstore, it can add mutations with out knowledge of these size accounting > and possibly OOME the RS. Similar way the bulk load related APIs. At HRegion > level, there are steps done (WAL write etc) after the bulk load HFile on > store. So bypassing these wont be correct. > In this jira, plan is to remove all such leaked APIs from Store. They are > called from HRegion and we can type cast to HStore to call them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[
https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575430#comment-15575430
]
Hadoop QA commented on HBASE-16799:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 52s
{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 1s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 23s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 135m 28s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.client.TestReplicasClient |
| | org.apache.hadoop.hbase.client.TestScannerTimeout |
| | org.apache.hadoop.hbase.client.TestIncrementFromClientSideWithCoprocessor
|
| | org.apache.hadoop.hbase.client.TestHCM |
| | org.apache.hadoop.hbase.client.TestTableSnapshotScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12833357/HBASE-16799_V4.patch |
| JIRA Issue | HBASE-16799 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 440dd1f465af 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 9a94dc9 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4016/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4016/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4016/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[
https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574919#comment-15574919
]
Hadoop QA commented on HBASE-16799:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 43s
{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 25s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 17s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 130m 40s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.TestCompactingToCellArrayMapMemStore |
| | hadoop.hbase.regionserver.TestDefaultMemStore |
| | hadoop.hbase.regionserver.TestCompactingMemStore |
| Timed out junit tests | org.apache.hadoop.hbase.client.TestReplicaWithCluster
|
| | org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12833321/HBASE-16799_V3.patch |
| JIRA Issue | HBASE-16799 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 93f0f2225576 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 9a94dc9 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4009/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/4009/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4009/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[ https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574663#comment-15574663 ] ramkrishna.s.vasudevan commented on HBASE-16799: +1. > CP exposed Store should not expose unwanted APIs > - > > Key: HBASE-16799 > URL: https://issues.apache.org/jira/browse/HBASE-16799 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16799.patch, HBASE-16799.patch, > HBASE-16799_V2.patch, HBASE-16799_V3.patch > > > Store is exposed to CPs. The main use cases I can think of are getting store > scanner and other getters which return different states like memstore size, > max seqId etc. Those make sense. > But we added many other APIs which changes the state of the memstore, bulk > load files etc into this interface. Even an API which expose the memstore > itself!. This allow adding mutations into memstore bypassing all steps in > region. We track the memstore size per region level as well as globally. > These only allow us to flush region at sizes and/or flush selected regions > because of global heap pressure. Now if a CP get hold of store and/or > memstore, it can add mutations with out knowledge of these size accounting > and possibly OOME the RS. Similar way the bulk load related APIs. At HRegion > level, there are steps done (WAL write etc) after the bulk load HFile on > store. So bypassing these wont be correct. > In this jira, plan is to remove all such leaked APIs from Store. They are > called from HRegion and we can type cast to HStore to call them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[ https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569250#comment-15569250 ] stack commented on HBASE-16799: --- Hmm. This is internals, marked for COPROCs, but yeah, a COPROC that messes w/ memstore will mess up all of our heap accounting. We could I suppose annotate each method that we don't want CPs to use w/ private but CP implementors will probably fail to read the special method notation? Probably best going conservative in this case given the damage misuse can cause. > CP exposed Store should not expose unwanted APIs > - > > Key: HBASE-16799 > URL: https://issues.apache.org/jira/browse/HBASE-16799 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16799.patch, HBASE-16799.patch, > HBASE-16799_V2.patch > > > Store is exposed to CPs. The main use cases I can think of are getting store > scanner and other getters which return different states like memstore size, > max seqId etc. Those make sense. > But we added many other APIs which changes the state of the memstore, bulk > load files etc into this interface. Even an API which expose the memstore > itself!. This allow adding mutations into memstore bypassing all steps in > region. We track the memstore size per region level as well as globally. > These only allow us to flush region at sizes and/or flush selected regions > because of global heap pressure. Now if a CP get hold of store and/or > memstore, it can add mutations with out knowledge of these size accounting > and possibly OOME the RS. Similar way the bulk load related APIs. At HRegion > level, there are steps done (WAL write etc) after the bulk load HFile on > store. So bypassing these wont be correct. > In this jira, plan is to remove all such leaked APIs from Store. They are > called from HRegion and we can type cast to HStore to call them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[ https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567847#comment-15567847 ] Anoop Sam John commented on HBASE-16799: It might be odd to have @private annotated so many methods in a public exposed interface. We do have that in some places now. (May be there also we can avoid I guess). I am not favoring to add @private annotation in an exposed class/interface > CP exposed Store should not expose unwanted APIs > - > > Key: HBASE-16799 > URL: https://issues.apache.org/jira/browse/HBASE-16799 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16799.patch, HBASE-16799.patch, > HBASE-16799_V2.patch > > > Store is exposed to CPs. The main use cases I can think of are getting store > scanner and other getters which return different states like memstore size, > max seqId etc. Those make sense. > But we added many other APIs which changes the state of the memstore, bulk > load files etc into this interface. Even an API which expose the memstore > itself!. This allow adding mutations into memstore bypassing all steps in > region. We track the memstore size per region level as well as globally. > These only allow us to flush region at sizes and/or flush selected regions > because of global heap pressure. Now if a CP get hold of store and/or > memstore, it can add mutations with out knowledge of these size accounting > and possibly OOME the RS. Similar way the bulk load related APIs. At HRegion > level, there are steps done (WAL write etc) after the bulk load HFile on > store. So bypassing these wont be correct. > In this jira, plan is to remove all such leaked APIs from Store. They are > called from HRegion and we can type cast to HStore to call them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[ https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567597#comment-15567597 ] ramkrishna.s.vasudevan commented on HBASE-16799: Why to remove - whether we can add @PRivate tag to those APIs? If any LimitedPrivate interfaces are exposed to CPs that can contain some @Private methods in that? I think that is what we discussed too. > CP exposed Store should not expose unwanted APIs > - > > Key: HBASE-16799 > URL: https://issues.apache.org/jira/browse/HBASE-16799 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16799.patch, HBASE-16799.patch, > HBASE-16799_V2.patch > > > Store is exposed to CPs. The main use cases I can think of are getting store > scanner and other getters which return different states like memstore size, > max seqId etc. Those make sense. > But we added many other APIs which changes the state of the memstore, bulk > load files etc into this interface. Even an API which expose the memstore > itself!. This allow adding mutations into memstore bypassing all steps in > region. We track the memstore size per region level as well as globally. > These only allow us to flush region at sizes and/or flush selected regions > because of global heap pressure. Now if a CP get hold of store and/or > memstore, it can add mutations with out knowledge of these size accounting > and possibly OOME the RS. Similar way the bulk load related APIs. At HRegion > level, there are steps done (WAL write etc) after the bulk load HFile on > store. So bypassing these wont be correct. > In this jira, plan is to remove all such leaked APIs from Store. They are > called from HRegion and we can type cast to HStore to call them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[ https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567511#comment-15567511 ] Anoop Sam John commented on HBASE-16799: Came across this while I was working on HBASE-16747. I had to change signature of APIs in Store and saw it is CP exposed. That made me to think this way. bq.We need to add a warning on the Store API? You mean on other APIs which will be there after this patch? Ya I will add comments in Store interface not to expose APIs that change the state of the store. Also any Store impl that we make have to extend HStore not directly impl this interface. That might happen very very rare. We have HMobStore which is any way extending HStore. > CP exposed Store should not expose unwanted APIs > - > > Key: HBASE-16799 > URL: https://issues.apache.org/jira/browse/HBASE-16799 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16799.patch, HBASE-16799.patch, > HBASE-16799_V2.patch > > > Store is exposed to CPs. The main use cases I can think of are getting store > scanner and other getters which return different states like memstore size, > max seqId etc. Those make sense. > But we added many other APIs which changes the state of the memstore, bulk > load files etc into this interface. Even an API which expose the memstore > itself!. This allow adding mutations into memstore bypassing all steps in > region. We track the memstore size per region level as well as globally. > These only allow us to flush region at sizes and/or flush selected regions > because of global heap pressure. Now if a CP get hold of store and/or > memstore, it can add mutations with out knowledge of these size accounting > and possibly OOME the RS. Similar way the bulk load related APIs. At HRegion > level, there are steps done (WAL write etc) after the bulk load HFile on > store. So bypassing these wont be correct. > In this jira, plan is to remove all such leaked APIs from Store. They are > called from HRegion and we can type cast to HStore to call them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[ https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566897#comment-15566897 ] stack commented on HBASE-16799: --- Good one [~anoop.hbase] What brought this on? When were the methods added? We need to add a warning on the Store API? > CP exposed Store should not expose unwanted APIs > - > > Key: HBASE-16799 > URL: https://issues.apache.org/jira/browse/HBASE-16799 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16799.patch, HBASE-16799.patch, > HBASE-16799_V2.patch > > > Store is exposed to CPs. The main use cases I can think of are getting store > scanner and other getters which return different states like memstore size, > max seqId etc. Those make sense. > But we added many other APIs which changes the state of the memstore, bulk > load files etc into this interface. Even an API which expose the memstore > itself!. This allow adding mutations into memstore bypassing all steps in > region. We track the memstore size per region level as well as globally. > These only allow us to flush region at sizes and/or flush selected regions > because of global heap pressure. Now if a CP get hold of store and/or > memstore, it can add mutations with out knowledge of these size accounting > and possibly OOME the RS. Similar way the bulk load related APIs. At HRegion > level, there are steps done (WAL write etc) after the bulk load HFile on > store. So bypassing these wont be correct. > In this jira, plan is to remove all such leaked APIs from Store. They are > called from HRegion and we can type cast to HStore to call them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[
https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562092#comment-15562092
]
Hadoop QA commented on HBASE-16799:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 16s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 81m 37s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 119m 41s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestDefaultMemStore |
| | hadoop.hbase.regionserver.TestCompactingMemStore |
| | hadoop.hbase.regionserver.TestCompactingToCellArrayMapMemStore |
| Timed out junit tests | org.apache.hadoop.hbase.TestMultiVersions |
| | org.apache.hadoop.hbase.filter.TestFilterListOrOperatorWithBlkCnt |
| | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832441/HBASE-16799_V2.patch |
| JIRA Issue | HBASE-16799 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 6931282c0783 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6b6a801 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/3918/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/3918/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3918/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[
https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561806#comment-15561806
]
Hadoop QA commented on HBASE-16799:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 16s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 26s
{color} | {color:red} hbase-server generated 1 new + 1 unchanged - 0 fixed = 2
total (was 1) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 12m 55s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 0s {color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy |
| | hadoop.hbase.regionserver.compactions.TestStripeCompactor |
| | hadoop.hbase.regionserver.compactions.TestDateTieredCompactor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832432/HBASE-16799.patch |
| JIRA Issue | HBASE-16799 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux f5e238aa94d2 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6b6a801 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| javadoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/3916/artifact/patchprocess/diff-javadoc-javadoc-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/3916/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/3916/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3916/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[
https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561646#comment-15561646
]
Hadoop QA commented on HBASE-16799:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 32s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 31s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 31s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 10s
{color} | {color:red} The patch causes 16 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 20s
{color} | {color:red} The patch causes 16 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 31s
{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 41s
{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 51s
{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 2s
{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m 12s
{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m 22s
{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 34s
{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 22s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 25s
{color} | {color:red} hbase-server generated 1 new + 1 unchanged - 0 fixed = 2
total (was 1) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 32s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 10s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832430/HBASE-16799.patch |
| JIRA Issue | HBASE-16799 |
| Optional Tests | asflicense javac javadoc unit
[jira] [Commented] (HBASE-16799) CP exposed Store should not expose unwanted APIs
[ https://issues.apache.org/jira/browse/HBASE-16799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561531#comment-15561531 ] Anoop Sam John commented on HBASE-16799: Any CP code is not supposed to call these APIs any way. If called, they might create issues at some time. So removing the APIs from 2.0 even if it is a BC break. Let us just deprecate the APIs in branch-1 > CP exposed Store should not expose unwanted APIs > - > > Key: HBASE-16799 > URL: https://issues.apache.org/jira/browse/HBASE-16799 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > > Store is exposed to CPs. The main use cases I can think of are getting store > scanner and other getters which return different states like memstore size, > max seqId etc. Those make sense. > But we added many other APIs which changes the state of the memstore, bulk > load files etc into this interface. Even an API which expose the memstore > itself!. This allow adding mutations into memstore bypassing all steps in > region. We track the memstore size per region level as well as globally. > These only allow us to flush region at sizes and/or flush selected regions > because of global heap pressure. Now if a CP get hold of store and/or > memstore, it can add mutations with out knowledge of these size accounting > and possibly OOME the RS. Similar way the bulk load related APIs. At HRegion > level, there are steps done (WAL write etc) after the bulk load HFile on > store. So bypassing these wont be correct. > In this jira, plan is to remove all such leaked APIs from Store. They are > called from HRegion and we can type cast to HStore to call them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
