[jira] [Assigned] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky reassigned HBASE-17662: --- Assignee: Anastasia Braginsky > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17675) ReplicationEndpoint should choose new sinks if a SaslException occurs
[ https://issues.apache.org/jira/browse/HBASE-17675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877738#comment-15877738 ] Hudson commented on HBASE-17675: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #1930 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1930/]) HBASE-17675 ReplicationEndpoint should choose new sinks if a (apurtell: rev 7a196b7fb89bcf5bb83a8110e78575ab18a53012) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java > ReplicationEndpoint should choose new sinks if a SaslException occurs > -- > > Key: HBASE-17675 > URL: https://issues.apache.org/jira/browse/HBASE-17675 > Project: HBase > Issue Type: Bug >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.9 > > Attachments: HBASE-17675.patch > > > We had an issue where a regionserver on our destination side failed to > refresh the keytabs. The source side's replication got stuck because the > HBaseInterClusterReplicationEndpoint only chooses new sinks if there happens > to be a ConnectException but the SaslException is an IOException, which does > not choose new sinks. > I'll put up a patch to check this exception and choose new sinks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17675) ReplicationEndpoint should choose new sinks if a SaslException occurs
[ https://issues.apache.org/jira/browse/HBASE-17675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877745#comment-15877745 ] Hudson commented on HBASE-17675: FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #109 (See [https://builds.apache.org/job/HBase-1.3-JDK7/109/]) HBASE-17675 ReplicationEndpoint should choose new sinks if a (apurtell: rev 00d79eb9fbc34b36b2c85a6d834106c8b69525c9) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java > ReplicationEndpoint should choose new sinks if a SaslException occurs > -- > > Key: HBASE-17675 > URL: https://issues.apache.org/jira/browse/HBASE-17675 > Project: HBase > Issue Type: Bug >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.9 > > Attachments: HBASE-17675.patch > > > We had an issue where a regionserver on our destination side failed to > refresh the keytabs. The source side's replication got stuck because the > HBaseInterClusterReplicationEndpoint only chooses new sinks if there happens > to be a ConnectException but the SaslException is an IOException, which does > not choose new sinks. > I'll put up a patch to check this exception and choose new sinks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17676) Get class name once for all in AbstractFSWAL
[
https://issues.apache.org/jira/browse/HBASE-17676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-17676:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0
Status: Resolved (was: Patch Available)
Pushed into master branch. Thanks for review [~Apache9] [~anoop.hbase].
> Get class name once for all in AbstractFSWAL
>
>
> Key: HBASE-17676
> URL: https://issues.apache.org/jira/browse/HBASE-17676
> Project: HBase
> Issue Type: Improvement
> Components: Performance
>Affects Versions: 2.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0
>
> Attachments: HBASE-17676.patch, HBASE-17676.v2.patch,
> HBASE-17676.v3.patch
>
>
> While verifying HBASE-17471 with high write workload, observed several
> handler thread at getting class name in jstack, as shown below:
> {noformat}
> "B.defaultRpcServer.handler=60,queue=3,port=16020" daemon prio=10
> tid=0x7f0673835800 nid=0x4dec runnable [0x7f06721b5000]
>java.lang.Thread.State: RUNNABLE
> at java.lang.Class.getEnclosingMethod0(Native Method)
> at java.lang.Class.getEnclosingMethodInfo(Class.java:964)
> at java.lang.Class.getEnclosingClass(Class.java:1137)
> at java.lang.Class.getSimpleBinaryName(Class.java:1282)
> at java.lang.Class.getSimpleName(Class.java:1174)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.stampSequenceIdAndPublishToRingBuffer(FSHLog.java:1251)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1238)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3173)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2874)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:823)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:785)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2259)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:848)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:756)
> {noformat}
> We could get the class name in constructor and use it for all places needed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877747#comment-15877747 ] Anastasia Braginsky commented on HBASE-17662: - [~anoop.hbase], [~ram_krish], I have addressed all the issues you raised in the code review. Please take a look on the recent version in the review board. Please consider for commit as the last patch passes the QA. Thanks! > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17676) Get class name once for all in AbstractFSWAL
[
https://issues.apache.org/jira/browse/HBASE-17676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877805#comment-15877805
]
Hadoop QA commented on HBASE-17676:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 27s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 106m 53s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 151m 31s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.TestIOFencing |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853885/HBASE-17676.v2.patch |
| JIRA Issue | HBASE-17676 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux bb9bb03f2640 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c579cf6 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5793/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5793/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5793/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5793/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Get class name once for
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877963#comment-15877963
]
Hudson commented on HBASE-17561:
FAILURE: Integrated in Jenkins build HBase-1.4 #639 (See
[https://builds.apache.org/job/HBase-1.4/639/])
HBASE-17561 table status page should escape values that may contain (busbey:
rev a404bfa0c2103c68ea0294c8e1bd3c1df0f79d8b)
* (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-13718) Add a pretty printed table description to the table detail page of HBase's master
[ https://issues.apache.org/jira/browse/HBASE-13718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877962#comment-15877962 ] Hudson commented on HBASE-13718: FAILURE: Integrated in Jenkins build HBase-1.4 #639 (See [https://builds.apache.org/job/HBase-1.4/639/]) HBASE-13718 added columns schema to table description in web view (busbey: rev e7efa23d0747755b4539a65a39d66764150fc74e) * (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp > Add a pretty printed table description to the table detail page of HBase's > master > - > > Key: HBASE-13718 > URL: https://issues.apache.org/jira/browse/HBASE-13718 > Project: HBase > Issue Type: Improvement > Components: hbase >Affects Versions: 2.0.0 >Reporter: Joao Girao >Assignee: Joao Girao >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: D38649.diff, D38649.diff.txt, Screen Shot 2015-05-18 at > 1.57.50 PM.png > > > HBase's master has an info server that's useful for debugging and getting a > general overview of what's in the cluster. It has a page dedicated to > describing a cluster. You can reach it by going to something like: > http://localhost:54677/table.jsp?name=cluster_test > That page currently doesn't have anything about the current table schema. It > would be nice to have a table that lists the different column families and > how they are set up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17608) Add suspend support for RawScanResultConsumer
[
https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877969#comment-15877969
]
Hadoop QA commented on HBASE-17608:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 27s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 9s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 53s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 19s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 118m 24s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 40s
{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
52s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 179m 57s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853903/HBASE-17608-v5.patch |
| JIRA Issue | HBASE-17608 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux b8c3f3b43322 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c579cf6 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5794/testReport/ |
|
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877994#comment-15877994
]
Hudson commented on HBASE-17561:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #836 (See
[https://builds.apache.org/job/HBase-1.3-IT/836/])
HBASE-17561 table status page should escape values that may contain (busbey:
rev 8b9455cd586728e71080da8804b0c0824d00cb2f)
* (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky updated HBASE-17662: Attachment: HBASE-17662-V04.patch > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17676) Get class name once for all in AbstractFSWAL
[
https://issues.apache.org/jira/browse/HBASE-17676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878058#comment-15878058
]
Hudson commented on HBASE-17676:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2550 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2550/])
HBASE-17676 Get class name once for all in AbstractFSWAL (liyu: rev
93e60153b0cef83ff6dafc03d771c9d41a23dc3a)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
> Get class name once for all in AbstractFSWAL
>
>
> Key: HBASE-17676
> URL: https://issues.apache.org/jira/browse/HBASE-17676
> Project: HBase
> Issue Type: Improvement
> Components: Performance
>Affects Versions: 2.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0
>
> Attachments: HBASE-17676.patch, HBASE-17676.v2.patch,
> HBASE-17676.v3.patch
>
>
> While verifying HBASE-17471 with high write workload, observed several
> handler thread at getting class name in jstack, as shown below:
> {noformat}
> "B.defaultRpcServer.handler=60,queue=3,port=16020" daemon prio=10
> tid=0x7f0673835800 nid=0x4dec runnable [0x7f06721b5000]
>java.lang.Thread.State: RUNNABLE
> at java.lang.Class.getEnclosingMethod0(Native Method)
> at java.lang.Class.getEnclosingMethodInfo(Class.java:964)
> at java.lang.Class.getEnclosingClass(Class.java:1137)
> at java.lang.Class.getSimpleBinaryName(Class.java:1282)
> at java.lang.Class.getSimpleName(Class.java:1174)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.stampSequenceIdAndPublishToRingBuffer(FSHLog.java:1251)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1238)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3173)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2874)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:823)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:785)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2259)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:848)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:756)
> {noformat}
> We could get the class name in constructor and use it for all places needed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17676) Get class name once for all in AbstractFSWAL
[
https://issues.apache.org/jira/browse/HBASE-17676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877948#comment-15877948
]
Hadoop QA commented on HBASE-17676:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 4s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 116m 22s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 160m 54s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853905/HBASE-17676.v3.patch |
| JIRA Issue | HBASE-17676 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux ffe6b7678124 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / c579cf6 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5795/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5795/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Get class name once for all in AbstractFSWAL
>
>
> Key: HBASE-17676
> URL: https://issues.apache.org/jira/browse/HBASE-17676
> Project: HBase
> Issue Type: Improvement
> Components: Performance
>Affects Versions: 2.0
>Reporter: Yu Li
>
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877904#comment-15877904
]
Hudson commented on HBASE-17561:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #105 (See
[https://builds.apache.org/job/HBase-1.2-JDK7/105/])
HBASE-17561 table status page should escape values that may contain (busbey:
rev 8b9455cd586728e71080da8804b0c0824d00cb2f)
* (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877979#comment-15877979 ] Anastasia Braginsky commented on HBASE-17343: - Hey [~anoop.hbase], [~ram_krish], [~stack], [~ebortnik], While working on HBASE-17662, I ran the full regression with BASIC Compacting Memstore as a default, and all the tests passed successfully. What else do you think is needed in order to change the default? > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878119#comment-15878119
]
Zheng Hu commented on HBASE-17672:
--
[~ted_yu] , Thanks for your reply. After re-check the code, I found that
permission RWXCA will be granted to new created table for the table owner. The
logic is in AccessController.postCompletedCreateTableAction() .
In HBASE-17472, We redefine grant semantic : later granted permissions will
merge with previous granted permissions. (see release notes for more details).
For failed ruby UT, user has RWXCA permissions when the user created a new
table, and if we grant RXCA to user for the specific table, user will has
permission (RWXCA + RXCA =) RWXCA, so the user have write permission
finally. ( + means merge behavior) .
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17608) Add suspend support for RawScanResultConsumer
[ https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17608: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master. Thanks [~stack] for reviewing. > Add suspend support for RawScanResultConsumer > - > > Key: HBASE-17608 > URL: https://issues.apache.org/jira/browse/HBASE-17608 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17608.patch, HBASE-17608-v1.patch, > HBASE-17608-v2.patch, HBASE-17608-v3.patch, HBASE-17608-v4.patch, > HBASE-17608-v5.patch > > > Now for the AsyncResultScanner, we can only close the scanner if we reach the > cache size limit and open a new scanner later. This will breaks the region > level consistency. > For example, you put 10 rows to the region, and open a scanner to scan it. > The scanner returns 5 rows at the first time and the cache is full, so it > closes the background scanner. And before you reopens the scanner to fetch > data, the remaining 5 rows has been deleted and a compaction makes them gone > for ever. Then after you reopen the scanner you can not see the remaining 5 > rows. > So here we should keep the scanner open at RS side to prevent the data below > the mvcc read point of this scanner being deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878191#comment-15878191
]
Zheng Hu commented on HBASE-17672:
--
In my mind, Both "the second user" and "the table owner" are the same,
because , we use the user org.apache.hadoop.hbase.security.User.getCurrent()
to grant , not other created user.
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878102#comment-15878102
]
Hudson commented on HBASE-17561:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #110 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/110/])
HBASE-17561 table status page should escape values that may contain (busbey:
rev de3177caaeaab96d47b236c4c35a2e143fba1563)
* (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17595) Add partial result support for small/limited scan
[ https://issues.apache.org/jira/browse/HBASE-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17595: -- Attachment: HBASE-17595.patch > Add partial result support for small/limited scan > - > > Key: HBASE-17595 > URL: https://issues.apache.org/jira/browse/HBASE-17595 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17595.patch > > > The partial result support is marked as a 'TODO' when implementing > HBASE-17045. And when implementing HBASE-17508, we found that if we make > small scan share the same logic with general scan, the scan request other > than open scanner will not have the small flag so the server may return > partial result to the client and cause some strange behavior. It is solved by > modifying the logic at server side, but this means the 1.4.x client is not > safe to contact with earlier 1.x server. So we'd better address the problem > at client side. Marked as blocker as this issue should be finished before any > 2.x and 1.4.x releases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17595) Add partial result support for small/limited scan
[ https://issues.apache.org/jira/browse/HBASE-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17595: -- Status: Patch Available (was: Open) Let's try the patch. > Add partial result support for small/limited scan > - > > Key: HBASE-17595 > URL: https://issues.apache.org/jira/browse/HBASE-17595 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17595.patch > > > The partial result support is marked as a 'TODO' when implementing > HBASE-17045. And when implementing HBASE-17508, we found that if we make > small scan share the same logic with general scan, the scan request other > than open scanner will not have the small flag so the server may return > partial result to the client and cause some strange behavior. It is solved by > modifying the logic at server side, but this means the 1.4.x client is not > safe to contact with earlier 1.x server. So we'd better address the problem > at client side. Marked as blocker as this issue should be finished before any > 2.x and 1.4.x releases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-16990:
-
Attachment: HBASE-16990.v2.patch
Upload patch v2:
1. refactor grants_dumper.rb to permission_dumper.rb , as [~tedyu] said,
grant is a verb.
2. handle exception with only prints rootCause.
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, HBASE-16990.v2.patch,
> hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17608) Add suspend support for RawScanResultConsumer
[ https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878104#comment-15878104 ] Duo Zhang commented on HBASE-17608: --- Will commit shortly. > Add suspend support for RawScanResultConsumer > - > > Key: HBASE-17608 > URL: https://issues.apache.org/jira/browse/HBASE-17608 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17608.patch, HBASE-17608-v1.patch, > HBASE-17608-v2.patch, HBASE-17608-v3.patch, HBASE-17608-v4.patch, > HBASE-17608-v5.patch > > > Now for the AsyncResultScanner, we can only close the scanner if we reach the > cache size limit and open a new scanner later. This will breaks the region > level consistency. > For example, you put 10 rows to the region, and open a scanner to scan it. > The scanner returns 5 rows at the first time and the cache is full, so it > closes the background scanner. And before you reopens the scanner to fetch > data, the remaining 5 rows has been deleted and a compaction makes them gone > for ever. Then after you reopen the scanner you can not see the remaining 5 > rows. > So here we should keep the scanner open at RS side to prevent the data below > the mvcc read point of this scanner being deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878169#comment-15878169
]
Ted Yu commented on HBASE-17672:
Why would the second user inherit write permission from table owner ?
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878131#comment-15878131
]
Zheng Hu commented on HBASE-16990:
--
[~esteban] , Thanks for your reply.
> I think it would be better if we could merge this patch and
> ./bin/replication/copy_tables_desc.rb into a single tool but that can be
> handled in a different JIRA.
Good idea, maybe I'll try to merge this two.
> could you also add better exception handling and verify that the user that is
> running this command has permissions to run
> AccessControlClient.getUserPermissions?.
Thanks for your advise, Will fix it.
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878204#comment-15878204
]
Hudson commented on HBASE-17561:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #98 (See
[https://builds.apache.org/job/HBase-1.2-JDK8/98/])
HBASE-17561 table status page should escape values that may contain (busbey:
rev 8b9455cd586728e71080da8804b0c0824d00cb2f)
* (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878241#comment-15878241
]
Ted Yu commented on HBASE-17672:
Can you modify the test so that the second user does the grant ?
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-16990:
-
Attachment: HBASE-16990.v2.patch
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, HBASE-16990.v2.patch,
> hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-16990:
-
Attachment: (was: HBASE-16990.v2.patch)
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[ https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878626#comment-15878626 ] Anoop Sam John commented on HBASE-17647: Thanks for the review Stack.. Now we have ByteBufferKeyValue which can be backed by both on heap as well as off heap BB. So the change in the heapSize calc. Ya I will rename the params and method as needed.. Changed some places and missed some may be. Ya old 'size' is what dataSize now. dataSize might be in on heap or off heap.. heapSize is tracks total heap space occupied by all cells. This includes cell POJO heap overheads and if a cell is on heap, the total length of this cells key + value which resides in on heap area any way. > OffheapKeyValue#heapSize() implementation is wrong > -- > > Key: HBASE-17647 > URL: https://issues.apache.org/jira/browse/HBASE-17647 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-17647.patch, HBASE-17647_V2.patch > > > We consider the key and data lengths also even though the data is actually in > off heap area. We should correct it. > The impact will be at ScannerContext limit tracking where we use heapSize of > cells to account the result size. So my proposal is to consider the cells > length and heap size in Limit tracking and accounting. We have a > maxResultSize which defaults to 2MB. When the sum of all cell's data size > reaches 'maxResultSize' OR the sum of all cell's heap size reaches > 'maxResultSize' , we need to send back the RPC response -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878646#comment-15878646
]
Hudson commented on HBASE-17561:
SUCCESS: Integrated in Jenkins build HBase-1.1-JDK7 #1847 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1847/])
HBASE-17561 table status page should escape values that may contain (busbey:
rev 0110bc9d4ef1aec67dcd8bb1919d4a9a6fefb19d)
* (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878295#comment-15878295
]
Zheng Hu commented on HBASE-17672:
--
Yeah, It's better to use a new created use for the UT. Will upload a new
patch.
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878327#comment-15878327
]
Hudson commented on HBASE-17561:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #120 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/120/])
HBASE-17561 table status page should escape values that may contain (busbey:
rev de3177caaeaab96d47b236c4c35a2e143fba1563)
* (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878434#comment-15878434 ] Anoop Sam John commented on HBASE-17343: All the tests passing , with out any change in tests, is a very good sign. Also we need to run some perf tests to prove that this wont make any perf regression at all. I mean the normal work load where there are no duplicate cells. A PE run over at least 100GB data size might be needed IMO. > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878480#comment-15878480
]
Hudson commented on HBASE-17561:
SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #1931 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1931/])
HBASE-17561 table status page should escape values that may contain (busbey:
rev 0110bc9d4ef1aec67dcd8bb1919d4a9a6fefb19d)
* (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17595) Add partial result support for small/limited scan
[
https://issues.apache.org/jira/browse/HBASE-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878504#comment-15878504
]
Hadoop QA commented on HBASE-17595:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 49s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 23s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 30s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 105m 49s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 166m 52s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestPartialResultsFromClientSide |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853957/HBASE-17595.patch |
| JIRA Issue | HBASE-17595 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 88fe23339957 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f037f23 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5797/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5797/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878459#comment-15878459
]
Hadoop QA commented on HBASE-17672:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 5s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 5s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 49s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 39s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 38s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853966/HBASE-17672.v1.patch |
| JIRA Issue | HBASE-17672 |
| Optional Tests | asflicense unit rubocop ruby_lint |
| uname | Linux 032063d6a0ed 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f037f23 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5801/testReport/ |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5801/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch, HBASE-17672.v1.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17608) Add suspend support for RawScanResultConsumer
[ https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878511#comment-15878511 ] Hudson commented on HBASE-17608: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2551 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2551/]) HBASE-17608 Add suspend support for RawScanResultConsumer (zhangduo: rev f037f230fd5a0b6f28e68b02f47efeb4dbc22694) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRawAsyncTableScan.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClientScanner.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncScanSingleRegionRpcRetryingCaller.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableScanRenewLease.java * (edit) hbase-endpoint/src/main/java/org/apache/hadoop/hbase/client/coprocessor/AsyncAggregationClient.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncTable.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCallerFactory.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncTableImpl.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTableResultScanner.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawScanResultConsumer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableScanner.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/Threads.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableScannerCloseWhileSuspending.java > Add suspend support for RawScanResultConsumer > - > > Key: HBASE-17608 > URL: https://issues.apache.org/jira/browse/HBASE-17608 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17608.patch, HBASE-17608-v1.patch, > HBASE-17608-v2.patch, HBASE-17608-v3.patch, HBASE-17608-v4.patch, > HBASE-17608-v5.patch > > > Now for the AsyncResultScanner, we can only close the scanner if we reach the > cache size limit and open a new scanner later. This will breaks the region > level consistency. > For example, you put 10 rows to the region, and open a scanner to scan it. > The scanner returns 5 rows at the first time and the cache is full, so it > closes the background scanner. And before you reopens the scanner to fetch > data, the remaining 5 rows has been deleted and a compaction makes them gone > for ever. Then after you reopen the scanner you can not see the remaining 5 > rows. > So here we should keep the scanner open at RS side to prevent the data below > the mvcc read point of this scanner being deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17677) ServerName parsing from directory name should be more robust to errors from guava's HostAndPort
[ https://issues.apache.org/jira/browse/HBASE-17677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879127#comment-15879127 ] stack commented on HBASE-17677: --- +1 Thanks [~busbey] Good test. > ServerName parsing from directory name should be more robust to errors from > guava's HostAndPort > --- > > Key: HBASE-17677 > URL: https://issues.apache.org/jira/browse/HBASE-17677 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0 > > Attachments: HBASE-17677.1.patch > > > our internal Address facade over HostAndPort directly passes on any failures > from the underlying Guava implementation. Right now when we parse a > ServerName from a WAL directory we properly handle if guava gives us an > IllegalArgumentException but we do not handle if it gives us a > IllegalStateException. e.g. it uses the former for "I couldn't parse anything > out of this string" and the latter for "I parsed a host name but didn't see a > port". -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878957#comment-15878957
]
Jerry He commented on HBASE-16990:
--
The shell command 'user_permission' does not work for you? Why have a separate
rb which we both need to maintain?
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, HBASE-16990.v2.patch,
> hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17678) ColumnPaginationFilter in a FilterList gives different results when using MUST_PASS_ONE vs MUST_PASS_ALL and a cell has multiple values for a given timestamp
[
https://issues.apache.org/jira/browse/HBASE-17678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878960#comment-15878960
]
Jason Tokayer commented on HBASE-17678:
---
The flush leaves only a single value for a timestamp. So yes, it seems like
this is only valid for data within the memstore.
> ColumnPaginationFilter in a FilterList gives different results when using
> MUST_PASS_ONE vs MUST_PASS_ALL and a cell has multiple values for a given
> timestamp
> -
>
> Key: HBASE-17678
> URL: https://issues.apache.org/jira/browse/HBASE-17678
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.3.0, 1.2.1
> Environment: RedHat 7.x
>Reporter: Jason Tokayer
>
> When combining ColumnPaginationFilter with a single-element filterList,
> MUST_PASS_ONE and MUST_PASS_ALL give different results when there are
> multiple cells with the same timestamp. This is unexpected since there is
> only a single filter in the list, and I would believe that MUST_PASS_ALL and
> MUST_PASS_ONE should only affect the behavior of the joined filter and not
> the behavior of any one of the individual filters. If this is not a bug then
> it would be nice if the documentation is updated to explain this nuanced
> behavior.
> I know that there was a decision made in an earlier Hbase version to keep
> multiple cells with the same timestamp. This is generally fine but presents
> an issue when using the aforementioned filter combination.
> Steps to reproduce:
> In the shell create a table and insert some data:
> {code:none}
> create 'ns:tbl',{NAME => 'family',VERSIONS => 100}
> put 'ns:tbl','row','family:name','John',1
> put 'ns:tbl','row','family:name','Jane',1
> put 'ns:tbl','row','family:name','Gil',1
> put 'ns:tbl','row','family:name','Jane',1
> {code}
> Then, use a Scala client as:
> {code:none}
> import org.apache.hadoop.hbase.filter._
> import org.apache.hadoop.hbase.util.Bytes
> import org.apache.hadoop.hbase.client._
> import org.apache.hadoop.hbase.{CellUtil, HBaseConfiguration, TableName}
> import scala.collection.mutable._
> val config = HBaseConfiguration.create()
> config.set("hbase.zookeeper.quorum", "localhost")
> config.set("hbase.zookeeper.property.clientPort", "2181")
> val connection = ConnectionFactory.createConnection(config)
> val logicalOp = FilterList.Operator.MUST_PASS_ONE
> val limit = 1
> var resultsList = ListBuffer[String]()
> for (offset <- 0 to 20 by limit) {
> val table = connection.getTable(TableName.valueOf("ns:tbl"))
> val paginationFilter = new ColumnPaginationFilter(limit,offset)
> val filterList: FilterList = new FilterList(logicalOp,paginationFilter)
> println("@ filterList = "+filterList)
> val results = table.get(new
> Get(Bytes.toBytes("row")).setFilter(filterList))
> val cells = results.rawCells()
> if (cells != null) {
> for (cell <- cells) {
> val value = new String(CellUtil.cloneValue(cell))
> val qualifier = new String(CellUtil.cloneQualifier(cell))
> val family = new String(CellUtil.cloneFamily(cell))
> val result = "OFFSET = "+offset+":"+family + "," + qualifier
> + "," + value + "," + cell.getTimestamp()
> resultsList.append(result)
> }
> }
> }
> resultsList.foreach(println)
> {code}
> Here are the results for different limit and logicalOp settings:
> {code:none}
> Limit = 1 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 1 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 1:family,name,Gil,1
> OFFSET = 2:family,name,Jane,1
> OFFSET = 3:family,name,John,1
> Limit = 2 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 2 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 2:family,name,Jane,1
> {code}
> So, it seems that MUST_PASS_ALL gives the expected behavior, but
> MUST_PASS_ONE does not. Furthermore, MUST_PASS_ONE seems to give only a
> single (not-duplicated) within a page, but not across pages.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879085#comment-15879085
]
stack commented on HBASE-17069:
---
Thanks [~abhishek.chouhan] I am enjoying reading your debug journeys. A
miscounting of sequenceid
bq. The mutations are not empty, however we still were going the route of
appending with inMemstore as false.
Am I reading in the wrong place, because seems like mutations have to be zero
to go the 'false' route.
Its called PONR because along w/ the writes to hbase:meta, we send a prayer
hoping we get to the next cleanup step (The new AM should help here).
bq. Now during the HRegionServer.postOpenDeployTasks we use
MetaTableAccessor.updateRegionLocation() which doesn't have the hregioninfo
I wonder why the hregioninfo previously written doesn't 'shine through'... is
the update of location writing null into the hregioninfo cell or is the read
not allowing old versions of the hregioninfo cell in hbase:meta?
bq. I think we should handle mergingnew the same way as splittingnew.
Yes. The states are myriad. There will be holes and takes someone of your
diligence to identify them.
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960,
[jira] [Updated] (HBASE-17677) ServerName parsing from directory name should be more robust to errors from guava's HostAndPort
[ https://issues.apache.org/jira/browse/HBASE-17677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-17677: Status: Patch Available (was: In Progress) > ServerName parsing from directory name should be more robust to errors from > guava's HostAndPort > --- > > Key: HBASE-17677 > URL: https://issues.apache.org/jira/browse/HBASE-17677 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0 > > Attachments: HBASE-17677.1.patch > > > our internal Address facade over HostAndPort directly passes on any failures > from the underlying Guava implementation. Right now when we parse a > ServerName from a WAL directory we properly handle if guava gives us an > IllegalArgumentException but we do not handle if it gives us a > IllegalStateException. e.g. it uses the former for "I couldn't parse anything > out of this string" and the latter for "I parsed a host name but didn't see a > port". -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[ https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879145#comment-15879145 ] stack commented on HBASE-17647: --- bq. Ya old 'size' is what dataSize now. dataSize might be in on heap or off heap.. heapSize is tracks total heap space occupied by all cells. This includes cell POJO heap overheads and if a cell is on heap, the total length of this cells key + value which resides in on heap area any way. Add this in as a comment on commit. +1 > OffheapKeyValue#heapSize() implementation is wrong > -- > > Key: HBASE-17647 > URL: https://issues.apache.org/jira/browse/HBASE-17647 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-17647.patch, HBASE-17647_V2.patch, > HBASE-17647_V3.patch > > > We consider the key and data lengths also even though the data is actually in > off heap area. We should correct it. > The impact will be at ScannerContext limit tracking where we use heapSize of > cells to account the result size. So my proposal is to consider the cells > length and heap size in Limit tracking and accounting. We have a > maxResultSize which defaults to 2MB. When the sum of all cell's data size > reaches 'maxResultSize' OR the sum of all cell's heap size reaches > 'maxResultSize' , we need to send back the RPC response -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17677) ServerName parsing from directory name should be more robust to errors from guava's HostAndPort
[ https://issues.apache.org/jira/browse/HBASE-17677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-17677: Attachment: HBASE-17677.1.patch -01 - add tests for sunny day wal path parsing and a test with stand-alone test paths we've used - add handling of the other way ServerName parsing can fail due to guava > ServerName parsing from directory name should be more robust to errors from > guava's HostAndPort > --- > > Key: HBASE-17677 > URL: https://issues.apache.org/jira/browse/HBASE-17677 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0 > > Attachments: HBASE-17677.1.patch > > > our internal Address facade over HostAndPort directly passes on any failures > from the underlying Guava implementation. Right now when we parse a > ServerName from a WAL directory we properly handle if guava gives us an > IllegalArgumentException but we do not handle if it gives us a > IllegalStateException. e.g. it uses the former for "I couldn't parse anything > out of this string" and the latter for "I parsed a host name but didn't see a > port". -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17312) [JDK8] Use default method for Observer Coprocessors
[
https://issues.apache.org/jira/browse/HBASE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879265#comment-15879265
]
stack commented on HBASE-17312:
---
+1 This is excellent cleanup. A few small things in rb that can go in on commit.
> [JDK8] Use default method for Observer Coprocessors
> ---
>
> Key: HBASE-17312
> URL: https://issues.apache.org/jira/browse/HBASE-17312
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Appy
> Labels: incompatible
> Attachments: HBASE-17312.master.001.patch,
> HBASE-17312.master.001.patch, HBASE-17312.master.002.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design doesn't
> permit extending even that single observer (see RSGroupAdminEndpoint), that
> leads to full copy of Base...Observer class into coprocessor class leading to
> 1000s of lines of code and this ugly mix of 5 main functions with 100 useless
> functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879300#comment-15879300
]
Abhishek Singh Chouhan commented on HBASE-17069:
[~stack] I'm still a bit new to the code. Hope i'm not misunderstanding the
bits.
bq. mutations have to be zero to go the 'false' route
>From what i understood we enter the below block only when the walEdit is not
>empty and we have some cells in the wal edit from the processed mutations.
>Hope i'm not terribly wrong here.
In Hregion.processRowsWithLocks
if (!walEdit.isEmpty()) {
// we use HLogKey here instead of WALKey directly to support legacy
coprocessors.
walKey = new HLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
this.htableDescriptor.getTableName(), WALKey.NO_SEQUENCE_ID, now,
processor.getClusterIds(), nonceGroup, nonce, mvcc);
txid = this.wal.append(this.htableDescriptor, this.getRegionInfo(),
walKey, walEdit, false); //this was false before the patch
}
bq. I wonder why the hregioninfo previously written doesn't 'shine through'...
is the update of location writing null into the hregioninfo cell or is the read
not allowing old versions of the hregioninfo cell in hbase:meta?
During the merge we send a multi that deletes the rows which had the
hregioninfo and never add it back.
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
>
[jira] [Updated] (HBASE-16942) Add FavoredStochasticLoadBalancer and FN Candidate generators
[ https://issues.apache.org/jira/browse/HBASE-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan updated HBASE-16942: - Summary: Add FavoredStochasticLoadBalancer and FN Candidate generators (was: FavoredNodes - Balancer improvements) > Add FavoredStochasticLoadBalancer and FN Candidate generators > - > > Key: HBASE-16942 > URL: https://issues.apache.org/jira/browse/HBASE-16942 > Project: HBase > Issue Type: Sub-task > Components: FavoredNodes >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan > Fix For: 2.0.0 > > Attachments: HBASE-16942.master.001.patch, > HBASE-16942.master.002.patch, HBASE-16942.master.003.patch, > HBASE-16942.master.004.patch, HBASE_16942_rough_draft.patch > > > This deals with the balancer based enhancements to favored nodes patch as > discussed in HBASE-15532. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879421#comment-15879421
]
stack commented on HBASE-17069:
---
bq. From what i understood we enter the below block only when the walEdit is
not empty and we have some cells in the wal edit from the processed mutations.
You are right. I was wrong (misreading). It was probably set to 'false' because
we haven't added the append to memstore yet... that happens next... which seems
reasonable.
bq. In SequenceIdAccounting.update() we pass false for the multirequest (lets
say sequence id here was 1) so lowestunflusedsequenceid is not updated.
The edit is in WAL, perhaps unsynced, but not in memstore (yet); a limbo.
Another limbo is the gap while we wait for an edit to be stamped with a
sequenceid (HBASE-17471 and friends help here)
bq. Now for the second put that goes through doMiniBatchMutation we pass true
correctly during append(Seq id 2). lowestUnflushedSequenceIds is set to 2 for
the metafamily.
Issue is we have two ways of writing (smile) and they don't agree which is
especially fun in times of high concurrency where behind the scenes we are
trying to keep an incrementing counter that aligns with order in which edits
arrive.
Setting true as the patch does seems like a low-risk way of keeping stuff
aligned in branch-1.2/1.3. We should do a follow-on for branch-1 to correct the
fuzzyness around inmemory flag and ensure that the rare edit that is not meant
for memstore doesn't cause an issue similar to here because it has us skip out
on sequenceid accounting.
Thanks [~abhishek.chouhan]
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
>
[jira] [Updated] (HBASE-17680) Run mini cluster through JNI in tests
[ https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17680: --- Description: Currently tests start local hbase cluster through hbase shell. There is less control over the configuration of the local cluster this way. This issue would replace hbase shell with JNI interface to mini cluster. We would have full control over the cluster behavior. Thanks to [~devaraj] who started this initiative. was: Currently tests start local hbase cluster through hbase shell. There is less control over the configuration of the local cluster this way. This issue would replace hbase shell with JNI interface to mini cluster. We would have full control over the cluster behavior. Thanks to [~ddas] who started this initiative. > Run mini cluster through JNI in tests > - > > Key: HBASE-17680 > URL: https://issues.apache.org/jira/browse/HBASE-17680 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > > Currently tests start local hbase cluster through hbase shell. > There is less control over the configuration of the local cluster this way. > This issue would replace hbase shell with JNI interface to mini cluster. > We would have full control over the cluster behavior. > Thanks to [~devaraj] who started this initiative. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17680) Run mini cluster through JNI in tests
Ted Yu created HBASE-17680: -- Summary: Run mini cluster through JNI in tests Key: HBASE-17680 URL: https://issues.apache.org/jira/browse/HBASE-17680 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu Currently tests start local hbase cluster through hbase shell. There is less control over the configuration of the local cluster this way. This issue would replace hbase shell with JNI interface to mini cluster. We would have full control over the cluster behavior. Thanks to [~ddas] who started this initiative. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17582) Drop page cache hint is broken
[ https://issues.apache.org/jira/browse/HBASE-17582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879392#comment-15879392 ] Gary Helmling commented on HBASE-17582: --- +1 on the patch. > Drop page cache hint is broken > -- > > Key: HBASE-17582 > URL: https://issues.apache.org/jira/browse/HBASE-17582 > Project: HBase > Issue Type: Bug > Components: Compaction, io >Affects Versions: 2.0.0 >Reporter: Ashu Pachauri >Assignee: Appy >Priority: Critical > Attachments: HBASE-17582.master.001.patch > > > We pass a boolean for pass page cache drop hint while creating store file > scanners and writers. > The hint is not passed on down the stack by StoreFileWriter and > StoreFileScanner in the master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16942) FavoredNodes - Balancer improvements
[ https://issues.apache.org/jira/browse/HBASE-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan updated HBASE-16942: - Attachment: HBASE-16942.master.004.patch > FavoredNodes - Balancer improvements > > > Key: HBASE-16942 > URL: https://issues.apache.org/jira/browse/HBASE-16942 > Project: HBase > Issue Type: Sub-task > Components: FavoredNodes >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan > Fix For: 2.0.0 > > Attachments: HBASE-16942.master.001.patch, > HBASE-16942.master.002.patch, HBASE-16942.master.003.patch, > HBASE-16942.master.004.patch, HBASE_16942_rough_draft.patch > > > This deals with the balancer based enhancements to favored nodes patch as > discussed in HBASE-15532. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17590) Drop cache hint should work for StoreFile write path
[
https://issues.apache.org/jira/browse/HBASE-17590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-17590:
--
Resolution: Fixed
Fix Version/s: 1.3.1
1.4.0
2.0.0
Status: Resolved (was: Patch Available)
Committed to branch-1.3+.
> Drop cache hint should work for StoreFile write path
>
>
> Key: HBASE-17590
> URL: https://issues.apache.org/jira/browse/HBASE-17590
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Ashu Pachauri
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: HBASE-17590.master.001.patch
>
>
> We have this in the code right now.
> {noformat}
> public Builder withShouldDropCacheBehind(boolean
> shouldDropCacheBehind/*NOT USED!!*/) {
> // TODO: HAS NO EFFECT!!! FIX!!
> return this;
> }
> {noformat}
> Creating jira to track it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Attachment: (was: HBASE-16188.patch)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16188-branch-1.3.patch, HBASE-16188.master.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Fix Version/s: 1.3.0
Status: Patch Available (was: In Progress)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16188-branch-1.3.patch, HBASE-16188.master.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Work started] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-16188 started by Gopi Krishnan Nambiar.
-
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16188-branch-1.3.patch, HBASE-16188.master.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Attachment: HBASE-16188.master.patch
HBASE-16188-branch-1.3.patch
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16188-branch-1.3.patch, HBASE-16188.master.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17312) [JDK8] Use default method for Observer Coprocessors
[
https://issues.apache.org/jira/browse/HBASE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879288#comment-15879288
]
Appy commented on HBASE-17312:
--
Thanks [~stack] for the review. Replied to comments in RB.
Waiting today if anyone else might want to review it too. Otherwise, will
commit tomorrow.
> [JDK8] Use default method for Observer Coprocessors
> ---
>
> Key: HBASE-17312
> URL: https://issues.apache.org/jira/browse/HBASE-17312
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Appy
> Labels: incompatible
> Attachments: HBASE-17312.master.001.patch,
> HBASE-17312.master.001.patch, HBASE-17312.master.002.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design doesn't
> permit extending even that single observer (see RSGroupAdminEndpoint), that
> leads to full copy of Base...Observer class into coprocessor class leading to
> 1000s of lines of code and this ugly mix of 5 main functions with 100 useless
> functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17677) ServerName parsing from directory name should be more robust to errors from guava's HostAndPort
[
https://issues.apache.org/jira/browse/HBASE-17677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879374#comment-15879374
]
Hadoop QA commented on HBASE-17677:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 7s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 98m 44s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 138m 30s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12854051/HBASE-17677.1.patch |
| JIRA Issue | HBASE-17677 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux f0302af6a0bb 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f037f23 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5803/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5803/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> ServerName parsing from directory name should be more robust to errors from
> guava's HostAndPort
> ---
>
> Key: HBASE-17677
> URL: https://issues.apache.org/jira/browse/HBASE-17677
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>
[jira] [Created] (HBASE-17679) Log arguments passed to hbck
Esteban Gutierrez created HBASE-17679: - Summary: Log arguments passed to hbck Key: HBASE-17679 URL: https://issues.apache.org/jira/browse/HBASE-17679 Project: HBase Issue Type: Bug Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial Sometimes hbck arguments get lost and we only end up with the output of hbck. This should log some basic info about our arguments passed to hbck for better supportability. Additional server side logging will be added later on HBase Admin calls in a different JIRA. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Status: Open (was: Patch Available)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-16188.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879300#comment-15879300
]
Abhishek Singh Chouhan edited comment on HBASE-17069 at 2/22/17 9:42 PM:
-
[~stack] I'm still a bit new to the code. Hope i'm not misunderstanding the
bits :).
bq. mutations have to be zero to go the 'false' route
>From what i understood we enter the below block only when the walEdit is not
>empty and we have some cells in the wal edit from the processed mutations.
>Hope i'm not terribly wrong here.
In Hregion.processRowsWithLocks
if (!walEdit.isEmpty()) {
// we use HLogKey here instead of WALKey directly to support legacy
coprocessors.
walKey = new HLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
this.htableDescriptor.getTableName(), WALKey.NO_SEQUENCE_ID, now,
processor.getClusterIds(), nonceGroup, nonce, mvcc);
txid = this.wal.append(this.htableDescriptor, this.getRegionInfo(),
walKey, walEdit, false); //this was false before the patch
}
bq. I wonder why the hregioninfo previously written doesn't 'shine through'...
is the update of location writing null into the hregioninfo cell or is the read
not allowing old versions of the hregioninfo cell in hbase:meta?
During the merge we send a multi that deletes the rows which had the
hregioninfo and never add it back.
was (Author: abhishek.chouhan):
[~stack] I'm still a bit new to the code. Hope i'm not misunderstanding the
bits.
bq. mutations have to be zero to go the 'false' route
>From what i understood we enter the below block only when the walEdit is not
>empty and we have some cells in the wal edit from the processed mutations.
>Hope i'm not terribly wrong here.
In Hregion.processRowsWithLocks
if (!walEdit.isEmpty()) {
// we use HLogKey here instead of WALKey directly to support legacy
coprocessors.
walKey = new HLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
this.htableDescriptor.getTableName(), WALKey.NO_SEQUENCE_ID, now,
processor.getClusterIds(), nonceGroup, nonce, mvcc);
txid = this.wal.append(this.htableDescriptor, this.getRegionInfo(),
walKey, walEdit, false); //this was false before the patch
}
bq. I wonder why the hregioninfo previously written doesn't 'shine through'...
is the update of location writing null into the hregioninfo cell or is the read
not allowing old versions of the hregioninfo cell in hbase:meta?
During the merge we send a multi that deletes the rows which had the
hregioninfo and never add it back.
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
>
[jira] [Updated] (HBASE-17680) Run mini cluster through JNI in tests
[ https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17680: --- Attachment: 17680.v1.txt > Run mini cluster through JNI in tests > - > > Key: HBASE-17680 > URL: https://issues.apache.org/jira/browse/HBASE-17680 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17680.v1.txt > > > Currently tests start local hbase cluster through hbase shell. > There is less control over the configuration of the local cluster this way. > This issue would replace hbase shell with JNI interface to mini cluster. > We would have full control over the cluster behavior. > Thanks to [~devaraj] who started this initiative. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879002#comment-15879002
]
Ted Yu commented on HBASE-17672:
{code}
+ assert(found_permission, "Permission for user test_grant_revoke does not
found.")
{code}
"does not found." -> "was not found."
{code}
+ security_admin.grant(test_grant_revoke_user,"W", @test_name)
{code}
The original test didn't include write permission. Is there reason for granting
test_grant_revoke_user write permission ?
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch, HBASE-17672.v1.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17678) ColumnPaginationFilter in a FilterList gives different results when using MUST_PASS_ONE vs MUST_PASS_ALL and a cell has multiple values for a given timestamp
[
https://issues.apache.org/jira/browse/HBASE-17678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878895#comment-15878895
]
Jean-Marc Spaggiari commented on HBASE-17678:
-
I'm currious here.
If you flush and compact, one one value for the same cell with the same TS will
remained, right? So this is valid only for data within the memstore?
> ColumnPaginationFilter in a FilterList gives different results when using
> MUST_PASS_ONE vs MUST_PASS_ALL and a cell has multiple values for a given
> timestamp
> -
>
> Key: HBASE-17678
> URL: https://issues.apache.org/jira/browse/HBASE-17678
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.3.0, 1.2.1
> Environment: RedHat 7.x
>Reporter: Jason Tokayer
>
> When combining ColumnPaginationFilter with a single-element filterList,
> MUST_PASS_ONE and MUST_PASS_ALL give different results when there are
> multiple cells with the same timestamp. This is unexpected since there is
> only a single filter in the list, and I would believe that MUST_PASS_ALL and
> MUST_PASS_ONE should only affect the behavior of the joined filter and not
> the behavior of any one of the individual filters. If this is not a bug then
> it would be nice if the documentation is updated to explain this nuanced
> behavior.
> I know that there was a decision made in an earlier Hbase version to keep
> multiple cells with the same timestamp. This is generally fine but presents
> an issue when using the aforementioned filter combination.
> Steps to reproduce:
> In the shell create a table and insert some data:
> {code:none}
> create 'ns:tbl',{NAME => 'family',VERSIONS => 100}
> put 'ns:tbl','row','family:name','John',1
> put 'ns:tbl','row','family:name','Jane',1
> put 'ns:tbl','row','family:name','Gil',1
> put 'ns:tbl','row','family:name','Jane',1
> {code}
> Then, use a Scala client as:
> {code:none}
> import org.apache.hadoop.hbase.filter._
> import org.apache.hadoop.hbase.util.Bytes
> import org.apache.hadoop.hbase.client._
> import org.apache.hadoop.hbase.{CellUtil, HBaseConfiguration, TableName}
> import scala.collection.mutable._
> val config = HBaseConfiguration.create()
> config.set("hbase.zookeeper.quorum", "localhost")
> config.set("hbase.zookeeper.property.clientPort", "2181")
> val connection = ConnectionFactory.createConnection(config)
> val logicalOp = FilterList.Operator.MUST_PASS_ONE
> val limit = 1
> var resultsList = ListBuffer[String]()
> for (offset <- 0 to 20 by limit) {
> val table = connection.getTable(TableName.valueOf("ns:tbl"))
> val paginationFilter = new ColumnPaginationFilter(limit,offset)
> val filterList: FilterList = new FilterList(logicalOp,paginationFilter)
> println("@ filterList = "+filterList)
> val results = table.get(new
> Get(Bytes.toBytes("row")).setFilter(filterList))
> val cells = results.rawCells()
> if (cells != null) {
> for (cell <- cells) {
> val value = new String(CellUtil.cloneValue(cell))
> val qualifier = new String(CellUtil.cloneQualifier(cell))
> val family = new String(CellUtil.cloneFamily(cell))
> val result = "OFFSET = "+offset+":"+family + "," + qualifier
> + "," + value + "," + cell.getTimestamp()
> resultsList.append(result)
> }
> }
> }
> resultsList.foreach(println)
> {code}
> Here are the results for different limit and logicalOp settings:
> {code:none}
> Limit = 1 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 1 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 1:family,name,Gil,1
> OFFSET = 2:family,name,Jane,1
> OFFSET = 3:family,name,John,1
> Limit = 2 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 2 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 2:family,name,Jane,1
> {code}
> So, it seems that MUST_PASS_ALL gives the expected behavior, but
> MUST_PASS_ONE does not. Furthermore, MUST_PASS_ONE seems to give only a
> single (not-duplicated) within a page, but not across pages.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[
https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879046#comment-15879046
]
Hadoop QA commented on HBASE-17647:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 19s
{color} | {color:red} hbase-prefix-tree in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 13s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 1s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s
{color} | {color:green} hbase-prefix-tree in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m 23s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 146m 42s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12854001/HBASE-17647_V3.patch |
| JIRA Issue | HBASE-17647 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 0ec42ed67cd7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f037f23 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| findbugs |
[jira] [Created] (HBASE-17678) ColumnPaginationFilter in a FilterList gives different results when using MUST_PASS_ONE vs MUST_PASS_ALL and a cell has multiple values for a given timestamp
Jason Tokayer created HBASE-17678:
-
Summary: ColumnPaginationFilter in a FilterList gives different
results when using MUST_PASS_ONE vs MUST_PASS_ALL and a cell has multiple
values for a given timestamp
Key: HBASE-17678
URL: https://issues.apache.org/jira/browse/HBASE-17678
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 1.2.1, 1.3.0
Environment: RedHat 7.x
Reporter: Jason Tokayer
When combining ColumnPaginationFilter with a single-element filterList,
MUST_PASS_ONE and MUST_PASS_ALL give different results when there are multiple
cells with the same timestamp. This is unexpected since there is only a single
filter in the list, and I would believe that MUST_PASS_ALL and MUST_PASS_ONE
should only affect the behavior of the joined filter and not the behavior of
any one of the individual filters. If this is not a bug then it would be nice
if the documentation is updated to explain this nuanced behavior.
I know that there was a decision made in an earlier Hbase version to keep
multiple cells with the same timestamp. This is generally fine but presents an
issue when using the aforementioned filter combination.
Steps to reproduce:
In the shell create a table and insert some data:
{code:none}
create 'ns:tbl',{NAME => 'family',VERSIONS => 100}
put 'ns:tbl','row','family:name','John',1
put 'ns:tbl','row','family:name','Jane',1
put 'ns:tbl','row','family:name','Gil',1
put 'ns:tbl','row','family:name','Jane',1
{code}
Then, use a Scala client as:
{code:none}
import org.apache.hadoop.hbase.filter._
import org.apache.hadoop.hbase.util.Bytes
import org.apache.hadoop.hbase.client._
import org.apache.hadoop.hbase.{CellUtil, HBaseConfiguration, TableName}
import scala.collection.mutable._
val config = HBaseConfiguration.create()
config.set("hbase.zookeeper.quorum", "localhost")
config.set("hbase.zookeeper.property.clientPort", "2181")
val connection = ConnectionFactory.createConnection(config)
val logicalOp = FilterList.Operator.MUST_PASS_ONE
val limit = 1
var resultsList = ListBuffer[String]()
for (offset <- 0 to 20 by limit) {
val table = connection.getTable(TableName.valueOf("ns:tbl"))
val paginationFilter = new ColumnPaginationFilter(limit,offset)
val filterList: FilterList = new FilterList(logicalOp,paginationFilter)
println("@ filterList = "+filterList)
val results = table.get(new
Get(Bytes.toBytes("row")).setFilter(filterList))
val cells = results.rawCells()
if (cells != null) {
for (cell <- cells) {
val value = new String(CellUtil.cloneValue(cell))
val qualifier = new String(CellUtil.cloneQualifier(cell))
val family = new String(CellUtil.cloneFamily(cell))
val result = "OFFSET = "+offset+":"+family + "," + qualifier
+ "," + value + "," + cell.getTimestamp()
resultsList.append(result)
}
}
}
resultsList.foreach(println)
{code}
Here are the results for different limit and logicalOp settings:
{code:none}
Limit = 1 & logicalOp = MUST_PASS_ALL:
scala> resultsList.foreach(println)
OFFSET = 0:family,name,Jane,1
Limit = 1 & logicalOp = MUST_PASS_ONE:
scala> resultsList.foreach(println)
OFFSET = 0:family,name,Jane,1
OFFSET = 1:family,name,Gil,1
OFFSET = 2:family,name,Jane,1
OFFSET = 3:family,name,John,1
Limit = 2 & logicalOp = MUST_PASS_ALL:
scala> resultsList.foreach(println)
OFFSET = 0:family,name,Jane,1
Limit = 2 & logicalOp = MUST_PASS_ONE:
scala> resultsList.foreach(println)
OFFSET = 0:family,name,Jane,1
OFFSET = 2:family,name,Jane,1
{code}
So, it seems that MUST_PASS_ALL gives the expected behavior, but MUST_PASS_ONE
does not. Furthermore, MUST_PASS_ONE seems to give only a single
(not-duplicated) within a page, but not across pages.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17681) Simplify ssh and 'Operation Not Permitted' issues when our IT tests execute monkeys
Appy created HBASE-17681:
Summary: Simplify ssh and 'Operation Not Permitted' issues when
our IT tests execute monkeys
Key: HBASE-17681
URL: https://issues.apache.org/jira/browse/HBASE-17681
Project: HBase
Issue Type: Improvement
Reporter: Appy
Assignee: Appy
{noformat}
Executing remote command: ps ux | grep proc_regionserver | grep -v grep | tr -s
' ' | cut -d ' ' -f2 | xargs kill -s SIGKILL , hostname:appy2-5.vpc.cloudera.com
util.Shell: Executing full command [/usr/bin/ssh appy2-5.vpc.cloudera.com
"sudo -u hbase ps ux | grep proc_regionserver | grep -v grep | tr -s ' ' | cut
-d ' ' -f2 | xargs kill -s SIGKILL"]
{noformat}
The issue with above command is:
- kill commands runs as the ssh'ed user, and if it's not hbase, it has to be
root
- which means, that either hbase or root needs to have passwordless access
setup to all the other hosts of the cluster, which may not be true in many
cases.
Generalizing it a bit so that it can also work with a setup when there is a
user (\*any\* user) in the cluster with passwordless access to other nodes, and
sudo permissions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879486#comment-15879486 ] stack commented on HBASE-17343: --- Yeah, a YCSB run would do too... as per [~anoop.hbase]. I could do it even. Put up a patch. > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879667#comment-15879667
]
Zheng Hu commented on HBASE-17672:
--
> Other than spelling, v2 seems to be same as v1 ?
Yes.
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch, HBASE-17672.v1.patch,
> HBASE-17672.v2.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879672#comment-15879672
]
Zheng Hu commented on HBASE-17672:
--
Any concerns ? [~tedyu] , Thanks.
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch, HBASE-17672.v1.patch,
> HBASE-17672.v2.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17595) Add partial result support for small/limited scan
[ https://issues.apache.org/jira/browse/HBASE-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879679#comment-15879679 ] Duo Zhang commented on HBASE-17595: --- Let me check the failed UT. > Add partial result support for small/limited scan > - > > Key: HBASE-17595 > URL: https://issues.apache.org/jira/browse/HBASE-17595 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17595.patch > > > The partial result support is marked as a 'TODO' when implementing > HBASE-17045. And when implementing HBASE-17508, we found that if we make > small scan share the same logic with general scan, the scan request other > than open scanner will not have the small flag so the server may return > partial result to the client and cause some strange behavior. It is solved by > modifying the logic at server side, but this means the 1.4.x client is not > safe to contact with earlier 1.x server. So we'd better address the problem > at client side. Marked as blocker as this issue should be finished before any > 2.x and 1.4.x releases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17428) Expand on shell commands for detailed insight
[
https://issues.apache.org/jira/browse/HBASE-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879684#comment-15879684
]
Ted Yu commented on HBASE-17428:
The patch is sizable.
Can you put it on review board ?
> Expand on shell commands for detailed insight
> -
>
> Key: HBASE-17428
> URL: https://issues.apache.org/jira/browse/HBASE-17428
> Project: HBase
> Issue Type: Sub-task
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: HBASE-16961
>
> Attachments: HBASE-17428.001.patch, HBASE-17428.002.patch,
> HBASE-17428.003.HBASE-16961.patch
>
>
> Was talking over how to build some tests with [~romil.choksi], and what kind
> of information we capture and expose via some more shell commands.
> * Current SpaceQuotaSnapshots that the Master sees
> * Current SpaceQuotaSnapshots that a RS sees (or all RSs)
> * Current SpaceViolationPolicyEnforcements that a RS has enabled
> We can build out on the {{status}} command, with some options for different
> levels of detail, perhaps. Shouldn't be too tricky -- we have the info, just
> need to wire up java API, RPC, and shell command.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-17428) Expand on shell commands for detailed insight
[
https://issues.apache.org/jira/browse/HBASE-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879442#comment-15879442
]
Josh Elser edited comment on HBASE-17428 at 2/22/17 11:14 PM:
--
.003 Has some cleanup over 002 and adds a ruby test.
FYI [~tedyu]
was (Author: elserj):
.003 Has some cleanup over 002 and adds a ruby test.
> Expand on shell commands for detailed insight
> -
>
> Key: HBASE-17428
> URL: https://issues.apache.org/jira/browse/HBASE-17428
> Project: HBase
> Issue Type: Sub-task
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: HBASE-16961
>
> Attachments: HBASE-17428.001.patch, HBASE-17428.002.patch,
> HBASE-17428.003.HBASE-16961.patch
>
>
> Was talking over how to build some tests with [~romil.choksi], and what kind
> of information we capture and expose via some more shell commands.
> * Current SpaceQuotaSnapshots that the Master sees
> * Current SpaceQuotaSnapshots that a RS sees (or all RSs)
> * Current SpaceViolationPolicyEnforcements that a RS has enabled
> We can build out on the {{status}} command, with some options for different
> levels of detail, perhaps. Shouldn't be too tricky -- we have the info, just
> need to wire up java API, RPC, and shell command.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17428) Expand on shell commands for detailed insight
[
https://issues.apache.org/jira/browse/HBASE-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-17428:
---
Attachment: HBASE-17428.003.HBASE-16961.patch
.003 Has some cleanup over 002 and adds a ruby test.
> Expand on shell commands for detailed insight
> -
>
> Key: HBASE-17428
> URL: https://issues.apache.org/jira/browse/HBASE-17428
> Project: HBase
> Issue Type: Sub-task
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: HBASE-16961
>
> Attachments: HBASE-17428.001.patch, HBASE-17428.002.patch,
> HBASE-17428.003.HBASE-16961.patch
>
>
> Was talking over how to build some tests with [~romil.choksi], and what kind
> of information we capture and expose via some more shell commands.
> * Current SpaceQuotaSnapshots that the Master sees
> * Current SpaceQuotaSnapshots that a RS sees (or all RSs)
> * Current SpaceViolationPolicyEnforcements that a RS has enabled
> We can build out on the {{status}} command, with some options for different
> levels of detail, perhaps. Shouldn't be too tricky -- we have the info, just
> need to wire up java API, RPC, and shell command.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17428) Expand on shell commands for detailed insight
[
https://issues.apache.org/jira/browse/HBASE-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-17428:
---
Fix Version/s: HBASE-16961
> Expand on shell commands for detailed insight
> -
>
> Key: HBASE-17428
> URL: https://issues.apache.org/jira/browse/HBASE-17428
> Project: HBase
> Issue Type: Sub-task
> Components: shell
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: HBASE-16961
>
> Attachments: HBASE-17428.001.patch, HBASE-17428.002.patch,
> HBASE-17428.003.HBASE-16961.patch
>
>
> Was talking over how to build some tests with [~romil.choksi], and what kind
> of information we capture and expose via some more shell commands.
> * Current SpaceQuotaSnapshots that the Master sees
> * Current SpaceQuotaSnapshots that a RS sees (or all RSs)
> * Current SpaceViolationPolicyEnforcements that a RS has enabled
> We can build out on the {{status}} command, with some options for different
> levels of detail, perhaps. Shouldn't be too tricky -- we have the info, just
> need to wire up java API, RPC, and shell command.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879490#comment-15879490 ] stack commented on HBASE-17662: --- bq. Therefore it is acceptable to just skip the in-memory flush action while the updates come as part of replay from WAL. IIRC, you would flush during replay when the replayed data was bigger than your memory could hold Turning it off would mess us up. Could you instead take the update lock during replay? > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17680) Run mini cluster through JNI in tests
[
https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879524#comment-15879524
]
Ted Yu commented on HBASE-17680:
The two tests where shell command is replaced:
{code}
RESULTS FOR //core:location-cache-test
PASS 5.2s 3 Passed 0 Skipped 0 Failed //core:location-cache-test
TESTS PASSED
RESULTS FOR //core:client-test
PASS 2.8s 6 Passed 0 Skipped 0 Failed //core:client-test
TESTS PASSED
{code}
> Run mini cluster through JNI in tests
> -
>
> Key: HBASE-17680
> URL: https://issues.apache.org/jira/browse/HBASE-17680
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17680.v1.txt
>
>
> Currently tests start local hbase cluster through hbase shell.
> There is less control over the configuration of the local cluster this way.
> This issue would replace hbase shell with JNI interface to mini cluster.
> We would have full control over the cluster behavior.
> Thanks to [~devaraj] who started this initiative.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-17672:
-
Attachment: HBASE-17672.v2.patch
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch, HBASE-17672.v1.patch,
> HBASE-17672.v2.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879665#comment-15879665
]
Zheng Hu commented on HBASE-17672:
--
Upload patch v2 , [~ted_yu]
The original purpose of the UT:
1. grant permission like W action.
2. assert action exists;
3. grant other permissions like RXCA.
4. assert that whether RXCA will override previous actions. (after
HBASE-17472, assert merge behavior)
The original test use table owner for testing, so we can skip step.1 because
RWXCA are granted for owner, but now we use a new created user
(test_grant_revoke_user), so we must grant some actions (we choose WRITE) first
for step.1 .
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch, HBASE-17672.v1.patch,
> HBASE-17672.v2.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879662#comment-15879662
]
Ted Yu commented on HBASE-17672:
Other than spelling, v2 seems to be same as v1 ?
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch, HBASE-17672.v1.patch,
> HBASE-17672.v2.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16942) Add FavoredStochasticLoadBalancer and FN Candidate generators
[
https://issues.apache.org/jira/browse/HBASE-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879694#comment-15879694
]
Hadoop QA commented on HBASE-16942:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 31s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 118m 12s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 161m 6s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12854083/HBASE-16942.master.004.patch
|
| JIRA Issue | HBASE-16942 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 1d437b7877b7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / e4c06c1 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5805/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5805/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Add FavoredStochasticLoadBalancer and FN Candidate generators
> -
>
> Key: HBASE-16942
> URL: https://issues.apache.org/jira/browse/HBASE-16942
> Project: HBase
> Issue Type: Sub-task
> Components: FavoredNodes
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 2.0.0
>
> Attachments:
[jira] [Commented] (HBASE-17680) Run mini cluster through JNI in tests
[ https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879430#comment-15879430 ] Ted Yu commented on HBASE-17680: docker hardcodes CLASSPATH as "/usr/src/buck/build/bootstrapper/bootstrapper.jar" Workaround is to embed the output of "bin/hbase classpath" command in a file (/tmp/clspath) which would be read by MiniCluster#create_vm() > Run mini cluster through JNI in tests > - > > Key: HBASE-17680 > URL: https://issues.apache.org/jira/browse/HBASE-17680 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17680.v1.txt > > > Currently tests start local hbase cluster through hbase shell. > There is less control over the configuration of the local cluster this way. > This issue would replace hbase shell with JNI interface to mini cluster. > We would have full control over the cluster behavior. > Thanks to [~devaraj] who started this initiative. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17680) Run mini cluster through JNI in tests
[ https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879481#comment-15879481 ] stack commented on HBASE-17680: --- We already write a cache of classpath at ./target/cached_classpath.txt > Run mini cluster through JNI in tests > - > > Key: HBASE-17680 > URL: https://issues.apache.org/jira/browse/HBASE-17680 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17680.v1.txt > > > Currently tests start local hbase cluster through hbase shell. > There is less control over the configuration of the local cluster this way. > This issue would replace hbase shell with JNI interface to mini cluster. > We would have full control over the cluster behavior. > Thanks to [~devaraj] who started this initiative. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17590) Drop cache hint should work for StoreFile write path
[
https://issues.apache.org/jira/browse/HBASE-17590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879482#comment-15879482
]
Hudson commented on HBASE-17590:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2553 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2553/])
HBASE-17590 Drop cache hint should work on store file write path (Ashu (garyh:
rev e4c06c120a8bb964fe72412cef216647b147de72)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileWriter.java
> Drop cache hint should work for StoreFile write path
>
>
> Key: HBASE-17590
> URL: https://issues.apache.org/jira/browse/HBASE-17590
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Ashu Pachauri
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: HBASE-17590.master.001.patch
>
>
> We have this in the code right now.
> {noformat}
> public Builder withShouldDropCacheBehind(boolean
> shouldDropCacheBehind/*NOT USED!!*/) {
> // TODO: HAS NO EFFECT!!! FIX!!
> return this;
> }
> {noformat}
> Creating jira to track it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879497#comment-15879497 ] stack commented on HBASE-17662: --- ...though taking the lock, would that mean no flush during replay... If so, then that wouldn't be correct either. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-13882) Fix RegionSplitPolicy section in HBase book
[
https://issues.apache.org/jira/browse/HBASE-13882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13882:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
Status: Resolved (was: Patch Available)
Pushed. Thanks [~Jan Hentschel]
> Fix RegionSplitPolicy section in HBase book
>
>
> Key: HBASE-13882
> URL: https://issues.apache.org/jira/browse/HBASE-13882
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Vladimir Rodionov
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-13882.master.001.patch
>
>
> 65.4.1. Custom Split Policies
> The section starts with the following statement:
> {quote}
> ou can override the default split policy using a custom
> RegionSplitPolicy(HBase 0.94+). Typically a custom split policy should extend
> HBase’s default split policy: IncreasingToUpperBoundRegionSplitPolicy.
> {quote}
> There is typo above as well.
> Then if we scroll down a little bit:
> {quote}
> The default split policy can be overwritten using a custom
> RegionSplitPolicy(HBase 0.94+). Typically a custom split policy should extend
> HBase’s default split policy: ConstantSizeRegionSplitPolicy.
> {quote}
> The link:
> http://hbase.apache.org/book.html#_custom_split_policies
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17680) Run mini cluster through JNI in tests
[ https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17680: --- Attachment: 17680.v3.txt Patch v3 removes commented out code. TestUtil is moved to test class level since TestUtil ctor starts mini cluster which should be reused by all the subtests in the same test class. > Run mini cluster through JNI in tests > - > > Key: HBASE-17680 > URL: https://issues.apache.org/jira/browse/HBASE-17680 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17680.v1.txt, 17680.v3.txt > > > Currently tests start local hbase cluster through hbase shell. > There is less control over the configuration of the local cluster this way. > This issue would replace hbase shell with JNI interface to mini cluster. > We would have full control over the cluster behavior. > Thanks to [~devaraj] who started this initiative. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879631#comment-15879631
]
stack commented on HBASE-14123:
---
Trying the patch:
What is this format when I list out my set named 'me' w/ three tables in it:
me={x,y,z}
Is it json or something else?
I started up a job with following:
./hbase/bin/hbase --config ~/conf_hbase backup create full
hdfs://ve0524.halxg.cloudera.com:8020/bkup
... it seems to have started. No progress. I killed the command. When I do
progress, it found a job:
Found ongoing session with backupId=backup_1487810721996
backup_1487810721996 progress=0%
Says... zero progress. I looked for the bkup dir but not created. Its not
running a mr job that I can tell. I do a describe and it says:
{ID=backup_1487810721996,Type=FULL,Tables={IntegrationTestBigLinkedList,x,y,z},State=RUNNING,Start
time=Wed Feb 22 16:45:22 PST 2017,Phase=REQUEST,Progress=0%}
(Is the above JSON?)
If I delete the backup, it seems to get rid of it.
Any idea what I did wrong?
The backup create usage mentions 'restore' when it should be 'backup'?
{code}
Usage: hbase backup create [options]
type "full" to create a full backup image
"incremental" to create an incremental backup image
backup_path Full path to store the backup image
Options:
-b Bandwidth per task (MapReduce task) in MB/s
-s Backup set to restore, mutually exclusive with -t (table list)
-t Table name list, comma-separated.
-w Number of parallel MapReduce tasks to execute
{code}
I retried creating a backup with three empty tables in a set It does this:
{code}
stack@ve0524:~$ ./hbase/bin/hbase --config ~/conf_hbase backup create full
hdfs://ve0524.halxg.cloudera.com:8020/bkup -s me
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/stack/hbase-2.0.0-SNAPSHOT/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/stack/hadoop-2.7.4-SNAPSHOT/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-02-22 16:59:53,327 DEBUG [main] util.ClassSize: Using Unsafe to estimate
memory layout
2017-02-22 16:59:53,332 DEBUG [main] ipc.AbstractRpcClient:
Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5644dc81, compressor=null,
tcpKeepAlive=true, tcpNoDelay=true, connectTO=1, readTO=2,
writeTO=6, minIdleTimeBeforeClose=12, maxRetries=0,
fallbackAllowed=false, bind address=null
2017-02-22 16:59:53,589 DEBUG [main] ipc.RpcConnection: Use SIMPLE
authentication for service ClientService, sasl=false
2017-02-22 16:59:53,754 DEBUG [main] ipc.NettyRpcConnection: Connecting to
ve0542.halxg.cloudera.com/10.17.240.29:16020
2017-02-22 16:59:53,908 DEBUG [main] ipc.RpcConnection: Use SIMPLE
authentication for service ClientService, sasl=false
2017-02-22 16:59:53,908 DEBUG [main] ipc.NettyRpcConnection: Connecting to
ve0542.halxg.cloudera.com/10.17.240.29:16020
2017-02-22 16:59:53,911 DEBUG [main] ipc.RpcConnection: Use SIMPLE
authentication for service ClientService, sasl=false
2017-02-22 16:59:53,912 DEBUG [main] ipc.NettyRpcConnection: Connecting to
ve0542.halxg.cloudera.com/10.17.240.29:16020
2017-02-22 16:59:53,959 DEBUG [hconnection-0x130e116b-shared-pool3-t1]
ipc.RpcConnection: Use SIMPLE authentication for service ClientService,
sasl=false
2017-02-22 16:59:53,959 DEBUG [hconnection-0x130e116b-shared-pool3-t1]
ipc.NettyRpcConnection: Connecting to
ve0542.halxg.cloudera.com/10.17.240.29:16020
2017-02-22 16:59:53,992 DEBUG
[hconnection-0x130e116b-metaLookup-shared--pool5-t1] ipc.RpcConnection: Use
SIMPLE authentication for service ClientService, sasl=false
2017-02-22 16:59:53,993 DEBUG
[hconnection-0x130e116b-metaLookup-shared--pool5-t1] ipc.NettyRpcConnection:
Connecting to ve0542.halxg.cloudera.com/10.17.240.29:16020
2017-02-22 16:59:54,002 DEBUG [main] ipc.RpcConnection: Use SIMPLE
authentication for service ClientService, sasl=false
2017-02-22 16:59:54,003 DEBUG [main] ipc.NettyRpcConnection: Connecting to
ve0538.halxg.cloudera.com/10.17.240.27:16020
2017-02-22 16:59:54,029 DEBUG [main] ipc.AbstractRpcClient: Stopping rpc client
2017-02-22 16:59:54,040 DEBUG [main] ipc.AbstractRpcClient:
Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c1dc8e, compressor=null,
tcpKeepAlive=true, tcpNoDelay=true, connectTO=1, readTO=2,
writeTO=6, minIdleTimeBeforeClose=12, maxRetries=0,
fallbackAllowed=false, bind address=null
2017-02-22 16:59:54,171 DEBUG [main] ipc.RpcConnection: Use SIMPLE
authentication for service ClientService, sasl=false
2017-02-22 16:59:54,171 DEBUG [main] ipc.NettyRpcConnection: Connecting to
[jira] [Updated] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-16990:
-
Attachment: (was: HBASE-16990.v2.patch)
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, HBASE-16990.v2.patch,
> hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-16990:
-
Attachment: HBASE-16990.v2.patch
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, HBASE-16990.v2.patch,
> hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17672) "Grant should set access rights appropriately" test fails
[
https://issues.apache.org/jira/browse/HBASE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-17672:
-
Attachment: HBASE-17672.v1.patch
Upload patch v1. [~tedyu] .
> "Grant should set access rights appropriately" test fails
> -
>
> Key: HBASE-17672
> URL: https://issues.apache.org/jira/browse/HBASE-17672
> Project: HBase
> Issue Type: Test
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Zheng Hu
> Fix For: 2.0.0
>
> Attachments: HBASE-17672.patch, HBASE-17672.v1.patch
>
>
> The following test failure is reproducible after HBASE-17472 went in:
> {code}
> 1) Failure:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest)
> [./src/test/ruby/hbase/security_admin_test.rb:66:in
> `test_Grant_should_set_access_rights_appropriately'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:154:in
> `user_permission'
>
> file:/Users/tyu/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/builtin/java/java.util.rb:7:in
> `each'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/security.rb:136:in
> `user_permission'
> ./src/test/ruby/hbase/security_admin_test.rb:65:in
> `test_Grant_should_set_access_rights_appropriately'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each']:
> {code}
> [~openinx]:
> Can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878365#comment-15878365 ] Anoop Sam John commented on HBASE-17662: So the need for volatile/Atomic boolean is there? Am not sure as did not see in detail which thread deals with this variable. (Single only or multiple).. Pls check once and answer. Ya at least moving down the check against AtomicBoolean and we are good there. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878320#comment-15878320
]
Hadoop QA commented on HBASE-16990:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 6s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 6s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 39s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 21s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853959/HBASE-16990.v2.patch |
| JIRA Issue | HBASE-16990 |
| Optional Tests | asflicense rubocop ruby_lint |
| uname | Linux 1cd9de3480e6 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f037f23 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5798/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, HBASE-16990.v2.patch,
> hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878322#comment-15878322
]
Hadoop QA commented on HBASE-16990:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 7s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 7s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 19s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 53s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853960/HBASE-16990.v2.patch |
| JIRA Issue | HBASE-16990 |
| Optional Tests | asflicense rubocop ruby_lint |
| uname | Linux 420a9873c289 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f037f23 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5799/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, HBASE-16990.v2.patch,
> hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878353#comment-15878353
]
Hadoop QA commented on HBASE-16990:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 13s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 13s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 12s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 1s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853961/HBASE-16990.v2.patch |
| JIRA Issue | HBASE-16990 |
| Optional Tests | asflicense rubocop ruby_lint |
| uname | Linux 9597d211bc3a 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / f037f23 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5800/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, HBASE-16990.v2.patch,
> hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878448#comment-15878448
]
Hudson commented on HBASE-17561:
FAILURE: Integrated in Jenkins build HBase-1.2-IT #603 (See
[https://builds.apache.org/job/HBase-1.2-IT/603/])
HBASE-17561 table status page should escape values that may contain (busbey:
rev 8b9455cd586728e71080da8804b0c0824d00cb2f)
* (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-14123) HBase Backup/Restore Phase 2
[
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879703#comment-15879703
]
Vladimir Rodionov edited comment on HBASE-14123 at 2/23/17 1:57 AM:
[~saint@gmail.com], make sure you enable backup fully in hbase-site.xml (on
all nodes)
{code}
hbase.backup.enable=true
hbase.master.logcleaner.plugins=YOUR_PUGINS,org.apache.hadoop.hbase.backup.master.BackupLogCleaner
hbase.procedure.master.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
hbase.procedure.regionserver.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager
{code}
was (Author: vrodionov):
[~saint@gmail.com], make sure you enable backup fully in hbase-site.xml
{code}
hbase.backup.enable=true
hbase.master.logcleaner.plugins=YOUR_PUGINS,org.apache.hadoop.hbase.backup.master.BackupLogCleaner
hbase.procedure.master.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
hbase.procedure.regionserver.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager
{code}
> HBase Backup/Restore Phase 2
>
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
> Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: HBASE-7912
>
> Attachments: 14123-master.v14.txt, 14123-master.v15.txt,
> 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt,
> 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt,
> 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt,
> 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt,
> 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt,
> 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt,
> 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt,
> 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch,
> 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch,
> 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch,
> 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch,
> 14123.master.v52.patch, 14123.master.v54.patch, 14123.master.v56.patch,
> 14123.master.v57.patch, 14123-master.v5.txt, 14123-master.v6.txt,
> 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt,
> Backup-restoreinHBase2.0 (1).pdf, Backup-restoreinHBase2.0.pdf,
> HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch,
> HBASE-14123-v10.patch, HBASE-14123-v11.patch, HBASE-14123-v12.patch,
> HBASE-14123-v13.patch, HBASE-14123-v15.patch, HBASE-14123-v16.patch,
> HBASE-14123-v1.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch,
> HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch,
> HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16990) Shell tool to dump table and namespace privileges
[
https://issues.apache.org/jira/browse/HBASE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879735#comment-15879735
]
Zheng Hu commented on HBASE-16990:
--
[~jerryhe], Thanks for reply. permission_dumper.rb is a wrapper for
user_permission. For developer, the script seems to be redundant. But for
HBase administrators , it's helpful to migrate privileges from one cluster to
another cluster.
Currently , user_permission shell command print following message:
{code}
hbase(main):005:0> user_permission 't1'
User
Namespace,Table,Family,Qualifier:Permission
openinx default,t1,,:
[Permission: actions=READ,WRITE,EXEC,CREATE,ADMIN]
...
{code}
If lots of permissions, then administrator has to spell it in sink hbase shell
for each permission. It's time wasting.
> Shell tool to dump table and namespace privileges
> -
>
> Key: HBASE-16990
> URL: https://issues.apache.org/jira/browse/HBASE-16990
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Attachments: HBASE-16990.v1.patch, HBASE-16990.v2.patch,
> hbase-site.xml
>
>
> Recently, we are trying to migrate tables from Cluster-A to Cluster-B, I
> found that HBase lack some useful tools :
> 1. dump table schema, like mysqldump in mysql
> 2. dump table privileges, like pt-show-grants in mysql provided by Percona.
> I think we can add a dump sub-command looks like (JUST simple demo) :
> {code}
> $ ./bin/hbase dump -t test_table --with-privileges > ~/test_table.hsh
> $ cat ~/test_table.hsh
> create 'test_table', {NAME=>'f1'}
> grant 'test_user', 'RW', 'test_table'
> {code}
> Maybe I can contribute ... :)
> How do you think ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
