[jira] [Commented] (HBASE-13153) Bulk Loaded HFile Replication
[ https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951863#comment-14951863 ] Ted Yu commented on HBASE-13153: +1 > Bulk Loaded HFile Replication > - > > Key: HBASE-13153 > URL: https://issues.apache.org/jira/browse/HBASE-13153 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: sunhaitao >Assignee: Ashish Singhi > Fix For: 2.0.0 > > Attachments: HBASE-13153-v1.patch, HBASE-13153-v2.patch, > HBASE-13153-v3.patch, HBASE-13153-v4.patch, HBASE-13153-v5.patch, > HBASE-13153.patch, HBase Bulk Load Replication-v1-1.pdf, HBase Bulk Load > Replication-v2.pdf, HBase Bulk Load Replication.pdf > > > Currently we plan to use HBase Replication feature to deal with disaster > tolerance scenario.But we encounter an issue that we will use bulkload very > frequently,because bulkload bypass write path, and will not generate WAL, so > the data will not be replicated to backup cluster. It's inappropriate to > bukload twice both on active cluster and backup cluster. So i advise do some > modification to bulkload feature to enable bukload to both active cluster and > backup cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951925#comment-14951925
]
Hudson commented on HBASE-14581:
SUCCESS: Integrated in HBase-1.1 #699 (See
[https://builds.apache.org/job/HBase-1.1/699/])
HBASE-14581 Znode cleanup throws auth exception in secure mode (tedyu: rev
f4898bb98f3937926b0af8c2682b2797944f0522)
* bin/hbase-daemon.sh
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14588) Stop accessing test resources from within src folder
[
https://issues.apache.org/jira/browse/HBASE-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952026#comment-14952026
]
Andrew Purtell edited comment on HBASE-14588 at 10/10/15 8:04 PM:
--
Edit: Rescinding a +1. The changes look fine but test execution thereafter
isn't. I only checked the tests modified by the patch:
{noformat}
---
T E S T S
---
Running org.apache.hadoop.hbase.io.TestReference
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.632 sec <<<
FAILURE! - in org.apache.hadoop.hbase.io.TestReference
testParsingWritableReference(org.apache.hadoop.hbase.io.TestReference) Time
elapsed: 0.955 sec <<< ERROR!
java.io.FileNotFoundException: File
/Users/apurtell/src/hbase/hbase-server/target/test-classes/a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c
does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:606)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:819)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:596)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:140)
at
org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:341)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767)
at org.apache.hadoop.hbase.io.Reference.read(Reference.java:169)
at
org.apache.hadoop.hbase.io.TestReference.testParsingWritableReference(TestReference.java:50)
Running org.apache.hadoop.hbase.regionserver.TestRecoveredEdits
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.124 sec <<<
FAILURE! - in org.apache.hadoop.hbase.regionserver.TestRecoveredEdits
testReplayWorksThoughLotsOfFlushing(org.apache.hadoop.hbase.regionserver.TestRecoveredEdits)
Time elapsed: 1.484 sec <<< ERROR!
java.io.FileNotFoundException: File
/Users/apurtell/src/hbase/hbase-server/target/test-classes/0016310
does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:606)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:819)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:596)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
at
org.apache.hadoop.fs.LocalFileSystem.copyToLocalFile(LocalFileSystem.java:88)
at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1975)
at
org.apache.hadoop.hbase.regionserver.TestRecoveredEdits.testReplayWorksThoughLotsOfFlushing(TestRecoveredEdits.java:124)
Results :
Tests in error:
TestReference.testParsingWritableReference:50 » FileNotFound File
/Users/apurt...
TestRecoveredEdits.testReplayWorksThoughLotsOfFlushing:124 » FileNotFound
File...
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0
{noformat}
was (Author: apurtell):
+1
> Stop accessing test resources from within src folder
>
>
> Key: HBASE-14588
> URL: https://issues.apache.org/jira/browse/HBASE-14588
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
> Attachments: hbase-14588.001.patch, hbase-14588.001.patch
>
>
> A few tests in hbase-server reach into the src/test/data folder to get test
> resources, which is naughty since tests are supposed to only operate within
> the target/ folder. It's better to put these into src/test/resources and let
> them be automatically copied into target/ via the resources plugin, like
> other test resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951967#comment-14951967 ] Andrew Purtell commented on HBASE-14583: That build failure may be noise. Let me check this locally [~tedyu] by running the test at this commit in a loop 20 times or so. If you have a moment you should as well. If we're not able to reproduce it, suggest we resolve this again. > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952024#comment-14952024
]
Andrew Purtell commented on HBASE-14578:
This only went into trunk Ted? Isn't it appropriate back through 0.98?
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.3.0, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter regions contain the reference of the
> original HFile until
[jira] [Commented] (HBASE-13153) Bulk Loaded HFile Replication
[
https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951861#comment-14951861
]
Hadoop QA commented on HBASE-13153:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765980/HBASE-13153-v5.patch
against master branch at commit 1b66ea71b69e8b24ba7ecb9ba6548d775777f417.
ATTACHMENT ID: 12765980
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 30 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15952//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15952//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15952//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15952//console
This message is automatically generated.
> Bulk Loaded HFile Replication
> -
>
> Key: HBASE-13153
> URL: https://issues.apache.org/jira/browse/HBASE-13153
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: sunhaitao
>Assignee: Ashish Singhi
> Fix For: 2.0.0
>
> Attachments: HBASE-13153-v1.patch, HBASE-13153-v2.patch,
> HBASE-13153-v3.patch, HBASE-13153-v4.patch, HBASE-13153-v5.patch,
> HBASE-13153.patch, HBase Bulk Load Replication-v1-1.pdf, HBase Bulk Load
> Replication-v2.pdf, HBase Bulk Load Replication.pdf
>
>
> Currently we plan to use HBase Replication feature to deal with disaster
> tolerance scenario.But we encounter an issue that we will use bulkload very
> frequently,because bulkload bypass write path, and will not generate WAL, so
> the data will not be replicated to backup cluster. It's inappropriate to
> bukload twice both on active cluster and backup cluster. So i advise do some
> modification to bulkload feature to enable bukload to both active cluster and
> backup cluster
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952005#comment-14952005 ] Andrew Purtell commented on HBASE-14583: I'm seeing a general issue with TestShell becoming a zombie on branch-1 both where this change went in and at the commit before. > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14583: --- Comment: was deleted (was: I'm seeing a general issue with TestShell becoming a zombie on branch-1 both where this change went in and at the commit before. ) > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952019#comment-14952019
]
Hudson commented on HBASE-14581:
FAILURE: Integrated in HBase-1.0 #1075 (See
[https://builds.apache.org/job/HBase-1.0/1075/])
HBASE-14581 Znode cleanup throws auth exception in secure mode (apurtell: rev
7c0a4073969fb79ab3c5b324645ddb0515835dcf)
* bin/hbase-daemon.sh
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951928#comment-14951928
]
Hudson commented on HBASE-14581:
FAILURE: Integrated in HBase-1.2 #238 (See
[https://builds.apache.org/job/HBase-1.2/238/])
HBASE-14581 Znode cleanup throws auth exception in secure mode (tedyu: rev
734def4fd672008525d747559edc92b63a222bb4)
* bin/hbase-daemon.sh
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951969#comment-14951969
]
Ted Yu commented on HBASE-14581:
You're right, Andrew
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952013#comment-14952013 ] Andrew Purtell commented on HBASE-14583: Yep, this change seems to make zombies [~ndimiduk] /cc [~stack] > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14586) Remove extraneous ${argLine} references in pom
[
https://issues.apache.org/jira/browse/HBASE-14586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14586:
---
Status: Open (was: Patch Available)
This change breaks HBASE-8808.
To test: {{mvn clean package -Dtest=TestCheckTestClasses
-Dhbase.skip-jacoco=false}}.
The unit test will run when the hbase-server module is built. If HBASE-8808
changes are working properly, you'll see at the 'report' phase:
{noformat}
[INFO] --- jacoco-maven-plugin:0.7.5.201505241946:report (report) @
hbase-server ---
[INFO] Analyzed bundle 'Apache HBase - Server' with 1595 classes
{noformat}
If argLine isn't set correctly, i.e. after this patch is applied, instead
you'll see a message about how jacoco didn't run due to a missing "exec" file.
> Remove extraneous ${argLine} references in pom
> --
>
> Key: HBASE-14586
> URL: https://issues.apache.org/jira/browse/HBASE-14586
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Attachments: hbase-14586.001.patch, hbase-14586.001.patch
>
>
> The pom.xml has a line like this for the Surefire argLine, which has an extra
> ${argLine} reference. Recommend changes like this:
> {noformat}
> -${hbase-surefire.argLine} ${argLine}
> +${hbase-surefire.argLine}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952008#comment-14952008 ] Andrew Purtell commented on HBASE-14583: We make TestShell zombies on branch-1 at where this change went in. Let me check if this is the case at the commit before. > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952023#comment-14952023
]
Andrew Purtell commented on HBASE-14578:
+1
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.3.0, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter regions contain the reference of the
> original HFile until compaction happens.
> In RestoreSnapshotHelper (while restoring snapshot
[jira] [Updated] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14581:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review, Andy.
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14581:
---
Fix Version/s: 0.98.16
1.0.3
This is relevant for 1.0 and 0.98, picked back there also
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14535) Unit test for rpc connection concurrency / deadlock testing
[ https://issues.apache.org/jira/browse/HBASE-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951970#comment-14951970 ] Andrew Purtell commented on HBASE-14535: If I parachuted in and found this test only failing in some cases but not in others, while trying to clean up a slew of flakes, I'd likely disable this or move it out to an IT. True, every failure should be looked at, but we let flakiness build up in our suite and leave it to angels to come in later and triage. When triaging, deep analysis is deferred, by definition. On the other hand, integration tests are expected to have some level of nondeterminism as they are designed to run against a full cluster with a lot of moving parts. > Unit test for rpc connection concurrency / deadlock testing > > > Key: HBASE-14535 > URL: https://issues.apache.org/jira/browse/HBASE-14535 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16 > > Attachments: hbase-14535_v1.patch, hbase-14535_v2.patch > > > As per parent jira and recent jiras HBASE-14449 + HBASE-14241 and > HBASE-14313, we seem to be lacking some testing rpc connection concurrency > issues in a UT env. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14588) Stop accessing test resources from within src folder
[ https://issues.apache.org/jira/browse/HBASE-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952026#comment-14952026 ] Andrew Purtell commented on HBASE-14588: +1 > Stop accessing test resources from within src folder > > > Key: HBASE-14588 > URL: https://issues.apache.org/jira/browse/HBASE-14588 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: hbase-14588.001.patch, hbase-14588.001.patch > > > A few tests in hbase-server reach into the src/test/data folder to get test > resources, which is naughty since tests are supposed to only operate within > the target/ folder. It's better to put these into src/test/resources and let > them be automatically copied into target/ via the resources plugin, like > other test resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951919#comment-14951919
]
Hudson commented on HBASE-14581:
SUCCESS: Integrated in HBase-1.2-IT #199 (See
[https://builds.apache.org/job/HBase-1.2-IT/199/])
HBASE-14581 Znode cleanup throws auth exception in secure mode (tedyu: rev
734def4fd672008525d747559edc92b63a222bb4)
* bin/hbase-daemon.sh
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14589) Builds are being killed...
[
https://issues.apache.org/jira/browse/HBASE-14589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951998#comment-14951998
]
stack commented on HBASE-14589:
---
BUILDS seems to have gone away, yeah. Let me build up some evidence then will
make an INFRA
> Builds are being killed...
> --
>
> Key: HBASE-14589
> URL: https://issues.apache.org/jira/browse/HBASE-14589
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: stack
>
> I see this in a build that started at two hours ago... about 6:45... its
> build 15941 on ubuntu-6
> {code}
> WARNING: 2 rogue build processes detected, terminating.
> /bin/kill -9 18640
> /bin/kill -9 22625
> {code}
> If I back up to build 15939, started about 3 1/2 hours ago, say, 5:15 I
> see:
> Running org.apache.hadoop.hbase.client.TestShell
> Killed
> ... but it was running on ubuntu-1 so it doesn't look like we are killing
> ourselves... when we do this in test-patch.sh
> ### Kill any rogue build processes from the last attempt
> $PS auxwww | $GREP ${PROJECT_NAME}PatchProcess | $AWK '{print $2}' |
> /usr/bin/xargs -t -I {} /bin/kill -9 {} > /dev/null
> The above code runs in a few places... in test-patch.sh.
> Let me try and add some more info around what is being killed...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952012#comment-14952012
]
Andrew Purtell commented on HBASE-14584:
Sigh. We should drop the simple string comparisons, pick a few fields relevant
to the model under test, and check that the serialized form contains the fields
of interest and they are of the correct value.
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14588) Stop accessing test resources from within src folder
[ https://issues.apache.org/jira/browse/HBASE-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14588: --- Status: Open (was: Patch Available) > Stop accessing test resources from within src folder > > > Key: HBASE-14588 > URL: https://issues.apache.org/jira/browse/HBASE-14588 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: hbase-14588.001.patch, hbase-14588.001.patch > > > A few tests in hbase-server reach into the src/test/data folder to get test > resources, which is naughty since tests are supposed to only operate within > the target/ folder. It's better to put these into src/test/resources and let > them be automatically copied into target/ via the resources plugin, like > other test resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952117#comment-14952117
]
Hudson commented on HBASE-14578:
FAILURE: Integrated in HBase-1.2 #239 (See
[https://builds.apache.org/job/HBase-1.2/239/])
HBASE-14578 URISyntaxException during snapshot restore for table with (tedyu:
rev 9117141ebb7b585921e2b17d1b82cb390576b6a2)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at
[jira] [Commented] (HBASE-14158) Add documentation for Initial Release for HBase-Spark Module integration
[ https://issues.apache.org/jira/browse/HBASE-14158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952135#comment-14952135 ] Ted Malaska commented on HBASE-14158: - ok Cool. Sorry for being away. Now I'm back. I will make a new patch early monday Thanks Misty > Add documentation for Initial Release for HBase-Spark Module integration > - > > Key: HBASE-14158 > URL: https://issues.apache.org/jira/browse/HBASE-14158 > Project: HBase > Issue Type: Improvement > Components: documentation, spark >Reporter: Ted Malaska >Assignee: Ted Malaska > Fix For: 2.0.0 > > Attachments: HBASE-14158.1.patch, HBASE-14158.2.patch, > HBASE-14158.5.patch, HBASE-14158.5.patch, HBASE-14158.6.patch > > > Add documentation for Initial Release for HBase-Spark Module integration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14479) Apply the Leader/Followers pattern to RpcServer's Reader
[
https://issues.apache.org/jira/browse/HBASE-14479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14479:
--
Attachment: gets.png
median.png
gc.png
io.png
Some pictures. I ran YCSB four loadings C (100% random read). The first was
branch-1, the second was branch-1 + this patch, the third run was also branch-1
+ this patch, and then the fourth run was w/o the patch (same as first run;
i.e. run 1 and 4 were without the patch and runs 2 and 3 were with the patch).
I restarted between run 1 and 2 and between 3 and 4.
High-level, throughput is slightly up (depedent on the run..the differences are
pretty erratic).
Median, 75th and 95th are about the same... maybe less w/ the patch applied.
GC time may be less w/ the patch (the fourth run which is w/o the patch has
less GC but it is doing way less gets).
Load w/ patch seems slightly less w/ patch... i/os are about same across the
runs.
High-level, no regression (probably some benefit discernible at this
macro-level).
I'll commit the patch.
> Apply the Leader/Followers pattern to RpcServer's Reader
>
>
> Key: HBASE-14479
> URL: https://issues.apache.org/jira/browse/HBASE-14479
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC, Performance
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Attachments: HBASE-14479-V2 (1).patch, HBASE-14479-V2.patch,
> HBASE-14479-V2.patch, HBASE-14479.patch, gc.png, gets.png, io.png, median.png
>
>
> {{RpcServer}} uses multiple selectors to read data for load distribution, but
> the distribution is just done by round-robin. It is uncertain, especially for
> long run, whether load is equally divided and resources are used without
> being wasted.
> Moreover, multiple selectors may cause excessive context switches which give
> priority to low latency (while we just add the requests to queues), and it is
> possible to reduce throughput of the whole server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14479) Apply the Leader/Followers pattern to RpcServer's Reader
[
https://issues.apache.org/jira/browse/HBASE-14479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952159#comment-14952159
]
stack commented on HBASE-14479:
---
[~nkeywal] FYI. You'll like this one.
[~ikeda] Should we use this patten elsewhere, say, in the handoff to syncer
threads in WAL? See
http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/wal/FSHLog.html#1770
if I understand the pattern right, we could purge Readers and have Handlers
themselves do the select read from the socket (one less handoff)?
> Apply the Leader/Followers pattern to RpcServer's Reader
>
>
> Key: HBASE-14479
> URL: https://issues.apache.org/jira/browse/HBASE-14479
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC, Performance
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Attachments: HBASE-14479-V2 (1).patch, HBASE-14479-V2.patch,
> HBASE-14479-V2.patch, HBASE-14479.patch, gc.png, gets.png, io.png, median.png
>
>
> {{RpcServer}} uses multiple selectors to read data for load distribution, but
> the distribution is just done by round-robin. It is uncertain, especially for
> long run, whether load is equally divided and resources are used without
> being wasted.
> Moreover, multiple selectors may cause excessive context switches which give
> priority to low latency (while we just add the requests to queues), and it is
> possible to reduce throughput of the whole server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14567: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Resolving. Pushed addendum to master. Very nice work [~jingcheng...@intel.com] Thanks. It was 2 minutes for me which is a big improvement (two minutes was running through all modules to find test, running it, and then spinning out to the end). Great. > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Fix For: 2.0.0 > > Attachments: 14567v2.txt, HBASE-14567-addendum-v4.patch, > HBASE-14567-v3.patch, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14420) Zombie Stomping Session
[ https://issues.apache.org/jira/browse/HBASE-14420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952092#comment-14952092 ] stack commented on HBASE-14420: --- Looking at last 30 hadoop qa runs, these are which hung and failed: 1 Hanging test : org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence 2 Hanging test : org.apache.hadoop.hbase.client.TestShell 1 Hanging test : org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence 1 Hanging test : org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd 2 Hanging test : org.apache.hadoop.hbase.io.encoding.TestDataBlockEncoders 2 Hanging test : org.apache.hadoop.hbase.mapreduce.TestCellCounter 2 Hanging test : org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat 1 Hanging test : org.apache.hadoop.hbase.mapreduce.TestHashTable 1 Hanging test : org.apache.hadoop.hbase.mapreduce.TestImportExport 1 Hanging test : org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint 1 Hanging test : org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat 1 Hanging test : org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1 1 Hanging test : org.apache.hadoop.hbase.mapreduce.TestWALPlayer 1 Hanging test : org.apache.hadoop.hbase.mob.compactions.TestMobCompactor 1 Hanging test : org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper 1 Hanging test : org.apache.hadoop.hbase.regionserver.TestClusterId 1 Hanging test : org.apache.hadoop.hbase.regionserver.TestHRegion 1 Hanging test : org.apache.hadoop.hbase.regionserver.TestHRegionFileSystem 1 Hanging test : org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner 1 Hanging test : org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush 1 Hanging test : org.apache.hadoop.hbase.regionserver.compactions.TestCompactionWithThroughputController 1 Hanging test : org.apache.hadoop.hbase.replication.TestMasterReplication 1 Hanging test : org.apache.hadoop.hbase.replication.TestPerTableCFReplication 1 Hanging test : org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL 1 Hanging test : org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager 1 Hanging test : org.apache.hadoop.hbase.thrift.TestThriftServer 1 Hanging test : org.apache.hadoop.hbase.util.TestHBaseFsck 1 Hanging test : org.apache.hadoop.hbase.wal.TestWALFiltering 1 Hanging test : org.apache.hadoop.hbase.wal.TestWALSplit 1 Hanging test : org.apache.hadoop.hbase.wal.TestWALSplitCompressed 2 Failing test : org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence 4 Failing test : org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence 2 Failing test : org.apache.hadoop.hbase.master.TestZKLessAMOnCluster 1 Failing test : org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures 1 Failing test : org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper 1 Failing test : org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster 1 Failing test : org.apache.hadoop.hbase.security.token.TestGenerateDelegationToken > Zombie Stomping Session > --- > > Key: HBASE-14420 > URL: https://issues.apache.org/jira/browse/HBASE-14420 > Project: HBase > Issue Type: Umbrella > Components: test >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: hangers.txt, none_fix (1).txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt > > > Patch build are now failing most of the time because we are dropping zombies. > I confirm we are doing this on non-apache build boxes too. > Left-over zombies consume resources on build boxes (OOME cannot create native > threads). Having to do multiple test runs in the hope that we can get a > non-zombie-making build or making (arbitrary) rulings that the zombies are > 'not related' is a productivity sink. And so on... > This is an umbrella issue for a zombie stomping session that started earlier > this week. Will hang sub-issues of this one. Am running builds back-to-back > on little cluster to turn out the monsters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14570) Cleanup hanging TestHBaseFsck
[ https://issues.apache.org/jira/browse/HBASE-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952091#comment-14952091 ] stack commented on HBASE-14570: --- I think the history shows whether a test passed or failed, not whether it hung (it could report success but might not die). The last time this test hung was here: Fetching https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/15925/consoleText > Cleanup hanging TestHBaseFsck > - > > Key: HBASE-14570 > URL: https://issues.apache.org/jira/browse/HBASE-14570 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack > > This one hangs regularly. Let me at least add timeouts. Looking in log, a > bunch of tests are potentially hanging tests since they don't see to clean up > after themselves. Will start watching and just disable likely candidates > unless someone wants to have a go at fixing this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14406) The dataframe datasource filter is wrong, and will result in data loss or unexpected behavior
[ https://issues.apache.org/jira/browse/HBASE-14406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952134#comment-14952134 ] Ted Malaska commented on HBASE-14406: - Sorry Hadoop World took me out of commition for awhile, but I'm back now. I looked at the review board. I don't see anything else to add. What else has to happen to close this jira out. I have the whole week to focus on this jira. > The dataframe datasource filter is wrong, and will result in data loss or > unexpected behavior > - > > Key: HBASE-14406 > URL: https://issues.apache.org/jira/browse/HBASE-14406 > Project: HBase > Issue Type: Bug > Components: spark >Affects Versions: 2.0.0 >Reporter: Zhan Zhang >Assignee: Ted Malaska >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-14406.1.patch, HBASE-14406.2.patch, > HBASE-14406.3.patch > > > Following condition will result in the same filter. It will have data loss > with the current filter construction. > col1 > 4 && col2 < 3 > col1 > 4 || col2 < 3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952049#comment-14952049 ] Hudson commented on HBASE-14583: SUCCESS: Integrated in HBase-1.3-IT #225 (See [https://builds.apache.org/job/HBase-1.3-IT/225/]) Revert "HBASE-14583 Enabled client-side metrics by default" Caused (stack: rev cab158173d34e93c5cb3dee1f42126b37aa7ac7b) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952067#comment-14952067
]
Hudson commented on HBASE-14581:
FAILURE: Integrated in HBase-0.98 #1149 (See
[https://builds.apache.org/job/HBase-0.98/1149/])
HBASE-14581 Znode cleanup throws auth exception in secure mode (apurtell: rev
aaf7e22461eb1a2720f746cb86f9fc1afc570198)
* bin/hbase-daemon.sh
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952033#comment-14952033 ] stack commented on HBASE-14583: --- Let me revert since a small change in master and branch-1 only. > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952073#comment-14952073 ] Hudson commented on HBASE-14583: FAILURE: Integrated in HBase-1.3 #251 (See [https://builds.apache.org/job/HBase-1.3/251/]) Revert "HBASE-14583 Enabled client-side metrics by default" Caused (stack: rev cab158173d34e93c5cb3dee1f42126b37aa7ac7b) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952105#comment-14952105 ] Hudson commented on HBASE-14567: FAILURE: Integrated in HBase-TRUNK #6895 (See [https://builds.apache.org/job/HBase-TRUNK/6895/]) HBASE-14567 Tuneup hanging test TestMobCompactor and TestMobSweeper; (stack: rev 587f5bc11f9d5d37557baf36c7df110af860a95c) * hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Fix For: 2.0.0 > > Attachments: 14567v2.txt, HBASE-14567-addendum-v4.patch, > HBASE-14567-v3.patch, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952106#comment-14952106 ] Hudson commented on HBASE-14583: FAILURE: Integrated in HBase-TRUNK #6895 (See [https://builds.apache.org/job/HBase-TRUNK/6895/]) Revert "HBASE-14583 Enabled client-side metrics by default" Causes (stack: rev e37e72c1601cc2006b2587f15e5ba1f790dff64b) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952121#comment-14952121
]
Hudson commented on HBASE-14578:
SUCCESS: Integrated in HBase-1.1 #700 (See
[https://builds.apache.org/job/HBase-1.1/700/])
HBASE-14578 URISyntaxException during snapshot restore for table with (tedyu:
rev f818d84423bdda825ee915af2f4af74371a890b7)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952036#comment-14952036
]
Ted Yu commented on HBASE-14578:
I stopped at branch-1 because of the reproducible TestShell failure.
Feel free to commit across the branches.
If you're busy, I can get to this Monday.
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.3.0, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter
[jira] [Updated] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14578:
---
Resolution: Fixed
Fix Version/s: 1.1.3
1.0.3
1.2.0
Status: Resolved (was: Patch Available)
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter regions contain the
[jira] [Commented] (HBASE-14557) MapReduce WALPlayer issue with NoTagsKeyValue
[
https://issues.apache.org/jira/browse/HBASE-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952072#comment-14952072
]
Jerry He commented on HBASE-14557:
--
+1
> MapReduce WALPlayer issue with NoTagsKeyValue
> -
>
> Key: HBASE-14557
> URL: https://issues.apache.org/jira/browse/HBASE-14557
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Jerry He
>Assignee: Anoop Sam John
> Attachments: HBASE-14557.patch, HBASE-14557.patch
>
>
> Running MapReduce WALPlayer to convert WAL into HFiles:
> {noformat}
> 15/10/05 20:28:08 INFO mapred.JobClient: Task Id :
> attempt_201508031611_0029_m_00_0, Status : FAILED
> java.io.IOException: Type mismatch in value from map: expected
> org.apache.hadoop.hbase.KeyValue, recieved
> org.apache.hadoop.hbase.NoTagsKeyValue
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:997)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:96)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at
> java.security.AccessController.doPrivileged(AccessController.java:369)
> at javax.security.auth.Subject.doAs(Subject.java:572)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952096#comment-14952096
]
Hudson commented on HBASE-14581:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1101 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1101/])
HBASE-14581 Znode cleanup throws auth exception in secure mode (apurtell: rev
aaf7e22461eb1a2720f746cb86f9fc1afc570198)
* bin/hbase-daemon.sh
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952110#comment-14952110
]
Hudson commented on HBASE-14578:
FAILURE: Integrated in HBase-1.0 #1076 (See
[https://builds.apache.org/job/HBase-1.0/1076/])
HBASE-14578 URISyntaxException during snapshot restore for table with (tedyu:
rev e42f46ab8344adf5f98d633f98dc84579481b0eb)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952124#comment-14952124
]
Hudson commented on HBASE-14578:
FAILURE: Integrated in HBase-1.3 #252 (See
[https://builds.apache.org/job/HBase-1.3/252/])
HBASE-14578 URISyntaxException during snapshot restore for table with (tedyu:
rev cb3e734ff0c2e34324dcb977678917596180ba1c)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at
[jira] [Commented] (HBASE-14268) Improve KeyLocker
[
https://issues.apache.org/jira/browse/HBASE-14268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952063#comment-14952063
]
stack commented on HBASE-14268:
---
Ok if I commit [~ikeda]?
> Improve KeyLocker
> -
>
> Key: HBASE-14268
> URL: https://issues.apache.org/jira/browse/HBASE-14268
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 14268-V5.patch, HBASE-14268-V2.patch,
> HBASE-14268-V3.patch, HBASE-14268-V4.patch, HBASE-14268-V5.patch,
> HBASE-14268-V5.patch, HBASE-14268-V6.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268.patch, KeyLockerIncrKeysPerformance.java,
> KeyLockerPerformance.java, ReferenceTestApp.java
>
>
> 1. In the implementation of {{KeyLocker}} it uses atomic variables inside a
> synchronized block, which doesn't make sense. Moreover, logic inside the
> synchronized block is not trivial so that it makes less performance in heavy
> multi-threaded environment.
> 2. {{KeyLocker}} gives an instance of {{RentrantLock}} which is already
> locked, but it doesn't follow the contract of {{ReentrantLock}} because you
> are not allowed to freely invoke lock/unlock methods under that contract.
> That introduces a potential risk; Whenever you see a variable of the type
> {{RentrantLock}}, you should pay attention to what the included instance is
> coming from.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952119#comment-14952119
]
Hudson commented on HBASE-14578:
FAILURE: Integrated in HBase-1.2-IT #200 (See
[https://builds.apache.org/job/HBase-1.2-IT/200/])
HBASE-14578 URISyntaxException during snapshot restore for table with (tedyu:
rev 9117141ebb7b585921e2b17d1b82cb390576b6a2)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952122#comment-14952122
]
Hudson commented on HBASE-14578:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1102 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1102/])
HBASE-14578 URISyntaxException during snapshot restore for table with (tedyu:
rev e6517d837eba86f66cdcc4b942938a6191e88551)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952125#comment-14952125
]
Hudson commented on HBASE-14578:
FAILURE: Integrated in HBase-0.98 #1150 (See
[https://builds.apache.org/job/HBase-0.98/1150/])
HBASE-14578 URISyntaxException during snapshot restore for table with (tedyu:
rev e6517d837eba86f66cdcc4b942938a6191e88551)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
>
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952035#comment-14952035 ] stack commented on HBASE-14583: --- Reverted temporarily from branch-1 and master branches. > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14564) Fix and reenable TestHFileOutputFormat2
[ https://issues.apache.org/jira/browse/HBASE-14564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952064#comment-14952064 ] stack commented on HBASE-14564: --- That sounds reasonable [~chenheng] Do we have another test suite that does increment load? Could go there? > Fix and reenable TestHFileOutputFormat2 > --- > > Key: HBASE-14564 > URL: https://issues.apache.org/jira/browse/HBASE-14564 > Project: HBase > Issue Type: Bug >Reporter: stack > > Was disabled as part of the zombie stomping session over in HBASE-14420. Test > needs a rewrite and/or being split up. Scope of the test needs to be shrunk > and made more targeted. Currently it does everything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952123#comment-14952123
]
Hudson commented on HBASE-14578:
SUCCESS: Integrated in HBase-1.3-IT #226 (See
[https://builds.apache.org/job/HBase-1.3-IT/226/])
HBASE-14578 URISyntaxException during snapshot restore for table with (tedyu:
rev cb3e734ff0c2e34324dcb977678917596180ba1c)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951711#comment-14951711
]
Hadoop QA commented on HBASE-14578:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765968/HBASE-14578-branch-1.patch
against branch-1 branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765968
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15947//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15947//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15947//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15947//console
This message is automatically generated.
> URISyntaxException during snapshot restore
> --
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
>
[jira] [Commented] (HBASE-14570) Cleanup hanging TestHBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951710#comment-14951710
]
Heng Chen commented on HBASE-14570:
---
It seems no failures in latest builds
https://builds.apache.org/job/HBase-TRUNK/lastCompletedBuild/testReport/org.apache.hadoop.hbase.util/TestHBaseFsck/history/#
And timeouts seems be added already in {{TestHBaseFsck}}
Is there anything i missed?
> Cleanup hanging TestHBaseFsck
> -
>
> Key: HBASE-14570
> URL: https://issues.apache.org/jira/browse/HBASE-14570
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: stack
>
> This one hangs regularly. Let me at least add timeouts. Looking in log, a
> bunch of tests are potentially hanging tests since they don't see to clean up
> after themselves. Will start watching and just disable likely candidates
> unless someone wants to have a go at fixing this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14581:
---
Fix Version/s: 1.1.3
1.3.0
1.2.0
2.0.0
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14578:
---
Hadoop Flags: Reviewed
Fix Version/s: 1.3.0
2.0.0
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.3.0, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter regions contain the reference of the
> original HFile until compaction happens.
> In RestoreSnapshotHelper (while
[jira] [Commented] (HBASE-13153) Bulk Loaded HFile Replication
[
https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951730#comment-14951730
]
Hadoop QA commented on HBASE-13153:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765969/HBASE-13153-v4.patch
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765969
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 30 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1764 checkstyle errors (more than the master's current 1762 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ new java.lang.String[] { "Entry",
"SourceBaseNamespaceDirPath", "SourceHFileArchiveDirPath", });
+ * {@link LoadIncrementalHFiles#bulkLoadPhase(Table, Connection,
ExecutorService, Deque, Multimap)}
+ HConstants.REPLICATION_BULKLOAD_ENABLE_KEY,
HConstants.REPLICATION_BULKLOAD_ENABLE_DEFAULT))) {
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15948//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15948//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15948//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15948//console
This message is automatically generated.
> Bulk Loaded HFile Replication
> -
>
> Key: HBASE-13153
> URL: https://issues.apache.org/jira/browse/HBASE-13153
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: sunhaitao
>Assignee: Ashish Singhi
> Fix For: 2.0.0
>
> Attachments: HBASE-13153-v1.patch, HBASE-13153-v2.patch,
> HBASE-13153-v3.patch, HBASE-13153-v4.patch, HBASE-13153.patch, HBase Bulk
> Load Replication-v1-1.pdf, HBase Bulk Load Replication-v2.pdf, HBase Bulk
> Load Replication.pdf
>
>
> Currently we plan to use HBase Replication feature to deal with disaster
> tolerance scenario.But we encounter an issue that we will use bulkload very
> frequently,because bulkload bypass write path, and will not generate WAL, so
> the data will not be replicated to backup cluster. It's inappropriate to
> bukload twice both on active cluster and backup cluster. So i advise do some
> modification to bulkload feature to enable bukload to both active cluster and
> backup cluster
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14564) Fix and reenable TestHFileOutputFormat2
[
https://issues.apache.org/jira/browse/HBASE-14564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951733#comment-14951733
]
Heng Chen commented on HBASE-14564:
---
Yeah, {{TestHFileOutputFormat2}} does a lot of things.
IMO testMRIncrementalLoadXXX should be extract to be another test case, because
it is used to test increment load with MR, and it has little relationship with
{{HFileOutputFormat2}}.
And testExcludeAllFromMinorCompaction and testExcludeMinorCompaction should be
extract too. It is used to test HBASE-6901.
> Fix and reenable TestHFileOutputFormat2
> ---
>
> Key: HBASE-14564
> URL: https://issues.apache.org/jira/browse/HBASE-14564
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>
> Was disabled as part of the zombie stomping session over in HBASE-14420. Test
> needs a rewrite and/or being split up. Scope of the test needs to be shrunk
> and made more targeted. Currently it does everything.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951735#comment-14951735
]
Ted Yu commented on HBASE-14578:
TestShell hangs but it has been hanging since at least build #248:
https://builds.apache.org/job/HBase-1.3/jdk=latest1.7,label=Hadoop/248/consoleFull
Planning to integrate soon.
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.3.0, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After
[jira] [Commented] (HBASE-14366) NPE in case visibility expression is not present in labels table during importtsv run
[
https://issues.apache.org/jira/browse/HBASE-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951748#comment-14951748
]
Hudson commented on HBASE-14366:
FAILURE: Integrated in HBase-1.3 #250 (See
[https://builds.apache.org/job/HBase-1.3/250/])
HBASE-14366 NPE in case visibility expression is not present in labels
(anoopsamjohn: rev 90f0644d9a3f334a422244f8de20bb7c1ae4ba76)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TextSortReducer.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithVisibilityLabels.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TsvImporterMapper.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityConstants.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/DefaultVisibilityExpressionResolver.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsCache.java
> NPE in case visibility expression is not present in labels table during
> importtsv run
> -
>
> Key: HBASE-14366
> URL: https://issues.apache.org/jira/browse/HBASE-14366
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Bhupendra Kumar Jain
>Priority: Minor
> Attachments: 0001-HBASE-14366.patch, 0001-HBASE-14366_1.patch,
> HBASE-14366-0.98.patch, HBASE-14366-branch-1.patch, HBASE-14366_2(1).patch,
> HBASE-14366_2.patch
>
>
> Below exception is shown in logs if visibility expression is not present in
> labels table during importtsv run. Appropriate exception / message should be
> logged for the user to take further action.
> {code}
> WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child :
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.mapreduce.DefaultVisibilityExpressionResolver$1.getLabelOrdinal(DefaultVisibilityExpressionResolver.java:127)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityUtils.getLabelOrdinals(VisibilityUtils.java:358)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityUtils.createVisibilityExpTags(VisibilityUtils.java:323)
> at
> org.apache.hadoop.hbase.mapreduce.DefaultVisibilityExpressionResolver.createVisibilityExpTags(DefaultVisibilityExpressionResolver.java:137)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.populatePut(TsvImporterMapper.java:205)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:165)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:1)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-14567: - Attachment: HBASE-14567-v3.patch Remove TestMobSweeper, and merge cases into one in TestMobCompactor. Now the TestMobCompactor takes around 140 seconds. > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: 14567v2.txt, HBASE-14567-v3.patch, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14578:
---
Summary: URISyntaxException during snapshot restore for table with user
defined namespace (was: URISyntaxException during snapshot restore)
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter regions contain the reference of the
> original HFile until
[jira] [Commented] (HBASE-14366) NPE in case visibility expression is not present in labels table during importtsv run
[
https://issues.apache.org/jira/browse/HBASE-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951729#comment-14951729
]
Hudson commented on HBASE-14366:
SUCCESS: Integrated in HBase-1.3-IT #224 (See
[https://builds.apache.org/job/HBase-1.3-IT/224/])
HBASE-14366 NPE in case visibility expression is not present in labels
(anoopsamjohn: rev 90f0644d9a3f334a422244f8de20bb7c1ae4ba76)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/DefaultVisibilityExpressionResolver.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsCache.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TsvImporterMapper.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithVisibilityLabels.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TextSortReducer.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityConstants.java
> NPE in case visibility expression is not present in labels table during
> importtsv run
> -
>
> Key: HBASE-14366
> URL: https://issues.apache.org/jira/browse/HBASE-14366
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Bhupendra Kumar Jain
>Priority: Minor
> Attachments: 0001-HBASE-14366.patch, 0001-HBASE-14366_1.patch,
> HBASE-14366-0.98.patch, HBASE-14366-branch-1.patch, HBASE-14366_2(1).patch,
> HBASE-14366_2.patch
>
>
> Below exception is shown in logs if visibility expression is not present in
> labels table during importtsv run. Appropriate exception / message should be
> logged for the user to take further action.
> {code}
> WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child :
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.mapreduce.DefaultVisibilityExpressionResolver$1.getLabelOrdinal(DefaultVisibilityExpressionResolver.java:127)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityUtils.getLabelOrdinals(VisibilityUtils.java:358)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityUtils.createVisibilityExpTags(VisibilityUtils.java:323)
> at
> org.apache.hadoop.hbase.mapreduce.DefaultVisibilityExpressionResolver.createVisibilityExpTags(DefaultVisibilityExpressionResolver.java:137)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.populatePut(TsvImporterMapper.java:205)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:165)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:1)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[
https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951732#comment-14951732
]
Hadoop QA commented on HBASE-14567:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765976/HBASE-14567-v3.patch
against master branch at commit 4969879df5f6d1d9a2774236cdee1710694ddf6a.
ATTACHMENT ID: 12765976
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15949//console
This message is automatically generated.
> Tuneup hanging test TestMobCompactor and TestMobSweeper
> ---
>
> Key: HBASE-14567
> URL: https://issues.apache.org/jira/browse/HBASE-14567
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: stack
>Assignee: Jingcheng Du
> Attachments: 14567v2.txt, HBASE-14567-v3.patch, HBASE-14567.patch
>
>
> These tests hang with some regularity. Please take a look
> [~jingcheng...@intel.com] since I believe they are yours. If not, let me know
> and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks.
> They hung here most recently:
> https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[
https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951745#comment-14951745
]
Ted Yu commented on HBASE-14583:
Turns out this is not so trivial.
In branch-1, TestShell hung.
Below was the commit which went in before this patch in branch-1:
{code}
commit ced679423084c208fa1e09723d5f61c5cab52408
Author: anoopsjohn
Date: Fri Oct 9 20:55:22 2015 +0530
HBASE-14525 Append and increment operation throws NullPointerException on
non-existing column families.(Abhishek)
{code}
I checked out the above commit and ran TestShell which passed.
Jenkins shows that TestShell hung when this patch went in:
https://builds.apache.org/job/HBase-1.3/jdk=latest1.7,label=Hadoop/248/consoleFull
{code}
Running org.apache.hadoop.hbase.client.TestShell
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 92.298 sec <<<
FAILURE! - in org.apache.hadoop.hbase.client.TestReplicationShell
testRunShellTests(org.apache.hadoop.hbase.client.TestReplicationShell) Time
elapsed: 91.935 sec <<< ERROR!
org.jruby.embed.EvalFailedException: (RuntimeError) Shell unit tests failed.
Check output file for details.
at
org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:136)
at
org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1263)
at
org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1308)
at
org.apache.hadoop.hbase.client.TestReplicationShell.testRunShellTests(TestReplicationShell.java:34)
Caused by: org.jruby.exceptions.RaiseException: (RuntimeError) Shell unit tests
failed. Check output file for details.
at (Anonymous).(root)(src/test/ruby/tests_runner.rb:84)
{code}
> Enabled client-side metrics by default
> --
>
> Key: HBASE-14583
> URL: https://issues.apache.org/jira/browse/HBASE-14583
> Project: HBase
> Issue Type: Task
> Components: Client, Operability, Performance
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 14583.00.branch-1.patch, 14583.00.patch
>
>
> Enabling this feature by default for master and branch-1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-14567: - Attachment: HBASE-14567-addendum-v4.patch Sorry for last patch that cannot be merged. Pull code and re-generate the patch v4. > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: 14567v2.txt, HBASE-14567-addendum-v4.patch, > HBASE-14567-v3.patch, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14268) Improve KeyLocker
[
https://issues.apache.org/jira/browse/HBASE-14268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951632#comment-14951632
]
Hadoop QA commented on HBASE-14268:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765955/HBASE-14268-V7.patch
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765955
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15941//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15941//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15941//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15941//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15941//console
This message is automatically generated.
> Improve KeyLocker
> -
>
> Key: HBASE-14268
> URL: https://issues.apache.org/jira/browse/HBASE-14268
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 14268-V5.patch, HBASE-14268-V2.patch,
> HBASE-14268-V3.patch, HBASE-14268-V4.patch, HBASE-14268-V5.patch,
> HBASE-14268-V5.patch, HBASE-14268-V6.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268.patch, KeyLockerIncrKeysPerformance.java,
> KeyLockerPerformance.java, ReferenceTestApp.java
>
>
> 1. In the implementation of {{KeyLocker}} it uses atomic variables inside a
> synchronized block, which doesn't make sense. Moreover, logic inside the
> synchronized block is not trivial so that it makes less performance in heavy
> multi-threaded environment.
> 2. {{KeyLocker}} gives an instance of {{RentrantLock}} which is already
> locked, but it doesn't follow the contract of {{ReentrantLock}} because you
> are not allowed to freely invoke lock/unlock methods under that contract.
> That introduces a potential risk; Whenever you see a variable of the type
> {{RentrantLock}}, you should pay attention to what the included instance is
> coming from.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14586) Remove extraneous ${argLine} references in pom
[
https://issues.apache.org/jira/browse/HBASE-14586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951641#comment-14951641
]
Hadoop QA commented on HBASE-14586:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765967/hbase-14586.001.patch
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765967
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15946//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15946//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15946//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15946//console
This message is automatically generated.
> Remove extraneous ${argLine} references in pom
> --
>
> Key: HBASE-14586
> URL: https://issues.apache.org/jira/browse/HBASE-14586
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Attachments: hbase-14586.001.patch, hbase-14586.001.patch
>
>
> The pom.xml has a line like this for the Surefire argLine, which has an extra
> ${argLine} reference. Recommend changes like this:
> {noformat}
> -${hbase-surefire.argLine} ${argLine}
> +${hbase-surefire.argLine}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14582) Regionserver status webpage bucketcache list can become huge
[
https://issues.apache.org/jira/browse/HBASE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951640#comment-14951640
]
Hadoop QA commented on HBASE-14582:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765958/14582.txt
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765958
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15943//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15943//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15943//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15943//console
This message is automatically generated.
> Regionserver status webpage bucketcache list can become huge
>
>
> Key: HBASE-14582
> URL: https://issues.apache.org/jira/browse/HBASE-14582
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: James Hartshorn
>Assignee: stack
>Priority: Trivial
> Attachments: 14582.txt, Screen Shot 2015-10-09 at 8.00.37 PM.png,
> Screen Shot 2015-10-09 at 8.00.37 PM.png
>
>
> The regionserver status page, such as http://127.0.0.1:60030/rs-status always
> downloads information about every bucket in the cache. In some cases this
> can be hundreds of thousands of buckets, causing megabytes of info to need to
> be downloaded and significant browser instability and memory usage:
> wc -l HBase-Region-Server-hb22.html
> 2010116 HBase-Region-Server-hb22.html
> ls -lah HBase-Region-Server-hb22.html
> 32M Oct 6 19:23 HBase-Region-Server-hb22.html
> Firefox "about:memory":
> 1,330.18 MB (48.22%) -- top(http://hb22:60030/rs-status#bc_l2, id=2010)
> 1,329.61 MB (48.20%) -- active/window(http://hb22:60030/rs-status#bc_l2)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14549) Simplify scanner stack reset logic
[ https://issues.apache.org/jira/browse/HBASE-14549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951645#comment-14951645 ] Lars Hofhansl commented on HBASE-14549: --- OK... I get it now. In KeyValueHeap we always have the top scanner removed from the heap and store in this.current. In the case of a scanner stack reset, which used to be the top scanner may no longer be and we would have to put it back on the heap and pull a new top scanner out, but we have no way of knowing that. I.e. the top StoreScanner in the RegionScanner's KeyValueHeap might have changed without notice. This all looks a bit brittle. What if _other_ StoreScanners (other than the top scanner) change their peek-element? We'd have to remove and re-add to the RegionScanner's heap as well... I think. Seems the correct way would be to reset the RegionScanner stack whenever any of the stores have been compacted. That would also naturally coarsen the lock to RegionScanner, but the RegionScanner may not have all the information to reset all StoreScanners. Tricky... > Simplify scanner stack reset logic > -- > > Key: HBASE-14549 > URL: https://issues.apache.org/jira/browse/HBASE-14549 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Attachments: 14549-0.98.txt > > > Looking at the code, I find that the logic is unnecessarily complex. > We indicate in updateReaders that the scanner stack needs to be reset. Then > almost all store scanner (and derived classes) methods need to check and > actually reset the scanner stack. > Compaction are rare, we should reset the scanner stack in update readers, and > hence avoid needing to check in all methods. > Patch forthcoming. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14366) NPE in case visibility expression is not present in labels table during importtsv run
[
https://issues.apache.org/jira/browse/HBASE-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951762#comment-14951762
]
Hudson commented on HBASE-14366:
FAILURE: Integrated in HBase-TRUNK #6893 (See
[https://builds.apache.org/job/HBase-TRUNK/6893/])
HBASE-14366 NPE in case visibility expression is not present in labels
(anoopsamjohn: rev 4969879df5f6d1d9a2774236cdee1710694ddf6a)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TextSortReducer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsCache.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityConstants.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TsvImporterMapper.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/DefaultVisibilityExpressionResolver.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithVisibilityLabels.java
> NPE in case visibility expression is not present in labels table during
> importtsv run
> -
>
> Key: HBASE-14366
> URL: https://issues.apache.org/jira/browse/HBASE-14366
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Bhupendra Kumar Jain
>Priority: Minor
> Attachments: 0001-HBASE-14366.patch, 0001-HBASE-14366_1.patch,
> HBASE-14366-0.98.patch, HBASE-14366-branch-1.patch, HBASE-14366_2(1).patch,
> HBASE-14366_2.patch
>
>
> Below exception is shown in logs if visibility expression is not present in
> labels table during importtsv run. Appropriate exception / message should be
> logged for the user to take further action.
> {code}
> WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child :
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.mapreduce.DefaultVisibilityExpressionResolver$1.getLabelOrdinal(DefaultVisibilityExpressionResolver.java:127)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityUtils.getLabelOrdinals(VisibilityUtils.java:358)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityUtils.createVisibilityExpTags(VisibilityUtils.java:323)
> at
> org.apache.hadoop.hbase.mapreduce.DefaultVisibilityExpressionResolver.createVisibilityExpTags(DefaultVisibilityExpressionResolver.java:137)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.populatePut(TsvImporterMapper.java:205)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:165)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:1)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-14567: - Attachment: (was: HBASE-14567-addendum-v4.patch) > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: 14567v2.txt, HBASE-14567-v3.patch, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[
https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951779#comment-14951779
]
Hadoop QA commented on HBASE-14567:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765977/HBASE-14567-addendum-v4.patch
against master branch at commit 1b66ea71b69e8b24ba7ecb9ba6548d775777f417.
ATTACHMENT ID: 12765977
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15950//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15950//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15950//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15950//console
This message is automatically generated.
> Tuneup hanging test TestMobCompactor and TestMobSweeper
> ---
>
> Key: HBASE-14567
> URL: https://issues.apache.org/jira/browse/HBASE-14567
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: stack
>Assignee: Jingcheng Du
> Attachments: 14567v2.txt, HBASE-14567-addendum-v4.patch,
> HBASE-14567-v3.patch, HBASE-14567.patch
>
>
> These tests hang with some regularity. Please take a look
> [~jingcheng...@intel.com] since I believe they are yours. If not, let me know
> and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks.
> They hung here most recently:
> https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13153) Bulk Loaded HFile Replication
[ https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13153: -- Attachment: HBASE-13153-v5.patch Patch fixes the long lines, same is reported by check style warnings. > Bulk Loaded HFile Replication > - > > Key: HBASE-13153 > URL: https://issues.apache.org/jira/browse/HBASE-13153 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: sunhaitao >Assignee: Ashish Singhi > Fix For: 2.0.0 > > Attachments: HBASE-13153-v1.patch, HBASE-13153-v2.patch, > HBASE-13153-v3.patch, HBASE-13153-v4.patch, HBASE-13153-v5.patch, > HBASE-13153.patch, HBase Bulk Load Replication-v1-1.pdf, HBase Bulk Load > Replication-v2.pdf, HBase Bulk Load Replication.pdf > > > Currently we plan to use HBase Replication feature to deal with disaster > tolerance scenario.But we encounter an issue that we will use bulkload very > frequently,because bulkload bypass write path, and will not generate WAL, so > the data will not be replicated to backup cluster. It's inappropriate to > bukload twice both on active cluster and backup cluster. So i advise do some > modification to bulkload feature to enable bukload to both active cluster and > backup cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-14567: - Attachment: HBASE-14567-addendum-v4.patch Upload the new patch v4 1. Merge part of cases into one. 2. Optimize the phase of preparing data. Now TestMobCompactor takes less than 90 seconds. > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: 14567v2.txt, HBASE-14567-addendum-v4.patch, > HBASE-14567-v3.patch, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[
https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951804#comment-14951804
]
Hadoop QA commented on HBASE-14567:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765978/HBASE-14567-addendum-v4.patch
against master branch at commit 1b66ea71b69e8b24ba7ecb9ba6548d775777f417.
ATTACHMENT ID: 12765978
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15951//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15951//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15951//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15951//console
This message is automatically generated.
> Tuneup hanging test TestMobCompactor and TestMobSweeper
> ---
>
> Key: HBASE-14567
> URL: https://issues.apache.org/jira/browse/HBASE-14567
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: stack
>Assignee: Jingcheng Du
> Attachments: 14567v2.txt, HBASE-14567-addendum-v4.patch,
> HBASE-14567-v3.patch, HBASE-14567.patch
>
>
> These tests hang with some regularity. Please take a look
> [~jingcheng...@intel.com] since I believe they are yours. If not, let me know
> and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks.
> They hung here most recently:
> https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reopened HBASE-14583: > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951811#comment-14951811
]
Matt Warhaftig commented on HBASE-14584:
Thanks [~ndimiduk] and [~stack].
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig reassigned HBASE-14584:
--
Assignee: Matt Warhaftig
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951815#comment-14951815
]
Matt Warhaftig commented on HBASE-14584:
Oh, I see. The commit was just for disabling tests. I'll take a look at fixing.
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore for table with user defined namespace
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951823#comment-14951823
]
Hudson commented on HBASE-14578:
FAILURE: Integrated in HBase-TRUNK #6894 (See
[https://builds.apache.org/job/HBase-TRUNK/6894/])
HBASE-14578 URISyntaxException during snapshot restore for table with (tedyu:
rev f1359129122c96ccc4af1b7d0bd9e692ab48ee65)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
> URISyntaxException during snapshot restore for table with user defined
> namespace
>
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.3.0, 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch, HBASE-14578-branch-1.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at
[jira] [Commented] (HBASE-14581) Znode cleanup throws auth exception in secure mode
[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951822#comment-14951822
]
Hudson commented on HBASE-14581:
FAILURE: Integrated in HBase-TRUNK #6894 (See
[https://builds.apache.org/job/HBase-TRUNK/6894/])
HBASE-14581 Znode cleanup throws auth exception in secure mode (tedyu: rev
1b66ea71b69e8b24ba7ecb9ba6548d775777f417)
* bin/hbase-daemon.sh
> Znode cleanup throws auth exception in secure mode
> --
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 4
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
