[jira] [Updated] (HBASE-10357) Failover RPC's for scans
[ https://issues.apache.org/jira/browse/HBASE-10357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-10357: Attachment: 10357-3.2.txt This is a more complete patch. The way this works is that ScannerCallable mechanics is used to handle the replica logic (and the layer above, ClientScanner, is not aware of the replica mechanics that much). The logic is that when the client latches on to a replica (default/primary or not), it tries to get the maximum data out of it, and when there is a failure, it switches to some other replica... > Failover RPC's for scans > > > Key: HBASE-10357 > URL: https://issues.apache.org/jira/browse/HBASE-10357 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Enis Soztutar > Fix For: 0.99.0 > > Attachments: 10357-1.txt, 10357-2.txt, 10357-3.2.txt, 10357-3.txt > > > This is extension of HBASE-10355 to add failover support for scans. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11048) Support setting custom priority per client RPC
[ https://issues.apache.org/jira/browse/HBASE-11048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976439#comment-13976439 ] Jonathan Hsieh commented on HBASE-11048: [~mbertozzi] should take a look -- he recently posted a design doc and some code for some basic rpc scheduling for QoS purposes. > Support setting custom priority per client RPC > -- > > Key: HBASE-11048 > URL: https://issues.apache.org/jira/browse/HBASE-11048 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.99.0, 0.98.2 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.99.0, 0.98.2 > > Attachments: hbase-11048-trunk-v0.patch > > > Servers have the ability to handle custom rpc priority levels, but currently > we are only using it to differentiate META/ROOT updates from replication and > other 'priority' updates (as specified by annotation tags per RS method). > However, some clients need the ability to create custom handlers (e.g. > PHOENIX-938) which can really only be cleanly tied together to requests by > the request priority. The disconnect is in that there is no way for the > client to overwrite the priority per table - the PayloadCarryingRpcController > will always just set priority per ROOT/META and otherwise just use the > generic priority. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10974) Improve DBEs read performance by avoiding byte array deep copies for key[] and value[]
[ https://issues.apache.org/jira/browse/HBASE-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976405#comment-13976405 ] Lars Hofhansl commented on HBASE-10974: --- I love the idea of pushing the decision to make a copy to the upper layers. That way we can delay extra copies to the point where they are actually needed (and maybe avoid the copies in some cases). > Improve DBEs read performance by avoiding byte array deep copies for key[] > and value[] > -- > > Key: HBASE-10974 > URL: https://issues.apache.org/jira/browse/HBASE-10974 > Project: HBase > Issue Type: Improvement > Components: Scanners >Affects Versions: 0.99.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-10974_1.patch > > > As part of HBASE-10801, we tried to reduce the copy of the value [] in > forming the KV from the DBEs. > The keys required copying and this was restricting us in using Cells and > always wanted to copy to be done. > The idea here is to replace the key byte[] as ByteBuffer and create a > consecutive stream of the keys (currently the same byte[] is used and hence > the copy). Use offset and length to track this key bytebuffer. > The copy of the encoded format to normal Key format is definitely needed and > can't be avoided but we could always avoid the deep copy of the bytes to form > a KV and thus use cells effectively. Working on a patch, will post it soon. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10929) Change ScanQueryMatcher to use Cells instead of KeyValue.
[ https://issues.apache.org/jira/browse/HBASE-10929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976403#comment-13976403 ] Lars Hofhansl commented on HBASE-10929: --- Belated +1. Nice change. Had been thinking to remove the manual ripping-apart of the KVs done by SQM, never got to it. Glad to see that is has no performance impact... Although... Speaking of the numbers in HBASE-10801: With a large value portion (1000 bytes) we're obviously not going to see performance changes in small keys. > Change ScanQueryMatcher to use Cells instead of KeyValue. > - > > Key: HBASE-10929 > URL: https://issues.apache.org/jira/browse/HBASE-10929 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-10929.patch, HBASE-10929_1.patch, > HBASE-10929_2.patch, HBASE-10929_3.patch, HBASE-10929_4.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11023) Port HBASE-10488 "'mvn site' is broken due to org.apache.jasper.JspC not found" to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976360#comment-13976360 ] Konstantin Boudnik commented on HBASE-11023: Yes, using release tarball in the old-fashioned Bigtop way. BTW, are you saying that the build arguments got changed between 0.94 (what we've built in 0.7.0) and now and I have to update the bigtop build? > Port HBASE-10488 "'mvn site' is broken due to org.apache.jasper.JspC not > found" to 0.98 > --- > > Key: HBASE-11023 > URL: https://issues.apache.org/jira/browse/HBASE-11023 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.98.2 > > Attachments: 11023.txt > > > This is to port the fix (mostly in hbase-thrift/pom.xml) for the site target. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11001) Shell support for granting cell permissions for testing
[
https://issues.apache.org/jira/browse/HBASE-11001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976352#comment-13976352
]
Hadoop QA commented on HBASE-11001:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641166/HBASE-11001.patch
against trunk revision .
ATTACHMENT ID: 12641166
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+raise ArgumentError, "Arguments should be a hash. Failed to parse
#{args.inspect}, #{args.class}"
+ raise(ArgumentError, "Authorizations must be a Array type") unless
authorizations.kind_of?(Array)
+ raise(ArgumentError, "Permissions are not of String type") unless
permissions.kind_of?(String)
+ raise(ArgumentError, "Table name is not of String type") unless
table_name.kind_of?(String)
+ raise(ArgumentError, "Qualifier is not of String type") unless
qualifier.kind_of?(String)
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9356//console
This message is automatically generated.
> Shell support for granting cell permissions for testing
> ---
>
> Key: HBASE-11001
> URL: https://issues.apache.org/jira/browse/HBASE-11001
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 11001.patch, HBASE-11001.patch
>
>
> For testing purposes it would be useful if the shell can support a simple
> syntax for adding cell ACLs to existing cells. Consider a combination of
> current 'grant' and 'scan' commands.
> {noformat}
> grant , ,
> {noformat}
> where is a string, a table name, optionally prefixed with a namespace
> where is a Hash type that maps permissions to users, for
> example: \\
> - { "user1" => "RW", "user2" => "R", "@group1" => "R" }
> where is a Hash type containing a scanner specification, for example
> (borrowed from scan.rb): \\
> - { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
> - { COLUMNS => 'c1', TIMERANGE => [123, 456] }
> - { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
> (QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10929) Change ScanQueryMatcher to use Cells instead of KeyValue.
[ https://issues.apache.org/jira/browse/HBASE-10929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-10929: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks for the review Stack. > Change ScanQueryMatcher to use Cells instead of KeyValue. > - > > Key: HBASE-10929 > URL: https://issues.apache.org/jira/browse/HBASE-10929 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-10929.patch, HBASE-10929_1.patch, > HBASE-10929_2.patch, HBASE-10929_3.patch, HBASE-10929_4.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11025) Infrastructure for pluggable consensus service
[ https://issues.apache.org/jira/browse/HBASE-11025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976339#comment-13976339 ] Mikhail Antonov commented on HBASE-11025: - Thanks for review [~stack] ! bq. Do we need to stop the consensus provider shutting down the Master or RegionServer? Stopping consensus provider should handle things like quorum membership change depending on the impl, so ZK provider (at least for now) shouldn't have to do anything, but I'm certain we need to receive callback when server shuts down. bq. You need it committed Mikhail Antonov to make progress? That would be very good as without it all other patches have to carry this portion of code, which makes them bigger than they need to be. Totally agree that having more opinions/reviews would be good. bq. HRS is a main entry point referenced not only in java code but in lots of scripts; changing its constructor, it will be hard to ensure all references have also been changed. True, I made a changes in quite a few places (including HRS commanline runner, test util classes and actual tests) and got all tests (those which hadoop-qa runs on patch submission) passing, but I'm not sure that it covers 100% of possible places where this constructor may be invoked (ruby scripts?..something else?). > Infrastructure for pluggable consensus service > -- > > Key: HBASE-11025 > URL: https://issues.apache.org/jira/browse/HBASE-11025 > Project: HBase > Issue Type: Sub-task > Components: Zookeeper >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Attachments: HBASE-11025(add param in HRS ctor).patch, > HBASE-11025(add param in HRS ctor).patch, HBASE-11025.patch > > > Related to HBASE-10915. > In this jira I will extract the changed for property changes, factory and > consensus provider interface (+ZK implementation), as it's going to be > required by other subtasks of HBASE-10909. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11032) Replace deprecated methods in FileSystem with their replacements
[
https://issues.apache.org/jira/browse/HBASE-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976337#comment-13976337
]
Hudson commented on HBASE-11032:
FAILURE: Integrated in HBase-TRUNK #5105 (See
[https://builds.apache.org/job/HBase-TRUNK/5105/])
HBASE-11032 Replace deprecated methods in FileSystem with their replacements
(Gustavo) (tedyu: rev 1588979)
*
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CoprocessorClassLoader.java
*
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/backup/example/LongTermArchivingHFileCleaner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HLogInputFormat.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSRegionScanner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/backup/example/TestZooKeeperTableArchiveClient.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestFileLink.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/handler/TestTableDeleteFamilyHandler.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/migration/TestUpgradeTo96.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/HFileReadWriteTest.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/rest/PerformanceEvaluation.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/HFileArchiveTestingUtil.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestFSVisitor.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
> Replace deprecated methods in FileSystem with their replacements
>
>
> Key: HBASE-11032
> URL: https://issues.apache.org/jira/browse/HBASE-11032
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-11032-v1.patch, HBASE-11032.patch
>
>
> FileStatus#isDir() is deprecated.
> FileStatus#isDirectory() should be called instead.
> Here is the list of deprecated methods in FileSystem :
> {code}
> public String getName()
> public static FileSystem getNamed(String name, Configuration conf)
> public FSDataOutputStream createNonRecursive(Path f,
> boolean overwrite,
> int bufferSize, short replication, long blockSize,
> Progressable progress) throws IOException {
>public FSDataOutputStream createNonRecursive(Path f, FsPermission
> permission,
>boolean overwrite, int bufferSize, short replication, long blockSize,
>Progressable progress) throws IOException {
> public short getReplication(Path src) throws IOException {
> public boolean delete(Path f) throws IOException {
> public long getLength(Path f) throws IOException {
> public long getBlockSize(Path f) thr
[jira] [Commented] (HBASE-10950) Add a configuration point for MaxVersion of Column Family
[ https://issues.apache.org/jira/browse/HBASE-10950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976319#comment-13976319 ] Andrew Purtell commented on HBASE-10950: Looks ok, or how about for description: "New column family descriptors will use this value as the default number of versions to keep." ? > Add a configuration point for MaxVersion of Column Family > -- > > Key: HBASE-10950 > URL: https://issues.apache.org/jira/browse/HBASE-10950 > Project: HBase > Issue Type: Improvement > Components: Admin >Affects Versions: 0.98.0, 0.96.0 >Reporter: Demai Ni >Assignee: Enoch Hsu > Fix For: 0.99.0, 0.98.2, 0.96.3 > > Attachments: HBASE_10950.patch, HBASE_10950_v2.patch, > HBASE_10950_v3.patch > > > Starting on 0.96.0. HColumnDescriptor.DEFAULT_VERSIONS change to 1 from 3. > So a columnfamily will be default have 1 version of data. Currently a user > can specifiy the maxVersion during create table time or alter the columnfam > later. This feature will add a configuration point in hbase-sit.xml so that > an admin can set the default globally. > a small discussion in > [HBASE-10941|https://issues.apache.org/jira/browse/HBASE-10941] lead to this > jira -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11031) Some HTable's are not closed in TestLogRolling
[
https://issues.apache.org/jira/browse/HBASE-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11031:
---
Fix Version/s: (was: 0.99.0)
> Some HTable's are not closed in TestLogRolling
> --
>
> Key: HBASE-11031
> URL: https://issues.apache.org/jira/browse/HBASE-11031
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Trivial
> Attachments: 11031-v1.txt
>
>
> The following pattern appears in several methods:
> {code}
> // When the hbase:meta table can be opened, the region servers are running
> new HTable(TEST_UTIL.getConfiguration(), TableName.META_TABLE_NAME);
> {code}
> The HTable instance should be closed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11023) Port HBASE-10488 "'mvn site' is broken due to org.apache.jasper.JspC not found" to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976317#comment-13976317 ] Andrew Purtell commented on HBASE-11023: Don't use '-Dhadoop.profile=23' Use '-Dhadoop.profile=2.0' Are you building from a release tarball [~cos]? > Port HBASE-10488 "'mvn site' is broken due to org.apache.jasper.JspC not > found" to 0.98 > --- > > Key: HBASE-11023 > URL: https://issues.apache.org/jira/browse/HBASE-11023 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.98.2 > > Attachments: 11023.txt > > > This is to port the fix (mostly in hbase-thrift/pom.xml) for the site target. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11018) ZKUtil.getChildDataAndWatchForNewChildren() will not return null as indicated
[
https://issues.apache.org/jira/browse/HBASE-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11018:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the patch, Jerry.
> ZKUtil.getChildDataAndWatchForNewChildren() will not return null as indicated
> -
>
> Key: HBASE-11018
> URL: https://issues.apache.org/jira/browse/HBASE-11018
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Affects Versions: 0.96.1, 0.98.1
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-11018-trunk.patch
>
>
> While working on HBase acl, I found out that
> ZKUtil.getChildDataAndWatchForNewChildren() will not return null as
> indicated. Here is the code:
> {code}
> /**
>
>* Returns null if the specified node does not exist. Otherwise returns a
>* list of children of the specified node. If the node exists but it has no
>* children, an empty list will be returned.
>
>*/
> public static List getChildDataAndWatchForNewChildren(
> ZooKeeperWatcher zkw, String baseNode) throws KeeperException {
> List nodes =
> ZKUtil.listChildrenAndWatchForNewChildren(zkw, baseNode);
> List newNodes = new ArrayList();
> if (nodes != null) {
> for (String node : nodes) {
> String nodePath = ZKUtil.joinZNode(baseNode, node);
> byte[] data = ZKUtil.getDataAndWatch(zkw, nodePath);
> newNodes.add(new NodeAndData(nodePath, data));
> }
> }
> return newNodes;
> }
> {code}
> We return 'newNodes' which will never be null.
> This is a deprecated method. But it is still used in HBase code.
> For example:
> org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.start()
> {code}
> public void start() throws KeeperException {
> watcher.registerListener(this);
> if (ZKUtil.watchAndCheckExists(watcher, aclZNode)) {
> List existing =
> ZKUtil.getChildDataAndWatchForNewChildren(watcher, aclZNode);
> if (existing != null) {
> refreshNodes(existing);
> }
> }
> }
> {code}
> We test the 'null' return from the call which becomes the problem.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11046) New Scanner API
[ https://issues.apache.org/jira/browse/HBASE-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Deng updated HBASE-11046: Description: A new scanner API for reducing unnecessary RPC calls: Motivation: # RPC is expensive to both client and server. # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation) Solution: # a new scanner API (*scanOpen*) which has an option of transfer data along with the scannerID back in this call # a new scanner API (*scanNext*) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner. # *scanClose* is still useful when you want to close the scanner before reach the end. For most of the meta-scan, only one RPC will fetch all lines. was: A new scanner API for reducing unnecessary RPC calls: Motivation: # RPC is expensive to both client and server. # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation) Solution: # a new scanner API (*scanOpen*) which has an option of transfer data along with the scannerID back in this call # a new scanner API (*scanNext*) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner. # *scanClose* is still useful when you want to close the scanner before reach the end. > New Scanner API > --- > > Key: HBASE-11046 > URL: https://issues.apache.org/jira/browse/HBASE-11046 > Project: HBase > Issue Type: New Feature > Components: Scanners >Reporter: Yi Deng > Labels: features > Fix For: 0.89-fb > > > A new scanner API for reducing unnecessary RPC calls: > Motivation: > # RPC is expensive to both client and server. > # The most important function for scanning is getting data, but for each > scanning process within a region, there are 3 times of RPC that doesn't > transfer data: open, last next, and close, I want to remove them all (for > most of the situation) > Solution: > # a new scanner API (*scanOpen*) which has an option of transfer data along > with the scannerID back in this call > # a new scanner API (*scanNext*) which is similar to current next, but > returns flags of whether more data is available and whether need to scan next > region. If no data left, automatically close the scanner. > # *scanClose* is still useful when you want to close the scanner before reach > the end. > For most of the meta-scan, only one RPC will fetch all lines. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11046) New Scanner API
[ https://issues.apache.org/jira/browse/HBASE-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Deng updated HBASE-11046: Description: A new scanner API for reducing unnecessary RPC calls: Motivation: # RPC is expensive to both client and server. # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation) Solution: # a new scanner API (*scanOpen*) which has an option of transfer data along with the scannerID back in this call # a new scanner API (*scanNext*) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner. # *scanClose* is still useful when you want to close the scanner before reach the end. was: A new scanner API for reducing unnecessary RPC calls: Motivation: # RPC is expensive to both client and server. # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last nextRows, and close, I want to remove them all (for most of the situation) Solution: # a new scanner API (*scanOpen*) which has an option of transfer data along with the scannerID back in this call # a new scanner API (*scanNext*) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner. # *scanClose* is still useful when you want to close the scanner before reach the end. > New Scanner API > --- > > Key: HBASE-11046 > URL: https://issues.apache.org/jira/browse/HBASE-11046 > Project: HBase > Issue Type: New Feature > Components: Scanners >Reporter: Yi Deng > Labels: features > Fix For: 0.89-fb > > > A new scanner API for reducing unnecessary RPC calls: > Motivation: > # RPC is expensive to both client and server. > # The most important function for scanning is getting data, but for each > scanning process within a region, there are 3 times of RPC that doesn't > transfer data: open, last next, and close, I want to remove them all (for > most of the situation) > Solution: > # a new scanner API (*scanOpen*) which has an option of transfer data along > with the scannerID back in this call > # a new scanner API (*scanNext*) which is similar to current next, but > returns flags of whether more data is available and whether need to scan next > region. If no data left, automatically close the scanner. > # *scanClose* is still useful when you want to close the scanner before reach > the end. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11046) New Scanner API
[
https://issues.apache.org/jira/browse/HBASE-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Deng updated HBASE-11046:
Description:
A new scanner API for reducing unnecessary RPC calls:
Motivation:
# RPC is expensive to both client and server.
# The most important function for scanning is getting data, but for each
scanning process within a region, there are 3 times of RPC that doesn't
transfer data: open, last nextRows, and close, I want to remove them all (for
most of the situation)
Solution:
# a new scanner API (*scanOpen*) which has an option of transfer data along
with the scannerID back in this call
# a new scanner API (*scanNext*) which is similar to current next, but returns
flags of whether more data is available and whether need to scan next region.
If no data left, automatically close the scanner.
# *scanClose* is still useful when you want to close the scanner before reach
the end.
was:
A new scanner API for reducing unnecessary RPC calls:
Motivation:
# RPC is expensive to both client and server.
# The most important function for scanning is getting data, but for each
scanning process within a region, there are 3 times of RPC that doesn't
transfer data: open, last nextRows, and close, I want to remove them all (for
most of the situation)
Solution:
# a new scanner API ({code}scanOpen{code}) which has an option of transfer data
along with the scannerID back in this call
# a new scanner API (scanNext) which is similar to current next, but returns
flags of whether more data is available and whether need to scan next region.
If no data left, automatically close the scanner.
# the current scannClose is still useful when you want to close the scanner
before reach the end.
> New Scanner API
> ---
>
> Key: HBASE-11046
> URL: https://issues.apache.org/jira/browse/HBASE-11046
> Project: HBase
> Issue Type: New Feature
> Components: Scanners
>Reporter: Yi Deng
> Labels: features
> Fix For: 0.89-fb
>
>
> A new scanner API for reducing unnecessary RPC calls:
> Motivation:
> # RPC is expensive to both client and server.
> # The most important function for scanning is getting data, but for each
> scanning process within a region, there are 3 times of RPC that doesn't
> transfer data: open, last nextRows, and close, I want to remove them all (for
> most of the situation)
> Solution:
> # a new scanner API (*scanOpen*) which has an option of transfer data along
> with the scannerID back in this call
> # a new scanner API (*scanNext*) which is similar to current next, but
> returns flags of whether more data is available and whether need to scan next
> region. If no data left, automatically close the scanner.
> # *scanClose* is still useful when you want to close the scanner before reach
> the end.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11046) New Scanner API
[
https://issues.apache.org/jira/browse/HBASE-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Deng updated HBASE-11046:
Description:
A new scanner API for reducing unnecessary RPC calls:
Motivation:
# RPC is expensive to both client and server.
# The most important function for scanning is getting data, but for each
scanning process within a region, there are 3 times of RPC that doesn't
transfer data: open, last nextRows, and close, I want to remove them all (for
most of the situation)
Solution:
# a new scanner API ({code}scanOpen{code}) which has an option of transfer data
along with the scannerID back in this call
# a new scanner API (scanNext) which is similar to current next, but returns
flags of whether more data is available and whether need to scan next region.
If no data left, automatically close the scanner.
# the current scannClose is still useful when you want to close the scanner
before reach the end.
was:
A new scanner API for reducing unnecessary RPC calls:
Motivation:
# RPC is expensive to both client and server.
# The most important function for scanning is getting data, but for each
scanning process within a region, there are 3 times of RPC that doesn't
transfer data: open, last nextRows, and close, I want to remove them all (for
most of the situation)
Solution:
# a new scanner API (scannerOpen) which has an option of transfer data along
with the scannerID back in this call
# a new scanner API (scannerNext) which is similar to current next, but returns
flags of whether more data is available and whether need to scan next region.
If no data left, automatically close the scanner.
# the current scannerClose is still useful when you want to close the scanner
before reach the end.
> New Scanner API
> ---
>
> Key: HBASE-11046
> URL: https://issues.apache.org/jira/browse/HBASE-11046
> Project: HBase
> Issue Type: New Feature
> Components: Scanners
>Reporter: Yi Deng
> Labels: features
> Fix For: 0.89-fb
>
>
> A new scanner API for reducing unnecessary RPC calls:
> Motivation:
> # RPC is expensive to both client and server.
> # The most important function for scanning is getting data, but for each
> scanning process within a region, there are 3 times of RPC that doesn't
> transfer data: open, last nextRows, and close, I want to remove them all (for
> most of the situation)
> Solution:
> # a new scanner API ({code}scanOpen{code}) which has an option of transfer
> data along with the scannerID back in this call
> # a new scanner API (scanNext) which is similar to current next, but returns
> flags of whether more data is available and whether need to scan next region.
> If no data left, automatically close the scanner.
> # the current scannClose is still useful when you want to close the scanner
> before reach the end.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11046) New Scanner API
[ https://issues.apache.org/jira/browse/HBASE-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Deng updated HBASE-11046: Description: A new scanner API for reducing unnecessary RPC calls: Motivation: # RPC is expensive to both client and server. # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last nextRows, and close, I want to remove them all (for most of the situation) Solution: # a new scanner API (scannerOpen) which has an option of transfer data along with the scannerID back in this call # a new scanner API (scannerNext) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner. # the current scannerClose is still useful when you want to close the scanner before reach the end. was: A new scanner API for reducing unnecessary RPC calls: Motivation: # RPC is expensive to both client and server. # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation) Solution: # a new scanner API (scannerOpen) which has an option of transfer data along with the scannerID back in this call # a new scanner API (scannerNext) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner. # the current scannerClose is still useful when you want to close the scanner before reach the end. > New Scanner API > --- > > Key: HBASE-11046 > URL: https://issues.apache.org/jira/browse/HBASE-11046 > Project: HBase > Issue Type: New Feature > Components: Scanners >Reporter: Yi Deng > Labels: features > Fix For: 0.89-fb > > > A new scanner API for reducing unnecessary RPC calls: > Motivation: > # RPC is expensive to both client and server. > # The most important function for scanning is getting data, but for each > scanning process within a region, there are 3 times of RPC that doesn't > transfer data: open, last nextRows, and close, I want to remove them all (for > most of the situation) > Solution: > # a new scanner API (scannerOpen) which has an option of transfer data along > with the scannerID back in this call > # a new scanner API (scannerNext) which is similar to current next, but > returns flags of whether more data is available and whether need to scan next > region. If no data left, automatically close the scanner. > # the current scannerClose is still useful when you want to close the scanner > before reach the end. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11001) Shell support for granting cell permissions for testing
[
https://issues.apache.org/jira/browse/HBASE-11001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11001:
---
Status: Patch Available (was: Open)
Tested manually. Created a table with two families with owner "hbase". Added
values to each family in two rows (4 values total). Scan of table with user
"test" returns 0 cells. With new variation of the grant command, granted 'R'
permission to only one CF to user "test" using a prefix filter that selects
only one row. Scan of table afterward returns one cell from the appropriate
family as expected.
I also confirmed global, table, and column family grants still work.
> Shell support for granting cell permissions for testing
> ---
>
> Key: HBASE-11001
> URL: https://issues.apache.org/jira/browse/HBASE-11001
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 11001.patch, HBASE-11001.patch
>
>
> For testing purposes it would be useful if the shell can support a simple
> syntax for adding cell ACLs to existing cells. Consider a combination of
> current 'grant' and 'scan' commands.
> {noformat}
> grant , ,
> {noformat}
> where is a string, a table name, optionally prefixed with a namespace
> where is a Hash type that maps permissions to users, for
> example: \\
> - { "user1" => "RW", "user2" => "R", "@group1" => "R" }
> where is a Hash type containing a scanner specification, for example
> (borrowed from scan.rb): \\
> - { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
> - { COLUMNS => 'c1', TIMERANGE => [123, 456] }
> - { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
> (QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11001) Shell support for granting cell permissions for testing
[
https://issues.apache.org/jira/browse/HBASE-11001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11001:
---
Attachment: HBASE-11001.patch
> Shell support for granting cell permissions for testing
> ---
>
> Key: HBASE-11001
> URL: https://issues.apache.org/jira/browse/HBASE-11001
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 11001.patch, HBASE-11001.patch
>
>
> For testing purposes it would be useful if the shell can support a simple
> syntax for adding cell ACLs to existing cells. Consider a combination of
> current 'grant' and 'scan' commands.
> {noformat}
> grant , ,
> {noformat}
> where is a string, a table name, optionally prefixed with a namespace
> where is a Hash type that maps permissions to users, for
> example: \\
> - { "user1" => "RW", "user2" => "R", "@group1" => "R" }
> where is a Hash type containing a scanner specification, for example
> (borrowed from scan.rb): \\
> - { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
> - { COLUMNS => 'c1', TIMERANGE => [123, 456] }
> - { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
> (QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11018) ZKUtil.getChildDataAndWatchForNewChildren() will not return null as indicated
[
https://issues.apache.org/jira/browse/HBASE-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976285#comment-13976285
]
chunhui shen commented on HBASE-11018:
--
+1
> ZKUtil.getChildDataAndWatchForNewChildren() will not return null as indicated
> -
>
> Key: HBASE-11018
> URL: https://issues.apache.org/jira/browse/HBASE-11018
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Affects Versions: 0.96.1, 0.98.1
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-11018-trunk.patch
>
>
> While working on HBase acl, I found out that
> ZKUtil.getChildDataAndWatchForNewChildren() will not return null as
> indicated. Here is the code:
> {code}
> /**
>
>* Returns null if the specified node does not exist. Otherwise returns a
>* list of children of the specified node. If the node exists but it has no
>* children, an empty list will be returned.
>
>*/
> public static List getChildDataAndWatchForNewChildren(
> ZooKeeperWatcher zkw, String baseNode) throws KeeperException {
> List nodes =
> ZKUtil.listChildrenAndWatchForNewChildren(zkw, baseNode);
> List newNodes = new ArrayList();
> if (nodes != null) {
> for (String node : nodes) {
> String nodePath = ZKUtil.joinZNode(baseNode, node);
> byte[] data = ZKUtil.getDataAndWatch(zkw, nodePath);
> newNodes.add(new NodeAndData(nodePath, data));
> }
> }
> return newNodes;
> }
> {code}
> We return 'newNodes' which will never be null.
> This is a deprecated method. But it is still used in HBase code.
> For example:
> org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.start()
> {code}
> public void start() throws KeeperException {
> watcher.registerListener(this);
> if (ZKUtil.watchAndCheckExists(watcher, aclZNode)) {
> List existing =
> ZKUtil.getChildDataAndWatchForNewChildren(watcher, aclZNode);
> if (existing != null) {
> refreshNodes(existing);
> }
> }
> }
> {code}
> We test the 'null' return from the call which becomes the problem.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10999) Cross-row Transaction : Implement Percolator Algorithm on HBase
[ https://issues.apache.org/jira/browse/HBASE-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976279#comment-13976279 ] cuijianwei commented on HBASE-10999: [~stack], thanks for your comment. Haeinsa is an interesting project to implement cross-row transaction on HBase. We analyzed Haeinsa's implementation before deciding to implement percolator algorithm. In my opinion, an important difference between percolator and Haeinsa is that percolator provides global database snapshot for read while Haeinsa always returns the data of newest committed transactions. If our analysis is right, the read of Haeinsa needs two phases. Firstly, Haeinsa needs to read back the data and locks of transaction rows where the data and locks will be both cached in client side. After this, Haeinsa needs to read back the locks of transaction rows again to check the locks are not changed, so that won't return incomplete transactions to users. The two-phase read might make Haeinsa not easy to read large volume of data for two reasons：a). it is not easy to cached data and locks for a large number of rows in client side; b) when scanning a large range of rows, newer writes have a greater possibility to change the locks of scanning rows which will make read fail more easily. On the other hand, percolator will use the a global incremental timestamp to define the database snapshot for read. The client will return the row to user if no lock conflict discovered, so that does not need to cache any data and lock in client side. The Haeinsa project does not provides global database snapshot so that it does not depend a Global Incremental Timestamp Service, which makes its implementation more independent. However, in my opinion, the global database snapshot is important for transactions as analyzed above; and we find it is not difficult to implement a Global Incremental Timestamp Service. Consequently, we implemented percolator algorithm to do cross-row transaction. > Cross-row Transaction : Implement Percolator Algorithm on HBase > --- > > Key: HBASE-10999 > URL: https://issues.apache.org/jira/browse/HBASE-10999 > Project: HBase > Issue Type: New Feature > Components: Transactions/MVCC >Affects Versions: 0.99.0 >Reporter: cuijianwei >Assignee: cuijianwei > > Cross-row transaction is a desired function for database. It is not easy to > keep ACID characteristics of cross-row transactions in distribute databases > such as HBase, because data of cross-transaction might locate in different > machines. In the paper http://research.google.com/pubs/pub36726.html, google > presents an algorithm(named percolator) to implement cross-row transactions > on BigTable. After analyzing the algorithm, we found percolator might also be > a choice to provide cross-row transaction on HBase. The reasons includes: > 1. Percolator could keep the ACID of cross-row transaction as described in > google's paper. Percolator depends on a Global Incremental Timestamp Service > to define the order of transactions, this is important to keep ACID of > transaction. > 2. Percolator algorithm could be totally implemented in client-side. This > means we do not need to change the logic of server side. Users could easily > include percolator in their client and adopt percolator APIs only when they > want cross-row transaction. > 3. Percolator is a general algorithm which could be implemented based on > databases providing single-row transaction. Therefore, it is feasible to > implement percolator on HBase. > In last few months, we have implemented percolator on HBase, did correctness > validation, performance test and finally successfully applied this algorithm > in our production environment. Our works include: > 1. percolator algorithm implementation on HBase. The current implementations > includes: > a). a Transaction module to provides put/delete/get/scan interfaces to do > cross-row/cross-table transaction. > b). a Global Incremental Timestamp Server to provide globally > monotonically increasing timestamp for transaction. > c). a LockCleaner module to resolve conflict when concurrent transactions > mutate the same column. > d). an internal module to implement prewrite/commit/get/scan logic of > percolator. >Although percolator logic could be totally implemented in client-side, we > use coprocessor framework of HBase in our implementation. This is because > coprocessor could provide percolator-specific Rpc interfaces such as > prewrite/commit to reduce Rpc rounds and improve efficiency. Another reason > to use coprocessor is that we want to decouple percolator's code from HBase > so that users will get clean HBase code if they don't need cross-row > transactions. In f
[jira] [Updated] (HBASE-11001) Shell support for granting cell permissions for testing
[
https://issues.apache.org/jira/browse/HBASE-11001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11001:
---
Description:
For testing purposes it would be useful if the shell can support a simple
syntax for adding cell ACLs to existing cells. Consider a combination of
current 'grant' and 'scan' commands.
{noformat}
grant , ,
{noformat}
where is a string, a table name, optionally prefixed with a namespace
where is a Hash type that maps permissions to users, for example:
\\
- { "user1" -> "RW", "user2" => "R", "@group1" => "R" }
where is a Hash type containing a scanner specification, for example
(borrowed from scan.rb): \\
- { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
- { COLUMNS => 'c1', TIMERANGE => [123, 456] }
- { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
(QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
was:
For testing purposes it would be useful if the shell can support a simple
syntax for adding cell ACLs to existing cells. Consider a combination of
current 'grant' and 'scan' commands.
{noformat}
grant '', '', '[:]', { ... }
{noformat}
where the last argument is a scanner specification, for example (borrowed from
scan.rb): \\
- { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
- { COLUMNS => 'c1', TIMERANGE => [123, 456] }
- { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
(QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
> Shell support for granting cell permissions for testing
> ---
>
> Key: HBASE-11001
> URL: https://issues.apache.org/jira/browse/HBASE-11001
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 11001.patch
>
>
> For testing purposes it would be useful if the shell can support a simple
> syntax for adding cell ACLs to existing cells. Consider a combination of
> current 'grant' and 'scan' commands.
> {noformat}
> grant , ,
> {noformat}
> where is a string, a table name, optionally prefixed with a namespace
> where is a Hash type that maps permissions to users, for
> example: \\
> - { "user1" -> "RW", "user2" => "R", "@group1" => "R" }
> where is a Hash type containing a scanner specification, for example
> (borrowed from scan.rb): \\
> - { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
> - { COLUMNS => 'c1', TIMERANGE => [123, 456] }
> - { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
> (QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11001) Shell support for granting cell permissions for testing
[
https://issues.apache.org/jira/browse/HBASE-11001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11001:
---
Description:
For testing purposes it would be useful if the shell can support a simple
syntax for adding cell ACLs to existing cells. Consider a combination of
current 'grant' and 'scan' commands.
{noformat}
grant , ,
{noformat}
where is a string, a table name, optionally prefixed with a namespace
where is a Hash type that maps permissions to users, for example:
\\
- { "user1" => "RW", "user2" => "R", "@group1" => "R" }
where is a Hash type containing a scanner specification, for example
(borrowed from scan.rb): \\
- { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
- { COLUMNS => 'c1', TIMERANGE => [123, 456] }
- { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
(QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
was:
For testing purposes it would be useful if the shell can support a simple
syntax for adding cell ACLs to existing cells. Consider a combination of
current 'grant' and 'scan' commands.
{noformat}
grant , ,
{noformat}
where is a string, a table name, optionally prefixed with a namespace
where is a Hash type that maps permissions to users, for example:
\\
- { "user1" -> "RW", "user2" => "R", "@group1" => "R" }
where is a Hash type containing a scanner specification, for example
(borrowed from scan.rb): \\
- { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
- { COLUMNS => 'c1', TIMERANGE => [123, 456] }
- { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
(QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
> Shell support for granting cell permissions for testing
> ---
>
> Key: HBASE-11001
> URL: https://issues.apache.org/jira/browse/HBASE-11001
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 11001.patch
>
>
> For testing purposes it would be useful if the shell can support a simple
> syntax for adding cell ACLs to existing cells. Consider a combination of
> current 'grant' and 'scan' commands.
> {noformat}
> grant , ,
> {noformat}
> where is a string, a table name, optionally prefixed with a namespace
> where is a Hash type that maps permissions to users, for
> example: \\
> - { "user1" => "RW", "user2" => "R", "@group1" => "R" }
> where is a Hash type containing a scanner specification, for example
> (borrowed from scan.rb): \\
> - { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
> - { COLUMNS => 'c1', TIMERANGE => [123, 456] }
> - { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
> (QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10948) Fix hbase table file 'x' mode
[
https://issues.apache.org/jira/browse/HBASE-10948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976269#comment-13976269
]
Jerry He commented on HBASE-10948:
--
Good point.
Should be an easy fix for 0.98 or less without using the Hadoop 2 API.
Will provide a patch.
> Fix hbase table file 'x' mode
> -
>
> Key: HBASE-10948
> URL: https://issues.apache.org/jira/browse/HBASE-10948
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration
>Affects Versions: 0.96.2, 0.98.1
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 0.99.0
>
> Attachments: HBASE-10948-trunk-v2.patch, HBASE-10948-trunk.patch
>
>
> The hbase table files currently all have 'x' mode in there:
> {code}
> $hadoop fs -ls -R /hbase/data/default/TestTable/
> drwxr-xr-x - hbase biadmin 0 2014-04-08 20:53
> /hbase/data/default/TestTable/.tabledesc
> -rw-r--r-- 1 hbase biadmin313 2014-04-08 20:53
> /hbase/data/default/TestTable/.tabledesc/.tableinfo.01
> drwxr-xr-x - hbase biadmin 0 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64
> -rwxr-xr-x 1 hbase biadmin 68 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/.regioninfo
> drwxr-xr-x - hbase biadmin 0 2014-04-08 21:54
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/info
> -rwxr-xr-x 1 hbase biadmin 272958577 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/info/7138e61cbcd24538b64726db13dab48e
> -rwxr-xr-x 1 hbase biadmin 108603714 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/info/9ce233fcdfde49679797d13f28e26129
> drwxr-xr-x - hbase biadmin 0 2014-04-08 20:53
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564
> -rwxr-xr-x 1 hbase biadmin 68 2014-04-08 20:53
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/.regioninfo
> drwxr-xr-x - hbase biadmin 0 2014-04-08 21:54
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/info
> -rwxr-xr-x 1 hbase biadmin 33800049 2014-04-08 21:54
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/info/576190de431341b9a02280654e3deb58
> -rwxr-xr-x 1 hbase biadmin 108650474 2014-04-08 20:53
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/info/7c54098fb62a4ef29aab0f5658b25260
> {code}
> If the user does not specify 'hbase.data.umask', we use the fs default:
> FsPermission.getDefault()
> Instead we should use FsPermission.getFileDefault().
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11001) Shell support for granting cell permissions for testing
[
https://issues.apache.org/jira/browse/HBASE-11001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976266#comment-13976266
]
Andrew Purtell commented on HBASE-11001:
I have a working version of the patch but have decided to go with a different
syntax, and am updating the description on this issue.
> Shell support for granting cell permissions for testing
> ---
>
> Key: HBASE-11001
> URL: https://issues.apache.org/jira/browse/HBASE-11001
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 11001.patch
>
>
> For testing purposes it would be useful if the shell can support a simple
> syntax for adding cell ACLs to existing cells. Consider a combination of
> current 'grant' and 'scan' commands.
> {noformat}
> grant '', '', '[:]', { ... }
> {noformat}
> where the last argument is a scanner specification, for example (borrowed
> from scan.rb): \\
> - { COLUMNS => 'c1', STARTROW => 'abc', ENDROW => 'xyz' }
> - { COLUMNS => 'c1', TIMERANGE => [123, 456] }
> - { COLUMNS => ['c1', 'c2'], FILTER => "(PrefixFilter ('foo') AND
> (QualifierFilter (=, 'bar:'))) AND (TimestampsFilter (123, 456))" }
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10994) HBase Multi-Tenancy
[ https://issues.apache.org/jira/browse/HBASE-10994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976254#comment-13976254 ] Elliott Clark commented on HBASE-10994: --- Can we put this on the schedule for the after HBaseCon hackathon ? > HBase Multi-Tenancy > --- > > Key: HBASE-10994 > URL: https://issues.apache.org/jira/browse/HBASE-10994 > Project: HBase > Issue Type: Umbrella >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: Phoenix, multitenancy, quota > > Currently HBase treats all tables, users, and workloads in the same way. > This is ok, until multiple users and workloads are applied on the same > cluster/table. Some workloads/users must be prioritized over others, and some > other workloads must not impact others. > We can separate the problem into three components. > * Isolation/Partitioning (Physically split on different machines) > * Scheduling (Prioritize small/interactive workloads vs long/batch workloads) > * Quotas (Limit a user/table requests/sec or size) > This is the umbrella jira tracking the multi-tenancy related tasks. > An initial design document is up for comments here: > https://docs.google.com/document/d/1ygIwZpDWQuMPdfcryckic6ODi5DHQkrzXKjmOJodfs0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10950) Add a configuration point for MaxVersion of Column Family
[ https://issues.apache.org/jira/browse/HBASE-10950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976250#comment-13976250 ] Enoch Hsu commented on HBASE-10950: --- Right. How does this look? hbase.column.max.version 1 The default number of versions to keep which is set in a table's Column Descriptor > Add a configuration point for MaxVersion of Column Family > -- > > Key: HBASE-10950 > URL: https://issues.apache.org/jira/browse/HBASE-10950 > Project: HBase > Issue Type: Improvement > Components: Admin >Affects Versions: 0.98.0, 0.96.0 >Reporter: Demai Ni >Assignee: Enoch Hsu > Fix For: 0.99.0, 0.98.2, 0.96.3 > > Attachments: HBASE_10950.patch, HBASE_10950_v2.patch, > HBASE_10950_v3.patch > > > Starting on 0.96.0. HColumnDescriptor.DEFAULT_VERSIONS change to 1 from 3. > So a columnfamily will be default have 1 version of data. Currently a user > can specifiy the maxVersion during create table time or alter the columnfam > later. This feature will add a configuration point in hbase-sit.xml so that > an admin can set the default globally. > a small discussion in > [HBASE-10941|https://issues.apache.org/jira/browse/HBASE-10941] lead to this > jira -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10994) HBase Multi-Tenancy
[ https://issues.apache.org/jira/browse/HBASE-10994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated HBASE-10994: - Labels: Phoenix multitenancy quota (was: multitenancy quota) > HBase Multi-Tenancy > --- > > Key: HBASE-10994 > URL: https://issues.apache.org/jira/browse/HBASE-10994 > Project: HBase > Issue Type: Umbrella >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: Phoenix, multitenancy, quota > > Currently HBase treats all tables, users, and workloads in the same way. > This is ok, until multiple users and workloads are applied on the same > cluster/table. Some workloads/users must be prioritized over others, and some > other workloads must not impact others. > We can separate the problem into three components. > * Isolation/Partitioning (Physically split on different machines) > * Scheduling (Prioritize small/interactive workloads vs long/batch workloads) > * Quotas (Limit a user/table requests/sec or size) > This is the umbrella jira tracking the multi-tenancy related tasks. > An initial design document is up for comments here: > https://docs.google.com/document/d/1ygIwZpDWQuMPdfcryckic6ODi5DHQkrzXKjmOJodfs0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10948) Fix hbase table file 'x' mode
[
https://issues.apache.org/jira/browse/HBASE-10948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976239#comment-13976239
]
Andrew Purtell commented on HBASE-10948:
No confusion. Fix it a different way for versions 0.98 and less if you think
its important. A shame this is only going into trunk because it uses a Hadoop 2
specific API.
> Fix hbase table file 'x' mode
> -
>
> Key: HBASE-10948
> URL: https://issues.apache.org/jira/browse/HBASE-10948
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration
>Affects Versions: 0.96.2, 0.98.1
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 0.99.0
>
> Attachments: HBASE-10948-trunk-v2.patch, HBASE-10948-trunk.patch
>
>
> The hbase table files currently all have 'x' mode in there:
> {code}
> $hadoop fs -ls -R /hbase/data/default/TestTable/
> drwxr-xr-x - hbase biadmin 0 2014-04-08 20:53
> /hbase/data/default/TestTable/.tabledesc
> -rw-r--r-- 1 hbase biadmin313 2014-04-08 20:53
> /hbase/data/default/TestTable/.tabledesc/.tableinfo.01
> drwxr-xr-x - hbase biadmin 0 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64
> -rwxr-xr-x 1 hbase biadmin 68 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/.regioninfo
> drwxr-xr-x - hbase biadmin 0 2014-04-08 21:54
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/info
> -rwxr-xr-x 1 hbase biadmin 272958577 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/info/7138e61cbcd24538b64726db13dab48e
> -rwxr-xr-x 1 hbase biadmin 108603714 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/info/9ce233fcdfde49679797d13f28e26129
> drwxr-xr-x - hbase biadmin 0 2014-04-08 20:53
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564
> -rwxr-xr-x 1 hbase biadmin 68 2014-04-08 20:53
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/.regioninfo
> drwxr-xr-x - hbase biadmin 0 2014-04-08 21:54
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/info
> -rwxr-xr-x 1 hbase biadmin 33800049 2014-04-08 21:54
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/info/576190de431341b9a02280654e3deb58
> -rwxr-xr-x 1 hbase biadmin 108650474 2014-04-08 20:53
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/info/7c54098fb62a4ef29aab0f5658b25260
> {code}
> If the user does not specify 'hbase.data.umask', we use the fs default:
> FsPermission.getDefault()
> Instead we should use FsPermission.getFileDefault().
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10994) HBase Multi-Tenancy
[ https://issues.apache.org/jira/browse/HBASE-10994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976225#comment-13976225 ] stack commented on HBASE-10994: --- bq. FWIW, a number of folks in the PhD program at Duke are interested as well and working in this area. Perhaps some collaboration is in order? Sure. How. Where. When (smile). We game. > HBase Multi-Tenancy > --- > > Key: HBASE-10994 > URL: https://issues.apache.org/jira/browse/HBASE-10994 > Project: HBase > Issue Type: Umbrella >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: multitenancy, quota > > Currently HBase treats all tables, users, and workloads in the same way. > This is ok, until multiple users and workloads are applied on the same > cluster/table. Some workloads/users must be prioritized over others, and some > other workloads must not impact others. > We can separate the problem into three components. > * Isolation/Partitioning (Physically split on different machines) > * Scheduling (Prioritize small/interactive workloads vs long/batch workloads) > * Quotas (Limit a user/table requests/sec or size) > This is the umbrella jira tracking the multi-tenancy related tasks. > An initial design document is up for comments here: > https://docs.google.com/document/d/1ygIwZpDWQuMPdfcryckic6ODi5DHQkrzXKjmOJodfs0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11043) Users with table's read/write permission can't get table's description
[
https://issues.apache.org/jira/browse/HBASE-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11043:
---
Resolution: Not a Problem
Assignee: (was: Liu Shaohui)
Status: Resolved (was: Patch Available)
> Users with table's read/write permission can't get table's description
> --
>
> Key: HBASE-11043
> URL: https://issues.apache.org/jira/browse/HBASE-11043
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.99.0
>Reporter: Liu Shaohui
>Priority: Minor
> Attachments: HBASE-11043-trunk-v1.diff
>
>
> AccessController#preGetTableDescriptors only allow users with admin or create
> permission to get table's description.
> {quote}
> requirePermission("getTableDescriptors", nameAsBytes, null, null,
> Permission.Action.ADMIN, Permission.Action.CREATE);
> {quote}
> I think Users with table's read/write permission should also be able to get
> table's description.
> Eg: when create a hive table on HBase, hive will get the table description
> to check if the mapping is right. Usually the hive users only have the read
> permission of table.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11043) Users with table's read/write permission can't get table's description
[
https://issues.apache.org/jira/browse/HBASE-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976224#comment-13976224
]
Andrew Purtell commented on HBASE-11043:
This is by design.
> Users with table's read/write permission can't get table's description
> --
>
> Key: HBASE-11043
> URL: https://issues.apache.org/jira/browse/HBASE-11043
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.99.0
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Attachments: HBASE-11043-trunk-v1.diff
>
>
> AccessController#preGetTableDescriptors only allow users with admin or create
> permission to get table's description.
> {quote}
> requirePermission("getTableDescriptors", nameAsBytes, null, null,
> Permission.Action.ADMIN, Permission.Action.CREATE);
> {quote}
> I think Users with table's read/write permission should also be able to get
> table's description.
> Eg: when create a hive table on HBase, hive will get the table description
> to check if the mapping is right. Usually the hive users only have the read
> permission of table.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11032) Replace deprecated methods in FileSystem with their replacements
[
https://issues.apache.org/jira/browse/HBASE-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11032:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Replace deprecated methods in FileSystem with their replacements
>
>
> Key: HBASE-11032
> URL: https://issues.apache.org/jira/browse/HBASE-11032
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-11032-v1.patch, HBASE-11032.patch
>
>
> FileStatus#isDir() is deprecated.
> FileStatus#isDirectory() should be called instead.
> Here is the list of deprecated methods in FileSystem :
> {code}
> public String getName()
> public static FileSystem getNamed(String name, Configuration conf)
> public FSDataOutputStream createNonRecursive(Path f,
> boolean overwrite,
> int bufferSize, short replication, long blockSize,
> Progressable progress) throws IOException {
>public FSDataOutputStream createNonRecursive(Path f, FsPermission
> permission,
>boolean overwrite, int bufferSize, short replication, long blockSize,
>Progressable progress) throws IOException {
> public short getReplication(Path src) throws IOException {
> public boolean delete(Path f) throws IOException {
> public long getLength(Path f) throws IOException {
> public long getBlockSize(Path f) throws IOException {
> public long getDefaultBlockSize() {
> public short getDefaultReplication()
> {code}
> Except for createNonRecursive() which doesn't have non-deprecated equivalent
> in DistributedFileSystem, deprecated methods are replaced with their
> non-deprecated counterparts.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10994) HBase Multi-Tenancy
[ https://issues.apache.org/jira/browse/HBASE-10994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976188#comment-13976188 ] James Taylor commented on HBASE-10994: -- Correct, [~stack]. I should have been more verbose. I just wanted to point out some similar, complimentary work that is also going on in the hope that all of this great work can converge (rather than be duplicated). FWIW, a number of folks in the PhD program at Duke are interested as well and working in this area. Perhaps some collaboration is in order? > HBase Multi-Tenancy > --- > > Key: HBASE-10994 > URL: https://issues.apache.org/jira/browse/HBASE-10994 > Project: HBase > Issue Type: Umbrella >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: multitenancy, quota > > Currently HBase treats all tables, users, and workloads in the same way. > This is ok, until multiple users and workloads are applied on the same > cluster/table. Some workloads/users must be prioritized over others, and some > other workloads must not impact others. > We can separate the problem into three components. > * Isolation/Partitioning (Physically split on different machines) > * Scheduling (Prioritize small/interactive workloads vs long/batch workloads) > * Quotas (Limit a user/table requests/sec or size) > This is the umbrella jira tracking the multi-tenancy related tasks. > An initial design document is up for comments here: > https://docs.google.com/document/d/1ygIwZpDWQuMPdfcryckic6ODi5DHQkrzXKjmOJodfs0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11048) Support setting custom priority per client RPC
[ https://issues.apache.org/jira/browse/HBASE-11048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976173#comment-13976173 ] Jesse Yates commented on HBASE-11048: - I was trying avoid changing the interfaces. With the above patch, you set it per call (rpc) and let the policy defined in your rpc controller determine the priority in the request header. > Support setting custom priority per client RPC > -- > > Key: HBASE-11048 > URL: https://issues.apache.org/jira/browse/HBASE-11048 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.99.0, 0.98.2 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.99.0, 0.98.2 > > Attachments: hbase-11048-trunk-v0.patch > > > Servers have the ability to handle custom rpc priority levels, but currently > we are only using it to differentiate META/ROOT updates from replication and > other 'priority' updates (as specified by annotation tags per RS method). > However, some clients need the ability to create custom handlers (e.g. > PHOENIX-938) which can really only be cleanly tied together to requests by > the request priority. The disconnect is in that there is no way for the > client to overwrite the priority per table - the PayloadCarryingRpcController > will always just set priority per ROOT/META and otherwise just use the > generic priority. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11048) Support setting custom priority per client RPC
[ https://issues.apache.org/jira/browse/HBASE-11048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976171#comment-13976171 ] Elliott Clark commented on HBASE-11048: --- Why put it per client ? Why not per call as is currently in the request header ? > Support setting custom priority per client RPC > -- > > Key: HBASE-11048 > URL: https://issues.apache.org/jira/browse/HBASE-11048 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.99.0, 0.98.2 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.99.0, 0.98.2 > > Attachments: hbase-11048-trunk-v0.patch > > > Servers have the ability to handle custom rpc priority levels, but currently > we are only using it to differentiate META/ROOT updates from replication and > other 'priority' updates (as specified by annotation tags per RS method). > However, some clients need the ability to create custom handlers (e.g. > PHOENIX-938) which can really only be cleanly tied together to requests by > the request priority. The disconnect is in that there is no way for the > client to overwrite the priority per table - the PayloadCarryingRpcController > will always just set priority per ROOT/META and otherwise just use the > generic priority. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11048) Support setting custom priority per client RPC
[ https://issues.apache.org/jira/browse/HBASE-11048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-11048: Attachment: hbase-11048-trunk-v0.patch Attaching initial patch. I went with adding another factory in keeping with the same pattern we are using the RpcCaller creation. We could combine them into the same factory (or use a wrapper factory that delegates to the correct factory for the specific object needed), but im not sure that it really gains us much beyond not passing around another object. > Support setting custom priority per client RPC > -- > > Key: HBASE-11048 > URL: https://issues.apache.org/jira/browse/HBASE-11048 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.99.0, 0.98.2 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.99.0, 0.98.2 > > Attachments: hbase-11048-trunk-v0.patch > > > Servers have the ability to handle custom rpc priority levels, but currently > we are only using it to differentiate META/ROOT updates from replication and > other 'priority' updates (as specified by annotation tags per RS method). > However, some clients need the ability to create custom handlers (e.g. > PHOENIX-938) which can really only be cleanly tied together to requests by > the request priority. The disconnect is in that there is no way for the > client to overwrite the priority per table - the PayloadCarryingRpcController > will always just set priority per ROOT/META and otherwise just use the > generic priority. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11044) [Shell] Show groups for user in 'whoami' output
[ https://issues.apache.org/jira/browse/HBASE-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976160#comment-13976160 ] Hudson commented on HBASE-11044: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #271 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/271/]) HBASE-11044 [Shell] Show groups for user in 'whoami' output (apurtell: rev 1588948) * /hbase/branches/0.98/hbase-shell/src/main/ruby/shell/commands/whoami.rb > [Shell] Show groups for user in 'whoami' output > --- > > Key: HBASE-11044 > URL: https://issues.apache.org/jira/browse/HBASE-11044 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 0.99.0, 0.98.2 > > Attachments: HBASE-11044.patch > > > The 'whoami' command should show the list of groups for the current user > returned by the configured group mapper. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11048) Support setting custom priority per client RPC
Jesse Yates created HBASE-11048: --- Summary: Support setting custom priority per client RPC Key: HBASE-11048 URL: https://issues.apache.org/jira/browse/HBASE-11048 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.99.0, 0.98.2 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.99.0, 0.98.2 Servers have the ability to handle custom rpc priority levels, but currently we are only using it to differentiate META/ROOT updates from replication and other 'priority' updates (as specified by annotation tags per RS method). However, some clients need the ability to create custom handlers (e.g. PHOENIX-938) which can really only be cleanly tied together to requests by the request priority. The disconnect is in that there is no way for the client to overwrite the priority per table - the PayloadCarryingRpcController will always just set priority per ROOT/META and otherwise just use the generic priority. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11037) Race condition in TestZKBasedOpenCloseRegion
[
https://issues.apache.org/jira/browse/HBASE-11037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11037:
--
Issue Type: Test (was: Bug)
> Race condition in TestZKBasedOpenCloseRegion
>
>
> Key: HBASE-11037
> URL: https://issues.apache.org/jira/browse/HBASE-11037
> Project: HBase
> Issue Type: Test
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.99.0, 0.94.19, 0.98.2, 0.96.3
>
> Attachments: 11037-0.94.txt, 11037-0.98.txt, 11037-trunk.txt
>
>
> testCloseRegion is called before testReOpenRegion.
> Here's the sequence of events:
> {code}
> 2014-04-18 20:58:05,645 INFO [Thread-380]
> master.TestZKBasedOpenCloseRegion(313): Running testCloseRegion
> 2014-04-18 20:58:05,645 INFO [Thread-380]
> master.TestZKBasedOpenCloseRegion(315): Number of region servers = 2
> 2014-04-18 20:58:05,645 INFO [Thread-380]
> master.TestZKBasedOpenCloseRegion(164): -ROOT-,,0.70236052
> 2014-04-18 20:58:05,646 DEBUG [Thread-380]
> master.TestZKBasedOpenCloseRegion(320): Asking RS to close region
> -ROOT-,,0.70236052
> ...
> 2014-04-18 20:58:06,237 INFO
> [RS_CLOSE_ROOT-hemera.apache.org,46533,1397854669633-0]
> regionserver.HRegion(1148): Closed -ROOT-,,0.70236052
> ...
> 2014-04-18 20:58:06,404 INFO [Thread-380]
> master.TestZKBasedOpenCloseRegion(333): Done with testCloseRegion
> {code}
> Then
> {code}
> 2014-04-18 20:58:06,431 INFO [pool-1-thread-1] hbase.ResourceChecker(157):
> before master.TestZKBasedOpenCloseRegion#testReOpenRegion: 234 threads, 388
> file descriptors 4 connections,
> ...
> 2014-04-18 20:58:06,466 DEBUG
> [MASTER_OPEN_REGION-hemera.apache.org,52650,1397854669138-3]
> zookeeper.ZKUtil(1597): master:52650-0x14576a1835d Retrieved 62 byte(s)
> of data from znode /hbase/unassigned/70236052; data=region=-ROOT-,,0,
> origin=hemera.apache.org,46533,1397854669633, state=RS_ZK_REGION_OPENED
> 2014-04-18 20:58:06,473 DEBUG [pool-1-thread-1] client.ClientScanner(191):
> Finished with scanning at {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '',
> ENCODED => 1028785192,}
> 2014-04-18 20:58:06,473 INFO [Thread-396]
> master.TestZKBasedOpenCloseRegion(123): Number of region servers = 2
> 2014-04-18 20:58:06,474 INFO [Thread-396]
> master.TestZKBasedOpenCloseRegion(164): -ROOT-,,0.70236052
> 2014-04-18 20:58:06,474 DEBUG [Thread-396]
> master.TestZKBasedOpenCloseRegion(130): Asking RS to close region
> -ROOT-,,0.70236052
> 2014-04-18 20:58:06,474 INFO [Thread-396]
> master.TestZKBasedOpenCloseRegion(147): Unassign -ROOT-,,0.70236052
> 2014-04-18 20:58:06,474 DEBUG [Thread-396] master.AssignmentManager(2126):
> Starting unassignment of region -ROOT-,,0.70236052 (offlining)
> 2014-04-18 20:58:06,475 DEBUG [Thread-396] master.AssignmentManager(2132):
> Attempted to unassign region -ROOT-,,0.70236052 but it is not currently
> assigned anywhere
> 2014-04-18 20:58:06,478 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(294): master:52650-0x14576a1835d Received
> ZooKeeper Event, type=NodeDeleted, state=SyncConnected,
> path=/hbase/unassigned/70236052
> 2014-04-18 20:58:06,478 DEBUG [pool-1-thread-1-EventThread]
> master.AssignmentManager(1176): The znode of region -ROOT-,,0.70236052 has
> been deleted.
> 2014-04-18 20:58:06,478 INFO [pool-1-thread-1-EventThread]
> master.AssignmentManager(1188): The master has opened the region
> -ROOT-,,0.70236052 that was online on hemera.apache.org,46533,1397854669633
> 2014-04-18 20:58:06,478 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(294): master:52650-0x14576a1835d Received
> ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/unassigned
> {code}
> Then nothing happens. So testCloseRegion unassigns the ROOT region and
> testReOpenRegion starts before ROOT is reassigned. Hence it waits forever for
> the close event, since it never happens.
> This is the key "master.AssignmentManager(2132): Attempted to unassign region
> -ROOT-,,0.70236052 but it is not currently assigned anywhere"
> The easiest fix is to just run testCloseRegion last (as it was before we
> switched junit).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11024) TestSecureLoadIncrementalHFilesSplitRecovery should wait longer for ACL table
[
https://issues.apache.org/jira/browse/HBASE-11024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11024:
--
Issue Type: Test (was: Bug)
> TestSecureLoadIncrementalHFilesSplitRecovery should wait longer for ACL table
> -
>
> Key: HBASE-11024
> URL: https://issues.apache.org/jira/browse/HBASE-11024
> Project: HBase
> Issue Type: Test
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.19
>
> Attachments: 11024.txt
>
>
> One more:
> {code}
> Error Message
> Timed out waiting for table to become available _acl_
> Stacktrace
> java.lang.AssertionError: Timed out waiting for table to become available
> _acl_
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.waitTableAvailable(HBaseTestingUtility.java:1799)
> at
> org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery.setupCluster(TestSecureLoadIncrementalHFilesSplitRecovery.java:57)
> {code}
> waitTableAvailable here only waits for 5s in 0.94 (30s almost everywhere else
> and also in trunk).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11042) TestForceCacheImportantBlocks OOMs occasionally in 0.94
[
https://issues.apache.org/jira/browse/HBASE-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11042:
--
Issue Type: Test (was: Bug)
> TestForceCacheImportantBlocks OOMs occasionally in 0.94
> ---
>
> Key: HBASE-11042
> URL: https://issues.apache.org/jira/browse/HBASE-11042
> Project: HBase
> Issue Type: Test
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.19
>
> Attachments: 11042-0.94.txt
>
>
> This trace:
> {code}
> Caused by: java.lang.OutOfMemoryError
> at java.util.zip.Deflater.init(Native Method)
> at java.util.zip.Deflater.(Deflater.java:169)
> at java.util.zip.GZIPOutputStream.(GZIPOutputStream.java:91)
> at java.util.zip.GZIPOutputStream.(GZIPOutputStream.java:110)
> at
> org.apache.hadoop.hbase.io.hfile.ReusableStreamGzipCodec$ReusableGzipOutputStream$ResetableGZIPOutputStream.(ReusableStreamGzipCodec.java:79)
> at
> org.apache.hadoop.hbase.io.hfile.ReusableStreamGzipCodec$ReusableGzipOutputStream.(ReusableStreamGzipCodec.java:90)
> at
> org.apache.hadoop.hbase.io.hfile.ReusableStreamGzipCodec.createOutputStream(ReusableStreamGzipCodec.java:130)
> at
> org.apache.hadoop.io.compress.GzipCodec.createOutputStream(GzipCodec.java:101)
> at
> org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.createPlainCompressionStream(Compression.java:299)
> at
> org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.createCompressionStream(Compression.java:283)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV1.getCompressingStream(HFileWriterV1.java:207)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV1.close(HFileWriterV1.java:356)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.close(StoreFile.java:1330)
> at
> org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:913)
> {code}
> Note that is caused specifically by HFileWriteV1 when using compression. It
> looks like the compression resources are not released.
> Not sure it's worth fixing this at this point. The test can be fixed by
> either not using compression (why are we using compression anyway), or by not
> testing for HFileV1.
> [~stack] it seems you know the the code in HFileWriterV1. Do you want to have
> a look? Maybe there is a quick fix in HFileWriterV1.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11022) Increase timeout for TestHBaseFsck.testSplitDaughtersNotInMeta
[ https://issues.apache.org/jira/browse/HBASE-11022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-11022: -- Issue Type: Test (was: Bug) > Increase timeout for TestHBaseFsck.testSplitDaughtersNotInMeta > -- > > Key: HBASE-11022 > URL: https://issues.apache.org/jira/browse/HBASE-11022 > Project: HBase > Issue Type: Test >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.19, 0.96.3 > > Attachments: 11022.txt > > > Just seen it failing in 0.94 and noticed that in trunk we increased the > timeout to 75s. Should do the same in 0.94. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10989) TestAccessController needs better timeout
[
https://issues.apache.org/jira/browse/HBASE-10989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10989:
--
Issue Type: Test (was: Bug)
> TestAccessController needs better timeout
> -
>
> Key: HBASE-10989
> URL: https://issues.apache.org/jira/browse/HBASE-10989
> Project: HBase
> Issue Type: Test
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.19
>
> Attachments: 10989.txt
>
>
> Here's another one:
> {code}
> Error Message
> Retry exhaust for waiting region to be opened.
> Stacktrace
> java.lang.AssertionError: Retry exhaust for waiting region to be opened.
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.hadoop.hbase.security.access.TestAccessController.testGlobalAuthorizationForNewRegisteredRS(TestAccessController.java:1857)
> {code}
> In trunk we wait up to 10x 1s and we also wait for the regions of the table
> we just requested to move. In 0.94 we waited 10x 200ms.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11029) Increase wait in TestSplitTransactionOnCluster.split
[ https://issues.apache.org/jira/browse/HBASE-11029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-11029: -- Issue Type: Test (was: Bug) > Increase wait in TestSplitTransactionOnCluster.split > > > Key: HBASE-11029 > URL: https://issues.apache.org/jira/browse/HBASE-11029 > Project: HBase > Issue Type: Test >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.19 > > Attachments: 11029.txt > > > Just failed a few time. In 0.94 we wait for 10s in trunk we wait for 30s. > Let's make it the same in 0.94. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10987) Increase timeout in TestZKLeaderManager.testLeaderSelection
[
https://issues.apache.org/jira/browse/HBASE-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10987:
--
Issue Type: Test (was: Bug)
> Increase timeout in TestZKLeaderManager.testLeaderSelection
> ---
>
> Key: HBASE-10987
> URL: https://issues.apache.org/jira/browse/HBASE-10987
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.19
>
> Attachments: 10987.txt
>
>
> In 0.94 we wait 2s find the current leader. In 0.96 and later we wait for 10s.
> The test just failed:
> {code}
> Error Message
> Leader should exist
> Stacktrace
> java.lang.AssertionError: Leader should exist
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:621)
> at
> org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager.testLeaderSelection(TestZKLeaderManager.java:148)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11010) TestChangingEncoding is unnecessarily slow
[
https://issues.apache.org/jira/browse/HBASE-11010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11010:
--
Issue Type: Test (was: Bug)
> TestChangingEncoding is unnecessarily slow
> --
>
> Key: HBASE-11010
> URL: https://issues.apache.org/jira/browse/HBASE-11010
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.99.0, 0.94.19, 0.98.2, 0.96.3
>
> Attachments: 11010-0.94.txt, 11010-trunk.txt
>
>
> The test runs for over 10m on the Jenkins boxes.
> Writing the test data is done like this:
> {code}
> for (int i = 0; i < NUM_ROWS_PER_BATCH; ++i) {
> Put put = new Put(getRowKey(batchId, i));
> for (int j = 0; j < NUM_COLS_PER_ROW; ++j) {
> put.add(CF_BYTES, getQualifier(j),
> getValue(batchId, i, j));
> table.put(put);
> }
> }
> {code}
> There are two problems:
> # the same Put is "putted" multiple times (once for each column added)
> # each Put issued this way causes its one RPC
> On my machine changing this bring the runtime from 247s to 169s.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11017) TestHRegionBusyWait.testWritesWhileScanning fails frequently in 0.94
[
https://issues.apache.org/jira/browse/HBASE-11017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11017:
--
Issue Type: Test (was: Bug)
> TestHRegionBusyWait.testWritesWhileScanning fails frequently in 0.94
>
>
> Key: HBASE-11017
> URL: https://issues.apache.org/jira/browse/HBASE-11017
> Project: HBase
> Issue Type: Test
>Reporter: Lars Hofhansl
>Assignee: stack
> Fix For: 0.94.19
>
> Attachments: 11017.txt
>
>
> Have seen a few of these:
> {code}
> Error Message
> Failed clearing memory after 6 attempts on region:
> testWritesWhileScanning,,1397727647509.2c968a587c4cb7e84a52c7aa8d2afcac.
> Stacktrace
> org.apache.hadoop.hbase.DroppedSnapshotException: Failed clearing memory
> after 6 attempts on region:
> testWritesWhileScanning,,1397727647509.2c968a587c4cb7e84a52c7aa8d2afcac.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1087)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1024)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:989)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.closeHRegion(HRegion.java:4346)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileScanning(TestHRegion.java:3406)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10988) Properly wait for server in TestThriftServerCmdLine
[
https://issues.apache.org/jira/browse/HBASE-10988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10988:
--
Issue Type: Test (was: Bug)
> Properly wait for server in TestThriftServerCmdLine
> ---
>
> Key: HBASE-10988
> URL: https://issues.apache.org/jira/browse/HBASE-10988
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.99.0, 0.94.19, 0.98.2, 0.96.3
>
> Attachments: 10988-0.94.txt, 10988-trunk.txt
>
>
> In 0.94 I find:
> {{Threads.sleepWithoutInterrupt(2000)}}
> In trunk I see:
> {code}
> while ( thriftServer.serverRunner == null ||
> thriftServer.serverRunner.tserver == null ){
>Thread.sleep(1);
> }
> {code}
> Both aren't good.
> The 0.94 version will fail if the server does not come up within 2s. The
> trunk version (1) might wait forever and cause a long timeout for the test
> and (2) wait quite busily with only 1ms of sleeping.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10996) TestTableSnapshotInputFormatScan fails frequently on 0.94
[
https://issues.apache.org/jira/browse/HBASE-10996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10996:
--
Issue Type: Test (was: Bug)
> TestTableSnapshotInputFormatScan fails frequently on 0.94
> -
>
> Key: HBASE-10996
> URL: https://issues.apache.org/jira/browse/HBASE-10996
> Project: HBase
> Issue Type: Test
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.19
>
> Attachments: 10996.txt
>
>
> Not entirely sure why.
> Interestingly it always fails before the tests even start:
> {code}
> Error Message
> No server address listed in .META. for region
> scantest,lll,1397560084603.a2e754fa8f9b4ce82bd6fa3f8c678e53. containing row
> lll
> Stacktrace
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region
> scantest,lll,1397560084603.a2e754fa8f9b4ce82bd6fa3f8c678e53. containing row
> lll
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1175)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1001)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:958)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1675)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1560)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:994)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:850)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:826)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1031)
> at
> org.apache.hadoop.hbase.mapreduce.TestTableSnapshotInputFormatScan.setUpBeforeClass(TestTableSnapshotInputFormatScan.java:85)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11040) TestAccessController, TestAccessControllerFilter, and TestTablePermissions need to wait longer to ACL table
[ https://issues.apache.org/jira/browse/HBASE-11040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-11040: -- Issue Type: Test (was: Sub-task) Parent: (was: HBASE-11024) > TestAccessController, TestAccessControllerFilter, and TestTablePermissions > need to wait longer to ACL table > --- > > Key: HBASE-11040 > URL: https://issues.apache.org/jira/browse/HBASE-11040 > Project: HBase > Issue Type: Test >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.19 > > Attachments: 11040.txt > > > See parent. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10982) TestZKProcedure.testMultiCohortWithMemberTimeoutDuringPrepare fails frequently in 0.94
[ https://issues.apache.org/jira/browse/HBASE-10982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-10982: -- Issue Type: Test (was: Bug) > TestZKProcedure.testMultiCohortWithMemberTimeoutDuringPrepare fails > frequently in 0.94 > -- > > Key: HBASE-10982 > URL: https://issues.apache.org/jira/browse/HBASE-10982 > Project: HBase > Issue Type: Test >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.19 > > Attachments: 10982.txt > > > Turns out this is fixed in trunk already. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11044) [Shell] Show groups for user in 'whoami' output
[ https://issues.apache.org/jira/browse/HBASE-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976141#comment-13976141 ] Hudson commented on HBASE-11044: FAILURE: Integrated in HBase-TRUNK #5104 (See [https://builds.apache.org/job/HBase-TRUNK/5104/]) HBASE-11044 [Shell] Show groups for user in 'whoami' output (apurtell: rev 1588945) * /hbase/trunk/hbase-shell/src/main/ruby/shell/commands/whoami.rb > [Shell] Show groups for user in 'whoami' output > --- > > Key: HBASE-11044 > URL: https://issues.apache.org/jira/browse/HBASE-11044 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 0.99.0, 0.98.2 > > Attachments: HBASE-11044.patch > > > The 'whoami' command should show the list of groups for the current user > returned by the configured group mapper. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10969) TestDistributedLogSplitting fails frequently in 0.94.
[
https://issues.apache.org/jira/browse/HBASE-10969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10969:
--
Issue Type: Test (was: Bug)
> TestDistributedLogSplitting fails frequently in 0.94.
> -
>
> Key: HBASE-10969
> URL: https://issues.apache.org/jira/browse/HBASE-10969
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.19
>
> Attachments: 10969-addendum.txt, 10969.txt
>
>
> Example:
> {code}
> Error Message
> null not an instance of org.apache.hadoop.hbase.MiniHBaseCluster
> Stacktrace
> java.lang.RuntimeException: null not an instance of
> org.apache.hadoop.hbase.MiniHBaseCluster
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getMiniHBaseCluster(HBaseTestingUtility.java:701)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getHBaseCluster(HBaseTestingUtility.java:1653)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.after(TestDistributedLogSplitting.java:125)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11025) Infrastructure for pluggable consensus service
[ https://issues.apache.org/jira/browse/HBASE-11025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976137#comment-13976137 ] ryan rawson commented on HBASE-11025: - Yes doing a full DI is totally out of question, but I thought perhaps at least just not adding yet another static factory method would be nice. > Infrastructure for pluggable consensus service > -- > > Key: HBASE-11025 > URL: https://issues.apache.org/jira/browse/HBASE-11025 > Project: HBase > Issue Type: Sub-task > Components: Zookeeper >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Attachments: HBASE-11025(add param in HRS ctor).patch, > HBASE-11025(add param in HRS ctor).patch, HBASE-11025.patch > > > Related to HBASE-10915. > In this jira I will extract the changed for property changes, factory and > consensus provider interface (+ZK implementation), as it's going to be > required by other subtasks of HBASE-10909. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11008) Align bulk load, flush, and compact to require Action.CREATE
[
https://issues.apache.org/jira/browse/HBASE-11008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976134#comment-13976134
]
Hadoop QA commented on HBASE-11008:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641132/HBASE-11008-0.94.patch
against trunk revision .
ATTACHMENT ID: 12641132
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9355//console
This message is automatically generated.
> Align bulk load, flush, and compact to require Action.CREATE
>
>
> Key: HBASE-11008
> URL: https://issues.apache.org/jira/browse/HBASE-11008
> Project: HBase
> Issue Type: Improvement
> Components: security
>Reporter: Jean-Daniel Cryans
>Assignee: Jean-Daniel Cryans
> Fix For: 0.99.0, 0.98.2, 0.96.3, 0.94.20
>
> Attachments: HBASE-11008-0.94.patch, HBASE-11008-v2.patch,
> HBASE-11008-v3.patch, HBASE-11008.patch
>
>
> Over in HBASE-10958 we noticed that it might make sense to require
> Action.CREATE for bulk load, flush, and compact since it is also required for
> things like enable and disable.
> This means the following changes:
> - preBulkLoadHFile goes from WRITE to CREATE
> - compact/flush go from ADMIN to ADMIN or CREATE
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10801) Ensure DBE interfaces can work with Cell
[ https://issues.apache.org/jira/browse/HBASE-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976132#comment-13976132 ] stack commented on HBASE-10801: --- bq. ByteRange is also fine. But till we finalise on the APIs in BR, I thought would work with BB only. Sounds good (I noticed that ByteBuf, the netty thing, is subclassable with helper facility to aid subclassers.. that we could make our own if wanted... but that is another issue). bq. Ok we can change it. Yeah, fail fast, especially in the new stuff. bq. First thing is, this goes exactly with the KV format because our current buffer is strictly KV based. Does the KV parse have to live outside of KV? bq. So here there are two different buffers so we cannot just wrap it as a KV. If I did a deep copy of the KeyValue -- key-only part -- then I would not need to introduce this new type, the SeekerStateMembers? Maybe I am not understanding what is going on here (pardon me). > Ensure DBE interfaces can work with Cell > > > Key: HBASE-10801 > URL: https://issues.apache.org/jira/browse/HBASE-10801 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-10801.patch, HBASE-10801_1.patch, > HBASE-10801_2.patch, HBASE-10801_3.patch, HBASE-10801_4.patch > > > Some changes to the interfaces may be needed for DBEs or may be the way it > works currently may be need to be modified inorder to make DBEs work with > Cells. Suggestions and ideas welcome. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11008) Align bulk load, flush, and compact to require Action.CREATE
[ https://issues.apache.org/jira/browse/HBASE-11008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-11008: --- Attachment: HBASE-11008-0.94.patch Attaching the patch for 0.94. I have to force using sequence ids else it won't exercise the flush that's coming in HBASE-10958. It doesn't change the semantics of the test itself. > Align bulk load, flush, and compact to require Action.CREATE > > > Key: HBASE-11008 > URL: https://issues.apache.org/jira/browse/HBASE-11008 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Jean-Daniel Cryans >Assignee: Jean-Daniel Cryans > Fix For: 0.99.0, 0.98.2, 0.96.3, 0.94.20 > > Attachments: HBASE-11008-0.94.patch, HBASE-11008-v2.patch, > HBASE-11008-v3.patch, HBASE-11008.patch > > > Over in HBASE-10958 we noticed that it might make sense to require > Action.CREATE for bulk load, flush, and compact since it is also required for > things like enable and disable. > This means the following changes: > - preBulkLoadHFile goes from WRITE to CREATE > - compact/flush go from ADMIN to ADMIN or CREATE -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10999) Cross-row Transaction : Implement Percolator Algorithm on HBase
[ https://issues.apache.org/jira/browse/HBASE-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976125#comment-13976125 ] stack commented on HBASE-10999: --- [~cuijianwei] Looking forward to it. Any comment on how it relates to the work of our friends at VCNC, https://github.com/VCNC/haeinsa announced here: https://www.mail-archive.com/user@hbase.apache.org/msg27565.html Thanks. > Cross-row Transaction : Implement Percolator Algorithm on HBase > --- > > Key: HBASE-10999 > URL: https://issues.apache.org/jira/browse/HBASE-10999 > Project: HBase > Issue Type: New Feature > Components: Transactions/MVCC >Affects Versions: 0.99.0 >Reporter: cuijianwei >Assignee: cuijianwei > > Cross-row transaction is a desired function for database. It is not easy to > keep ACID characteristics of cross-row transactions in distribute databases > such as HBase, because data of cross-transaction might locate in different > machines. In the paper http://research.google.com/pubs/pub36726.html, google > presents an algorithm(named percolator) to implement cross-row transactions > on BigTable. After analyzing the algorithm, we found percolator might also be > a choice to provide cross-row transaction on HBase. The reasons includes: > 1. Percolator could keep the ACID of cross-row transaction as described in > google's paper. Percolator depends on a Global Incremental Timestamp Service > to define the order of transactions, this is important to keep ACID of > transaction. > 2. Percolator algorithm could be totally implemented in client-side. This > means we do not need to change the logic of server side. Users could easily > include percolator in their client and adopt percolator APIs only when they > want cross-row transaction. > 3. Percolator is a general algorithm which could be implemented based on > databases providing single-row transaction. Therefore, it is feasible to > implement percolator on HBase. > In last few months, we have implemented percolator on HBase, did correctness > validation, performance test and finally successfully applied this algorithm > in our production environment. Our works include: > 1. percolator algorithm implementation on HBase. The current implementations > includes: > a). a Transaction module to provides put/delete/get/scan interfaces to do > cross-row/cross-table transaction. > b). a Global Incremental Timestamp Server to provide globally > monotonically increasing timestamp for transaction. > c). a LockCleaner module to resolve conflict when concurrent transactions > mutate the same column. > d). an internal module to implement prewrite/commit/get/scan logic of > percolator. >Although percolator logic could be totally implemented in client-side, we > use coprocessor framework of HBase in our implementation. This is because > coprocessor could provide percolator-specific Rpc interfaces such as > prewrite/commit to reduce Rpc rounds and improve efficiency. Another reason > to use coprocessor is that we want to decouple percolator's code from HBase > so that users will get clean HBase code if they don't need cross-row > transactions. In future, we will also explore the concurrent running > characteristic of coprocessor to do cross-row mutations more efficiently. > 2. an AccountTransfer simulation program to validate the correctness of > implementation. This program will distribute initial values in different > tables, rows and columns in HBase. Each column represents an account. Then, > configured client threads will be concurrently started to read out a number > of account values from different tables and rows by percolator's get; after > this, clients will randomly transfer values among these accounts while > keeping the sum unchanged, which simulates concurrent cross-table/cross-row > transactions. To check the correctness of transactions, a checker thread will > periodically scan account values from all columns, make sure the current > total value is the same as the initial total value. We run this validation > program while developing, this help us correct errors of implementation. > 3. performance evaluation under various test situations. We compared > percolator's APIs with HBase's with different data size and client thread > count for single-column transaction which represents the worst performance > case for percolator. We get the performance comparison result as (below): > a) For read, the performance of percolator is 90% of HBase; > b) For write, the performance of percolator is 23% of HBase. > The drop derives from the overhead of percolator logic, the performance test > result is similar as the result reported by google's paper. > 4. Performance improvement. The write performance
[jira] [Commented] (HBASE-11043) Users with table's read/write permission can't get table's description
[
https://issues.apache.org/jira/browse/HBASE-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976108#comment-13976108
]
Jean-Daniel Cryans commented on HBASE-11043:
It's this way because of HBASE-8692.
> Users with table's read/write permission can't get table's description
> --
>
> Key: HBASE-11043
> URL: https://issues.apache.org/jira/browse/HBASE-11043
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.99.0
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Attachments: HBASE-11043-trunk-v1.diff
>
>
> AccessController#preGetTableDescriptors only allow users with admin or create
> permission to get table's description.
> {quote}
> requirePermission("getTableDescriptors", nameAsBytes, null, null,
> Permission.Action.ADMIN, Permission.Action.CREATE);
> {quote}
> I think Users with table's read/write permission should also be able to get
> table's description.
> Eg: when create a hive table on HBase, hive will get the table description
> to check if the mapping is right. Usually the hive users only have the read
> permission of table.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11025) Infrastructure for pluggable consensus service
[ https://issues.apache.org/jira/browse/HBASE-11025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976065#comment-13976065 ] stack commented on HBASE-11025: --- I am +1 on commit. Will let this hang out for a while in case anyone else has an opinion on the patch. > Infrastructure for pluggable consensus service > -- > > Key: HBASE-11025 > URL: https://issues.apache.org/jira/browse/HBASE-11025 > Project: HBase > Issue Type: Sub-task > Components: Zookeeper >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Attachments: HBASE-11025(add param in HRS ctor).patch, > HBASE-11025(add param in HRS ctor).patch, HBASE-11025.patch > > > Related to HBASE-10915. > In this jira I will extract the changed for property changes, factory and > consensus provider interface (+ZK implementation), as it's going to be > required by other subtasks of HBASE-10909. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11025) Infrastructure for pluggable consensus service
[ https://issues.apache.org/jira/browse/HBASE-11025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976053#comment-13976053 ] Konstantin Boudnik commented on HBASE-11025: bq. Looks like this patch is prereq to all refactorings that follow. You need it commtted Mikhail Antonov to make progress? That would be great actually - otherwise, the other patches are getting more complex by carrying around dups of this code. > Infrastructure for pluggable consensus service > -- > > Key: HBASE-11025 > URL: https://issues.apache.org/jira/browse/HBASE-11025 > Project: HBase > Issue Type: Sub-task > Components: Zookeeper >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Attachments: HBASE-11025(add param in HRS ctor).patch, > HBASE-11025(add param in HRS ctor).patch, HBASE-11025.patch > > > Related to HBASE-10915. > In this jira I will extract the changed for property changes, factory and > consensus provider interface (+ZK implementation), as it's going to be > required by other subtasks of HBASE-10909. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10873) Control number of regions assigned to backup masters
[ https://issues.apache.org/jira/browse/HBASE-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976050#comment-13976050 ] Jimmy Xiang commented on HBASE-10873: - bq. In that case, we do not even need this issue? This patch makes it possible to put some load on backup masters. So that users can make adjustments per their need. I hope we don't need this issue, if not for the region moving concern during master failover. bq. I'm not sure how we currently keep backup masters 'clear'. Wouldn't we want it done in the LB? Yes, it's done in the LB. > Control number of regions assigned to backup masters > > > Key: HBASE-10873 > URL: https://issues.apache.org/jira/browse/HBASE-10873 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.99.0 > > Attachments: hbase-10873.patch, hbase-10873_v2.patch > > > By default, a backup master is treated just like another regionserver. So it > can host as many regions as other regionserver does. When the backup master > becomes the active one, region balancer needs to move those user regions on > this master to other region servers. To minimize the impact, it's better not > to assign too many regions on backup masters. It may not be good to leave the > backup masters idle and not host any region either. > We should make this adjustable so that users can control how many regions to > assign to each backup master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10873) Control number of regions assigned to backup masters
[ https://issues.apache.org/jira/browse/HBASE-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976031#comment-13976031 ] stack commented on HBASE-10873: --- bq. In that case, we do not even need this issue? Perhaps. I'm not sure how we currently keep backup masters 'clear'. Wouldn't we want it done in the LB? Or how is it done currently [~jxiang]? Thanks. > Control number of regions assigned to backup masters > > > Key: HBASE-10873 > URL: https://issues.apache.org/jira/browse/HBASE-10873 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.99.0 > > Attachments: hbase-10873.patch, hbase-10873_v2.patch > > > By default, a backup master is treated just like another regionserver. So it > can host as many regions as other regionserver does. When the backup master > becomes the active one, region balancer needs to move those user regions on > this master to other region servers. To minimize the impact, it's better not > to assign too many regions on backup masters. It may not be good to leave the > backup masters idle and not host any region either. > We should make this adjustable so that users can control how many regions to > assign to each backup master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11047) Remove TimeoutMontior
[ https://issues.apache.org/jira/browse/HBASE-11047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976022#comment-13976022 ] stack commented on HBASE-11047: --- +1 > Remove TimeoutMontior > - > > Key: HBASE-11047 > URL: https://issues.apache.org/jira/browse/HBASE-11047 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > > With HBASE-8002, the TimeoutMonitor is disabled by default. Lately, we > haven't see much problem of region assignments during integration testing > with CM. I was thinking it may be time to remove the timeout monitor now? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11025) Infrastructure for pluggable consensus service
[ https://issues.apache.org/jira/browse/HBASE-11025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976021#comment-13976021 ] stack commented on HBASE-11025: --- Skimming the patch, it looks like you made it work Mikhail. Do we need to stop the consensus provider shutting down the Master or RegionServer? Looks like this patch is prereq to all refactorings that follow. You need it commtted [~mantonov] to make progress? Patch lgtm. > Infrastructure for pluggable consensus service > -- > > Key: HBASE-11025 > URL: https://issues.apache.org/jira/browse/HBASE-11025 > Project: HBase > Issue Type: Sub-task > Components: Zookeeper >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Attachments: HBASE-11025(add param in HRS ctor).patch, > HBASE-11025(add param in HRS ctor).patch, HBASE-11025.patch > > > Related to HBASE-10915. > In this jira I will extract the changed for property changes, factory and > consensus provider interface (+ZK implementation), as it's going to be > required by other subtasks of HBASE-10909. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11032) Replace deprecated methods in FileSystem with their replacements
[
https://issues.apache.org/jira/browse/HBASE-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11032:
---
Hadoop Flags: Reviewed
Integrated to trunk.
Thanks for the patch, Gustavo.
> Replace deprecated methods in FileSystem with their replacements
>
>
> Key: HBASE-11032
> URL: https://issues.apache.org/jira/browse/HBASE-11032
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-11032-v1.patch, HBASE-11032.patch
>
>
> FileStatus#isDir() is deprecated.
> FileStatus#isDirectory() should be called instead.
> Here is the list of deprecated methods in FileSystem :
> {code}
> public String getName()
> public static FileSystem getNamed(String name, Configuration conf)
> public FSDataOutputStream createNonRecursive(Path f,
> boolean overwrite,
> int bufferSize, short replication, long blockSize,
> Progressable progress) throws IOException {
>public FSDataOutputStream createNonRecursive(Path f, FsPermission
> permission,
>boolean overwrite, int bufferSize, short replication, long blockSize,
>Progressable progress) throws IOException {
> public short getReplication(Path src) throws IOException {
> public boolean delete(Path f) throws IOException {
> public long getLength(Path f) throws IOException {
> public long getBlockSize(Path f) throws IOException {
> public long getDefaultBlockSize() {
> public short getDefaultReplication()
> {code}
> Except for createNonRecursive() which doesn't have non-deprecated equivalent
> in DistributedFileSystem, deprecated methods are replaced with their
> non-deprecated counterparts.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11025) Infrastructure for pluggable consensus service
[ https://issues.apache.org/jira/browse/HBASE-11025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976015#comment-13976015 ] stack commented on HBASE-11025: --- bq. Instead of a static factory, would you consider using dependency injection? It makes unit testing a lot easier, and is a better direction to head. Sorry. Late to the game. The above suggestion of Ryans' (welcome back!) is a nice ideal but a tough one to put on new committer Mikhail who has a full enough plate as is. DI is coolio but adding it to HRS now is going to be a pain at least w/o first breaking up HRS into smaller classes: * HRS is a main entry point referenced not only in java code but in lots of scripts; changing its constructor, it will be hard to ensure all references have also been changed. * HRS scope is broad. DI'ing all it covers would make for a very wide constructor. Here are some of HRS concerns that could (should) be DI'able: ** UserProvider ** ServerNonceManager ** RPC provider ** Log Splitter Service ** RegionServerAccounting implementation ** Filesystem ** tracer Sounds like the work has been done now but above is some pushback in case we want to undo this barrier on Mikhails' patch. > Infrastructure for pluggable consensus service > -- > > Key: HBASE-11025 > URL: https://issues.apache.org/jira/browse/HBASE-11025 > Project: HBase > Issue Type: Sub-task > Components: Zookeeper >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Attachments: HBASE-11025(add param in HRS ctor).patch, > HBASE-11025(add param in HRS ctor).patch, HBASE-11025.patch > > > Related to HBASE-10915. > In this jira I will extract the changed for property changes, factory and > consensus provider interface (+ZK implementation), as it's going to be > required by other subtasks of HBASE-10909. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11031) Some HTable's are not closed in TestLogRolling
[
https://issues.apache.org/jira/browse/HBASE-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976013#comment-13976013
]
Ted Yu commented on HBASE-11031:
If every change of priority were accompanied by proper comment, it would have
made communication smoother.
Will do that in the future.
Probably I didn't read comments carefully enough, let me re-read them.
> Some HTable's are not closed in TestLogRolling
> --
>
> Key: HBASE-11031
> URL: https://issues.apache.org/jira/browse/HBASE-11031
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Trivial
> Fix For: 0.99.0
>
> Attachments: 11031-v1.txt
>
>
> The following pattern appears in several methods:
> {code}
> // When the hbase:meta table can be opened, the region servers are running
> new HTable(TEST_UTIL.getConfiguration(), TableName.META_TABLE_NAME);
> {code}
> The HTable instance should be closed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10873) Control number of regions assigned to backup masters
[ https://issues.apache.org/jira/browse/HBASE-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976006#comment-13976006 ] Enis Soztutar commented on HBASE-10873: --- bq. Sounds like you'd like us to replicate the old deploy layout Enis Soztutar but with option to move to the 'new'. Yes. Least amount of surprises. Since we are right now defaulting to master having no user level regions, we should also default to backup masters not having any user level regions. It may be ok to change the defaults so that both active and backup masters have normal region load (weight 1), but I think we should be consistent for region loads of active and backup masters. Since META is already shared in 0.98- setups, it should be fine to have master be just another region server. In that case, we do not even need this issue? > Control number of regions assigned to backup masters > > > Key: HBASE-10873 > URL: https://issues.apache.org/jira/browse/HBASE-10873 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.99.0 > > Attachments: hbase-10873.patch, hbase-10873_v2.patch > > > By default, a backup master is treated just like another regionserver. So it > can host as many regions as other regionserver does. When the backup master > becomes the active one, region balancer needs to move those user regions on > this master to other region servers. To minimize the impact, it's better not > to assign too many regions on backup masters. It may not be good to leave the > backup masters idle and not host any region either. > We should make this adjustable so that users can control how many regions to > assign to each backup master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11044) [Shell] Show groups for user in 'whoami' output
[ https://issues.apache.org/jira/browse/HBASE-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976005#comment-13976005 ] Hudson commented on HBASE-11044: FAILURE: Integrated in HBase-0.98 #287 (See [https://builds.apache.org/job/HBase-0.98/287/]) HBASE-11044 [Shell] Show groups for user in 'whoami' output (apurtell: rev 1588948) * /hbase/branches/0.98/hbase-shell/src/main/ruby/shell/commands/whoami.rb > [Shell] Show groups for user in 'whoami' output > --- > > Key: HBASE-11044 > URL: https://issues.apache.org/jira/browse/HBASE-11044 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 0.99.0, 0.98.2 > > Attachments: HBASE-11044.patch > > > The 'whoami' command should show the list of groups for the current user > returned by the configured group mapper. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11045) Replace deprecated method FileSystem#createNonRecursive
[ https://issues.apache.org/jira/browse/HBASE-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975988#comment-13975988 ] Enis Soztutar commented on HBASE-11045: --- [~ted_yu] can you please like the Hdfs issue here. > Replace deprecated method FileSystem#createNonRecursive > --- > > Key: HBASE-11045 > URL: https://issues.apache.org/jira/browse/HBASE-11045 > Project: HBase > Issue Type: Task >Reporter: Gustavo Anatoly >Assignee: Gustavo Anatoly >Priority: Minor > Fix For: 0.99.0 > > > This change affect directly ProtobufLogWriter#init() associated to > TestHLog#testFailedToCreateHLogIfParentRenamed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11032) Replace deprecated methods in FileSystem with their replacements
[
https://issues.apache.org/jira/browse/HBASE-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975986#comment-13975986
]
Enis Soztutar commented on HBASE-11032:
---
Linked HBASE-11045.
+1 for the v1 patch.
> Replace deprecated methods in FileSystem with their replacements
>
>
> Key: HBASE-11032
> URL: https://issues.apache.org/jira/browse/HBASE-11032
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-11032-v1.patch, HBASE-11032.patch
>
>
> FileStatus#isDir() is deprecated.
> FileStatus#isDirectory() should be called instead.
> Here is the list of deprecated methods in FileSystem :
> {code}
> public String getName()
> public static FileSystem getNamed(String name, Configuration conf)
> public FSDataOutputStream createNonRecursive(Path f,
> boolean overwrite,
> int bufferSize, short replication, long blockSize,
> Progressable progress) throws IOException {
>public FSDataOutputStream createNonRecursive(Path f, FsPermission
> permission,
>boolean overwrite, int bufferSize, short replication, long blockSize,
>Progressable progress) throws IOException {
> public short getReplication(Path src) throws IOException {
> public boolean delete(Path f) throws IOException {
> public long getLength(Path f) throws IOException {
> public long getBlockSize(Path f) throws IOException {
> public long getDefaultBlockSize() {
> public short getDefaultReplication()
> {code}
> Except for createNonRecursive() which doesn't have non-deprecated equivalent
> in DistributedFileSystem, deprecated methods are replaced with their
> non-deprecated counterparts.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11032) Replace deprecated methods in FileSystem with their replacements
[
https://issues.apache.org/jira/browse/HBASE-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975984#comment-13975984
]
Hadoop QA commented on HBASE-11032:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641081/HBASE-11032-v1.patch
against trunk revision .
ATTACHMENT ID: 12641081
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 48 new
or modified tests.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9354//console
This message is automatically generated.
> Replace deprecated methods in FileSystem with their replacements
>
>
> Key: HBASE-11032
> URL: https://issues.apache.org/jira/browse/HBASE-11032
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-11032-v1.patch, HBASE-11032.patch
>
>
> FileStatus#isDir() is deprecated.
> FileStatus#isDirectory() should be called instead.
> Here is the list of deprecated methods in FileSystem :
> {code}
> public String getName()
> public static FileSystem getNamed(String name, Configuration conf)
> public FSDataOutputStream createNonRecursive(Path f,
> boolean overwrite,
> int bufferSize, short replication, long blockSize,
> Progressable progress) throws IOException {
>public FSDataOutputStream createNonRecursive(Path f, FsPermission
> permission,
>boolean overwrite, int bufferSize, short replication, long blockSize,
>Progressable progress) throws IOException {
> public short getReplication(Path src) throws IOException {
> public boolean delete(Path f) throws IOException {
> public long getLength(Path f) throws IOException {
> public long getBlockSize(Path f) throws IOException {
> public long getDefaultBlockSize() {
> public short getDefaultReplication()
> {code}
> Except for createNonRecursive() which doesn't have non-deprecated equivalent
> in DistributedFileSystem, deprecated methods are replaced with their
> non-deprecated counterparts.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11047) Remove TimeoutMontior
Jimmy Xiang created HBASE-11047: --- Summary: Remove TimeoutMontior Key: HBASE-11047 URL: https://issues.apache.org/jira/browse/HBASE-11047 Project: HBase Issue Type: Improvement Components: Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor With HBASE-8002, the TimeoutMonitor is disabled by default. Lately, we haven't see much problem of region assignments during integration testing with CM. I was thinking it may be time to remove the timeout monitor now? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11026) Provide option to filter out all rows in PerformanceEvaluation tool
[ https://issues.apache.org/jira/browse/HBASE-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975976#comment-13975976 ] stack commented on HBASE-11026: --- I don't see filterAll option in the usage string (I think you added something else recently w/o adding it to the usage string). It should be in the usage string so folks know it exists. Does this thing work for you? You can add to usage on commit (if it works for you). +1 Good on you [~ramkrishna.s.vasude...@gmail.com] > Provide option to filter out all rows in PerformanceEvaluation tool > --- > > Key: HBASE-11026 > URL: https://issues.apache.org/jira/browse/HBASE-11026 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.99.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-11026_1.patch, HBASE-11026_2.patch > > > Performance Evaluation could also be used to check the actual performance of > the scans on the Server side by passing Filters that filters out all the > rows. We can create a test filter and add it to the Filter.proto and set > this filter based on input params. Could be helpful in testing. > If you feel this is not needed pls feel free to close this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10873) Control number of regions assigned to backup masters
[ https://issues.apache.org/jira/browse/HBASE-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975974#comment-13975974 ] stack commented on HBASE-10873: --- bq. I was thinking of setting the value to 0, which gives us the 0.98- behavior of backup masters not hosting any regions. If users want to use extra capacity they can enable this setting manually. I suggest we let this issue in and then elsewhere discuss what 1.0 rolls out with elsewhere -- in subtask on the 1.0 issue. I'd be in favor of master and backup masters carrying full complement of regions in 1.0 (if we can ensure that the master handlers are processed ahead of any others); i.e. more radical than Jimmy has it his adding support for master and backup masters carrying 'light' loadings. Sounds like you'd like us to replicate the old deploy layout [~enis] but with option to move to the 'new'. > Control number of regions assigned to backup masters > > > Key: HBASE-10873 > URL: https://issues.apache.org/jira/browse/HBASE-10873 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.99.0 > > Attachments: hbase-10873.patch, hbase-10873_v2.patch > > > By default, a backup master is treated just like another regionserver. So it > can host as many regions as other regionserver does. When the backup master > becomes the active one, region balancer needs to move those user regions on > this master to other region servers. To minimize the impact, it's better not > to assign too many regions on backup masters. It may not be good to leave the > backup masters idle and not host any region either. > We should make this adjustable so that users can control how many regions to > assign to each backup master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10994) HBase Multi-Tenancy
[ https://issues.apache.org/jira/browse/HBASE-10994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975963#comment-13975963 ] stack commented on HBASE-10994: --- [~giacomotaylor] You are not suggesting that apache phoenix 'solves' multi-tenancy, are you? Instead, I think you meant to write encouraging words about the great work [~mbertozzi] is doing here adding primitives to hbase that projects like phoenix can make use of and publish nice apis against -- right? (smile). > HBase Multi-Tenancy > --- > > Key: HBASE-10994 > URL: https://issues.apache.org/jira/browse/HBASE-10994 > Project: HBase > Issue Type: Umbrella >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: multitenancy, quota > > Currently HBase treats all tables, users, and workloads in the same way. > This is ok, until multiple users and workloads are applied on the same > cluster/table. Some workloads/users must be prioritized over others, and some > other workloads must not impact others. > We can separate the problem into three components. > * Isolation/Partitioning (Physically split on different machines) > * Scheduling (Prioritize small/interactive workloads vs long/batch workloads) > * Quotas (Limit a user/table requests/sec or size) > This is the umbrella jira tracking the multi-tenancy related tasks. > An initial design document is up for comments here: > https://docs.google.com/document/d/1ygIwZpDWQuMPdfcryckic6ODi5DHQkrzXKjmOJodfs0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11031) Some HTable's are not closed in TestLogRolling
[
https://issues.apache.org/jira/browse/HBASE-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975961#comment-13975961
]
stack commented on HBASE-11031:
---
[~ted_yu] I'm trying to follow along what is going on in this project and your
reopens and reverts w/o comments, non-justification of issue priority after
changing it, or of your non-justification of what an issue is fixing makes
keeping up take more time than it needs to.
> Some HTable's are not closed in TestLogRolling
> --
>
> Key: HBASE-11031
> URL: https://issues.apache.org/jira/browse/HBASE-11031
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Trivial
> Fix For: 0.99.0
>
> Attachments: 11031-v1.txt
>
>
> The following pattern appears in several methods:
> {code}
> // When the hbase:meta table can be opened, the region servers are running
> new HTable(TEST_UTIL.getConfiguration(), TableName.META_TABLE_NAME);
> {code}
> The HTable instance should be closed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10929) Change ScanQueryMatcher to use Cells instead of KeyValue.
[ https://issues.apache.org/jira/browse/HBASE-10929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975955#comment-13975955 ] stack commented on HBASE-10929: --- I skimmed v4. lgtm. > Change ScanQueryMatcher to use Cells instead of KeyValue. > - > > Key: HBASE-10929 > URL: https://issues.apache.org/jira/browse/HBASE-10929 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-10929.patch, HBASE-10929_1.patch, > HBASE-10929_2.patch, HBASE-10929_3.patch, HBASE-10929_4.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11031) Some HTable's are not closed in TestLogRolling
[
https://issues.apache.org/jira/browse/HBASE-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975953#comment-13975953
]
stack commented on HBASE-11031:
---
[~ted_yu] Misquote. Reread.
> Some HTable's are not closed in TestLogRolling
> --
>
> Key: HBASE-11031
> URL: https://issues.apache.org/jira/browse/HBASE-11031
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Trivial
> Fix For: 0.99.0
>
> Attachments: 11031-v1.txt
>
>
> The following pattern appears in several methods:
> {code}
> // When the hbase:meta table can be opened, the region servers are running
> new HTable(TEST_UTIL.getConfiguration(), TableName.META_TABLE_NAME);
> {code}
> The HTable instance should be closed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11044) [Shell] Show groups for user in 'whoami' output
[
https://issues.apache.org/jira/browse/HBASE-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975948#comment-13975948
]
Hadoop QA commented on HBASE-11044:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641083/HBASE-11044.patch
against trunk revision .
ATTACHMENT ID: 12641083
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.backup.TestHFileArchiving
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestTableMapReduceBase.testMultiRegionTable(TestTableMapReduceBase.java:96)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9353//console
This message is automatically generated.
> [Shell] Show groups for user in 'whoami' output
> ---
>
> Key: HBASE-11044
> URL: https://issues.apache.org/jira/browse/HBASE-11044
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.98.1
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 0.99.0, 0.98.2
>
> Attachments: HBASE-11044.patch
>
>
> The 'whoami' command should show the list of groups for the current user
> returned by the configured group mapper.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11046) New Scanner API
Yi Deng created HBASE-11046: --- Summary: New Scanner API Key: HBASE-11046 URL: https://issues.apache.org/jira/browse/HBASE-11046 Project: HBase Issue Type: New Feature Components: Scanners Reporter: Yi Deng Fix For: 0.89-fb A new scanner API for reducing unnecessary RPC calls: Motivation: # RPC is expensive to both client and server. # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation) Solution: # a new scanner API (scannerOpen) which has an option of transfer data along with the scannerID back in this call # a new scanner API (scannerNext) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner. # the current scannerClose is still useful when you want to close the scanner before reach the end. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11023) Port HBASE-10488 "'mvn site' is broken due to org.apache.jasper.JspC not found" to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975910#comment-13975910 ] Ted Yu commented on HBASE-11023: With the command line above, I hit the same error. > Port HBASE-10488 "'mvn site' is broken due to org.apache.jasper.JspC not > found" to 0.98 > --- > > Key: HBASE-11023 > URL: https://issues.apache.org/jira/browse/HBASE-11023 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.98.2 > > Attachments: 11023.txt > > > This is to port the fix (mostly in hbase-thrift/pom.xml) for the site target. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11023) Port HBASE-10488 "'mvn site' is broken due to org.apache.jasper.JspC not found" to 0.98
[
https://issues.apache.org/jira/browse/HBASE-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975903#comment-13975903
]
Konstantin Boudnik commented on HBASE-11023:
Hitting this everytime. Now right on the top-level module:
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project
hbase: failed to get report for org.apache.maven.plugins:maven-javadoc-plugin:
Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run
(generate) on project hbase-thrift: An Ant BuildException has occured: taskdef
class org.apache.jasper.JspC cannot be found
{noformat}
Here's the command line used by Bigtop:
{noformat}
mvn -DskipTests -Dhadoop.profile=23 -Dslf4j.version=1.6.1
-Dhadoop.version=2.3.0 -Dzookeeper.version=3.4.5 -Phadoop-2.0 clean install
site assembly:assembly
{noformat}
Shall it be changed somehow?
> Port HBASE-10488 "'mvn site' is broken due to org.apache.jasper.JspC not
> found" to 0.98
> ---
>
> Key: HBASE-11023
> URL: https://issues.apache.org/jira/browse/HBASE-11023
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.2
>
> Attachments: 11023.txt
>
>
> This is to port the fix (mostly in hbase-thrift/pom.xml) for the site target.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11045) Replace deprecated method FileSystem#createNonRecursive
[ https://issues.apache.org/jira/browse/HBASE-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975893#comment-13975893 ] Gustavo Anatoly commented on HBASE-11045: - Thanks Enis. You have any suggestion? Or this issue should be closed, for while? > Replace deprecated method FileSystem#createNonRecursive > --- > > Key: HBASE-11045 > URL: https://issues.apache.org/jira/browse/HBASE-11045 > Project: HBase > Issue Type: Task >Reporter: Gustavo Anatoly >Assignee: Gustavo Anatoly >Priority: Minor > Fix For: 0.99.0 > > > This change affect directly ProtobufLogWriter#init() associated to > TestHLog#testFailedToCreateHLogIfParentRenamed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10948) Fix hbase table file 'x' mode
[
https://issues.apache.org/jira/browse/HBASE-10948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975890#comment-13975890
]
Jerry He commented on HBASE-10948:
--
Hi, Andrew
Thanks for the info and reminder.
The method FsPermission.getFileDefault() was introduced to Hadoop 2.0 with
HADOOP-9155. Therefore it won't work on hadoop1.
Sorry for the confusion again.
> Fix hbase table file 'x' mode
> -
>
> Key: HBASE-10948
> URL: https://issues.apache.org/jira/browse/HBASE-10948
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration
>Affects Versions: 0.96.2, 0.98.1
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 0.99.0
>
> Attachments: HBASE-10948-trunk-v2.patch, HBASE-10948-trunk.patch
>
>
> The hbase table files currently all have 'x' mode in there:
> {code}
> $hadoop fs -ls -R /hbase/data/default/TestTable/
> drwxr-xr-x - hbase biadmin 0 2014-04-08 20:53
> /hbase/data/default/TestTable/.tabledesc
> -rw-r--r-- 1 hbase biadmin313 2014-04-08 20:53
> /hbase/data/default/TestTable/.tabledesc/.tableinfo.01
> drwxr-xr-x - hbase biadmin 0 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64
> -rwxr-xr-x 1 hbase biadmin 68 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/.regioninfo
> drwxr-xr-x - hbase biadmin 0 2014-04-08 21:54
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/info
> -rwxr-xr-x 1 hbase biadmin 272958577 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/info/7138e61cbcd24538b64726db13dab48e
> -rwxr-xr-x 1 hbase biadmin 108603714 2014-04-08 20:53
> /hbase/data/default/TestTable/724c8c3075da516b450ce4826327ce64/info/9ce233fcdfde49679797d13f28e26129
> drwxr-xr-x - hbase biadmin 0 2014-04-08 20:53
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564
> -rwxr-xr-x 1 hbase biadmin 68 2014-04-08 20:53
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/.regioninfo
> drwxr-xr-x - hbase biadmin 0 2014-04-08 21:54
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/info
> -rwxr-xr-x 1 hbase biadmin 33800049 2014-04-08 21:54
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/info/576190de431341b9a02280654e3deb58
> -rwxr-xr-x 1 hbase biadmin 108650474 2014-04-08 20:53
> /hbase/data/default/TestTable/b5350c581363f786e49ff6f32e71f564/info/7c54098fb62a4ef29aab0f5658b25260
> {code}
> If the user does not specify 'hbase.data.umask', we use the fs default:
> FsPermission.getDefault()
> Instead we should use FsPermission.getFileDefault().
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10873) Control number of regions assigned to backup masters
[ https://issues.apache.org/jira/browse/HBASE-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975883#comment-13975883 ] Jimmy Xiang commented on HBASE-10873: - The patch is just for trunk. I thought we prefer 1.0 to have the new behavior. > Control number of regions assigned to backup masters > > > Key: HBASE-10873 > URL: https://issues.apache.org/jira/browse/HBASE-10873 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.99.0 > > Attachments: hbase-10873.patch, hbase-10873_v2.patch > > > By default, a backup master is treated just like another regionserver. So it > can host as many regions as other regionserver does. When the backup master > becomes the active one, region balancer needs to move those user regions on > this master to other region servers. To minimize the impact, it's better not > to assign too many regions on backup masters. It may not be good to leave the > backup masters idle and not host any region either. > We should make this adjustable so that users can control how many regions to > assign to each backup master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10958) [dataloss] Bulk loading with seqids can prevent some log entries from being replayed
[ https://issues.apache.org/jira/browse/HBASE-10958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-10958: -- Fix Version/s: (was: 0.94.19) 0.94.20 Moving to 0.94.20. > [dataloss] Bulk loading with seqids can prevent some log entries from being > replayed > > > Key: HBASE-10958 > URL: https://issues.apache.org/jira/browse/HBASE-10958 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.94.18 >Reporter: Jean-Daniel Cryans >Assignee: Jean-Daniel Cryans >Priority: Blocker > Fix For: 0.99.0, 0.98.2, 0.96.3, 0.94.20 > > Attachments: HBASE-10958-0.94.patch, > HBASE-10958-less-intrusive-hack-0.96.patch, > HBASE-10958-quick-hack-0.96.patch, HBASE-10958-v2.patch, > HBASE-10958-v3.patch, HBASE-10958.patch > > > We found an issue with bulk loads causing data loss when assigning sequence > ids (HBASE-6630) that is triggered when replaying recovered edits. We're > nicknaming this issue *Blindspot*. > The problem is that the sequence id given to a bulk loaded file is higher > than those of the edits in the region's memstore. When replaying recovered > edits, the rule to skip some of them is that they have to be _lower than the > highest sequence id_. In other words, the edits that have a sequence id lower > than the highest one in the store files *should* have also been flushed. This > is not the case with bulk loaded files since we now have an HFile with a > sequence id higher than unflushed edits. > The log recovery code takes this into account by simply skipping the bulk > loaded files, but this "bulk loaded status" is *lost* on compaction. The > edits in the logs that have a sequence id lower than the bulk loaded file > that got compacted are put in a blind spot and are skipped during replay. > Here's the easiest way to recreate this issue: > - Create an empty table > - Put one row in it (let's say it gets seqid 1) > - Bulk load one file (it gets seqid 2). I used ImporTsv and set > hbase.mapreduce.bulkload.assign.sequenceNumbers. > - Bulk load a second file the same way (it gets seqid 3). > - Major compact the table (the new file has seqid 3 and isn't considered > bulk loaded). > - Kill the region server that holds the table's region. > - Scan the table once the region is made available again. The first row, at > seqid 1, will be missing since the HFile with seqid 3 makes us believe that > everything that came before it was flushed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-9740) A corrupt HFile could cause endless attempts to assign the region without a chance of success
[ https://issues.apache.org/jira/browse/HBASE-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9740: - Fix Version/s: (was: 0.94.19) 0.94.20 I am going to push this one more (last hopefully) time. The issue is that as long as we have the region in transition we know we have something to do. With this we no longer have that after a while and that would be a behavior change in 0.94. Not saying it's wrong, but I would like to discuss this aspect. > A corrupt HFile could cause endless attempts to assign the region without a > chance of success > - > > Key: HBASE-9740 > URL: https://issues.apache.org/jira/browse/HBASE-9740 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.16 >Reporter: Aditya Kishore >Assignee: Ping > Fix For: 0.94.20 > > Attachments: HBase-9740_0.94_v4.patch, HBase-9749_0.94_v2.patch, > HBase-9749_0.94_v3.patch, patch-9740_0.94.txt > > > As described in HBASE-9737, a corrupt HFile in a region could lead to an > assignment storm in the cluster since the Master will keep trying to assign > the region to each region server one after another and obviously none will > succeed. > The region server, upon detecting such a scenario should mark the region as > "RS_ZK_REGION_FAILED_ERROR" (or something to the effect) in the Zookeeper > which should indicate the Master to stop assigning the region until the error > has been resolved (via an HBase shell command, probably "assign"?) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11045) Replace deprecated method FileSystem#createNonRecursive
[ https://issues.apache.org/jira/browse/HBASE-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975852#comment-13975852 ] Enis Soztutar commented on HBASE-11045: --- Any more context on this? FileSystem#createNonRecursive() is deprecated, but Hadoop unfortunately fails to provide a viable alternative.The FileContext API which is supposed to replace the FileSystem API is not mature yet for us to switch. createNonRecursive() is extremely important to do fencing of dead region servers properly. > Replace deprecated method FileSystem#createNonRecursive > --- > > Key: HBASE-11045 > URL: https://issues.apache.org/jira/browse/HBASE-11045 > Project: HBase > Issue Type: Task >Reporter: Gustavo Anatoly >Assignee: Gustavo Anatoly >Priority: Minor > Fix For: 0.99.0 > > > This change affect directly ProtobufLogWriter#init() associated to > TestHLog#testFailedToCreateHLogIfParentRenamed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11041) HBaseTestingUtil.createMultiRegions deals incorrectly with missing column family
[
https://issues.apache.org/jira/browse/HBASE-11041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11041:
--
Fix Version/s: (was: 0.94.19)
0.94.20
> HBaseTestingUtil.createMultiRegions deals incorrectly with missing column
> family
>
>
> Key: HBASE-11041
> URL: https://issues.apache.org/jira/browse/HBASE-11041
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.99.0, 0.98.2, 0.96.3, 0.94.20
>
>
> Just found a test failing like this:
> {code}
> Error Message
> HTableDescriptor is read-only
> Stacktrace
> java.lang.UnsupportedOperationException: HTableDescriptor is read-only
> at
> org.apache.hadoop.hbase.client.UnmodifyableHTableDescriptor.addFamily(UnmodifyableHTableDescriptor.java:64)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.createMultiRegions(HBaseTestingUtility.java:1302)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.createMultiRegions(HBaseTestingUtility.java:1291)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.createMultiRegions(HBaseTestingUtility.java:1286)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.installTable(TestDistributedLogSplitting.java:485)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testMasterStartsUpWithLogSplittingWork(TestDistributedLogSplitting.java:282)
> {code}
> The code that causes this looks like this:
> {code}
> HTableDescriptor htd = table.getTableDescriptor();
> if(!htd.hasFamily(columnFamily)) {
> HColumnDescriptor hcd = new HColumnDescriptor(columnFamily);
> htd.addFamily(hcd);
> }
> {code}
> But note that table.getTableDescriptor() returns an
> UnmodifyableHTableDescriptor, so the add would *always* fail.
> The specific test that failed was
> TestDistributedLogSplitting.testMasterStartsUpWithLogSplittingWork.
> Looks like the HMaster did not have the last table descriptor state, yet.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10994) HBase Multi-Tenancy
[ https://issues.apache.org/jira/browse/HBASE-10994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975849#comment-13975849 ] James Taylor commented on HBASE-10994: -- For existing support of multi-tenancy in Apache Phoenix, see here: http://phoenix.incubator.apache.org/multi-tenancy.html. I think to do multi-tenancy in a good way, you need to hide a lot of details behind a good client API. This is what Apache Phoenix provides. > HBase Multi-Tenancy > --- > > Key: HBASE-10994 > URL: https://issues.apache.org/jira/browse/HBASE-10994 > Project: HBase > Issue Type: Umbrella >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: multitenancy, quota > > Currently HBase treats all tables, users, and workloads in the same way. > This is ok, until multiple users and workloads are applied on the same > cluster/table. Some workloads/users must be prioritized over others, and some > other workloads must not impact others. > We can separate the problem into three components. > * Isolation/Partitioning (Physically split on different machines) > * Scheduling (Prioritize small/interactive workloads vs long/batch workloads) > * Quotas (Limit a user/table requests/sec or size) > This is the umbrella jira tracking the multi-tenancy related tasks. > An initial design document is up for comments here: > https://docs.google.com/document/d/1ygIwZpDWQuMPdfcryckic6ODi5DHQkrzXKjmOJodfs0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10873) Control number of regions assigned to backup masters
[ https://issues.apache.org/jira/browse/HBASE-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975844#comment-13975844 ] Enis Soztutar commented on HBASE-10873: --- bq. Should we set the default value to 1 (backup master is treated the same as a normal region server) or a big number (less regions)? If big, how big should be proper? I was thinking of setting the value to 0, which gives us the 0.98- behavior of backup masters not hosting any regions. If users want to use extra capacity they can enable this setting manually. > Control number of regions assigned to backup masters > > > Key: HBASE-10873 > URL: https://issues.apache.org/jira/browse/HBASE-10873 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.99.0 > > Attachments: hbase-10873.patch, hbase-10873_v2.patch > > > By default, a backup master is treated just like another regionserver. So it > can host as many regions as other regionserver does. When the backup master > becomes the active one, region balancer needs to move those user regions on > this master to other region servers. To minimize the impact, it's better not > to assign too many regions on backup masters. It may not be good to leave the > backup masters idle and not host any region either. > We should make this adjustable so that users can control how many regions to > assign to each backup master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11031) Some HTable's are not closed in TestLogRolling
[
https://issues.apache.org/jira/browse/HBASE-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975826#comment-13975826
]
Ted Yu commented on HBASE-11031:
The feedback I got was:
bq. This fixes nothing
> Some HTable's are not closed in TestLogRolling
> --
>
> Key: HBASE-11031
> URL: https://issues.apache.org/jira/browse/HBASE-11031
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Trivial
> Fix For: 0.99.0
>
> Attachments: 11031-v1.txt
>
>
> The following pattern appears in several methods:
> {code}
> // When the hbase:meta table can be opened, the region servers are running
> new HTable(TEST_UTIL.getConfiguration(), TableName.META_TABLE_NAME);
> {code}
> The HTable instance should be closed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11031) Some HTable's are not closed in TestLogRolling
[
https://issues.apache.org/jira/browse/HBASE-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975819#comment-13975819
]
stack commented on HBASE-11031:
---
Revert but no explanation why:
"
r1588627 | tedyu | 2014-04-19 02:11:02 -0700 (Sat, 19 Apr 2014) | 3 lines
HBASE-11031 Revert"
> Some HTable's are not closed in TestLogRolling
> --
>
> Key: HBASE-11031
> URL: https://issues.apache.org/jira/browse/HBASE-11031
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Trivial
> Fix For: 0.99.0
>
> Attachments: 11031-v1.txt
>
>
> The following pattern appears in several methods:
> {code}
> // When the hbase:meta table can be opened, the region servers are running
> new HTable(TEST_UTIL.getConfiguration(), TableName.META_TABLE_NAME);
> {code}
> The HTable instance should be closed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10929) Change ScanQueryMatcher to use Cells instead of KeyValue.
[ https://issues.apache.org/jira/browse/HBASE-10929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975803#comment-13975803 ] ramkrishna.s.vasudevan commented on HBASE-10929: Can i commit the latest patch? > Change ScanQueryMatcher to use Cells instead of KeyValue. > - > > Key: HBASE-10929 > URL: https://issues.apache.org/jira/browse/HBASE-10929 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-10929.patch, HBASE-10929_1.patch, > HBASE-10929_2.patch, HBASE-10929_3.patch, HBASE-10929_4.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11026) Provide option to filter out all rows in PerformanceEvaluation tool
[ https://issues.apache.org/jira/browse/HBASE-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975802#comment-13975802 ] ramkrishna.s.vasudevan commented on HBASE-11026: Ping!! > Provide option to filter out all rows in PerformanceEvaluation tool > --- > > Key: HBASE-11026 > URL: https://issues.apache.org/jira/browse/HBASE-11026 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.99.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-11026_1.patch, HBASE-11026_2.patch > > > Performance Evaluation could also be used to check the actual performance of > the scans on the Server side by passing Filters that filters out all the > rows. We can create a test filter and add it to the Filter.proto and set > this filter based on input params. Could be helpful in testing. > If you feel this is not needed pls feel free to close this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11044) [Shell] Show groups for user in 'whoami' output
[ https://issues.apache.org/jira/browse/HBASE-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11044: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the ack Matteo. TestShell passes locally. Committed to trunk and 0.98 > [Shell] Show groups for user in 'whoami' output > --- > > Key: HBASE-11044 > URL: https://issues.apache.org/jira/browse/HBASE-11044 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 0.99.0, 0.98.2 > > Attachments: HBASE-11044.patch > > > The 'whoami' command should show the list of groups for the current user > returned by the configured group mapper. -- This message was sent by Atlassian JIRA (v6.2#6252)
