[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049661#comment-14049661 ] stack commented on HBASE-11316: --- bq. Well, it seems like you and JD are in disagreement on this so I am unsure how to proceed. Sorry for confusion caused. Go storefile and store. JD and Esteban have a point especially given that is how we talk up files in UI (metrics also talk storefile). Suggest talking up how relates to hfile and column family since these are notions that are all over the hbase shop too. Thanks Misty. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316-6.patch, HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049654#comment-14049654
]
ramkrishna.s.vasudevan commented on HBASE-11440:
Sometimes even without patch this test hangs for me. Checking it closely.
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11422) Specification of scope is missing for certain Hadoop dependencies
[
https://issues.apache.org/jira/browse/HBASE-11422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049651#comment-14049651
]
stack commented on HBASE-11422:
---
[~cos] Sounds good boss. Will try and learn more about why hbase-testing-util
in meantime.
> Specification of scope is missing for certain Hadoop dependencies
> -
>
> Key: HBASE-11422
> URL: https://issues.apache.org/jira/browse/HBASE-11422
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Konstantin Boudnik
> Fix For: 0.99.0, 1.0.0, 0.98.4
>
> Attachments: HBASE-11422.0.98.patch
>
>
> [~cos] reported in the mailing thread, 'HBase 0.98.x dependency problems',
> that specifying hadoop-two.version with value other than 2.2.0 pulls in
> incorrect dependencies such as:
> {code}
> [INFO] | \- org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.2.0:compile
> {code}
> This is due to missing specification of scope in the pom.xml files.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11294) IntegrationTestIngestWithACL should automatically set the superuser when running on local minicluster
[
https://issues.apache.org/jira/browse/HBASE-11294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049650#comment-14049650
]
ramkrishna.s.vasudevan commented on HBASE-11294:
Andy if this works for you we can commit this to 0.98.4 itself.
> IntegrationTestIngestWithACL should automatically set the superuser when
> running on local minicluster
> -
>
> Key: HBASE-11294
> URL: https://issues.apache.org/jira/browse/HBASE-11294
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.3
>Reporter: Andrew Purtell
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.5
>
> Attachments: HBASE-11294_1.patch, HBASE-11294_2.patch
>
>
> To reproduce:
> {noformat}
> $ mvn -DskipTests clean install
> $ cd hbase-it
> $ mvn verify -Dit.test=IntegrationTestIngestWithACL
> {noformat}
> This should execute successfully according to
> http://hbase.apache.org/book/hbase.tests.html section 16.7.5.1.
> Instead no tables can deploy because the superuser is not automatically set
> to the running user, as what used to happen once upon a time:
> {noformat}
> 2014-06-03 20:15:10,067 WARN [htable-pool12-t1] client.AsyncProcess(675):
> #7, table=hbase:meta, attempt=1/350 failed 1 ops, last exception:
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
> permissions (user=apurtell, scope=hbase:meta, family=info:regioninfo,
> action=WRITE)
> at
> org.apache.hadoop.hbase.security.access.AccessController.prePut(AccessController.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:1122)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:2269)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2244)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2200)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2204)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4263)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3479)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3369)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29503)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:168)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:39)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:111)
> at java.lang.Thread.run(Thread.java:745)
> on localhost,59092,1401826507071, tracking started Tue Jun 03 20:15:10 UTC
> 2014 - FAILED, NOT RETRYING ANYMORE
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049642#comment-14049642
]
Hudson commented on HBASE-11440:
FAILURE: Integrated in HBase-TRUNK #5259 (See
[https://builds.apache.org/job/HBase-TRUNK/5259/])
Revert "HBASE-11440 Make KeyValueCodecWithTags as the default codec for
replication in trunk" (apurtell: rev 7a2527da4b1a654cd8dabedb9475c7173237b1f6)
* hbase-common/src/main/resources/hbase-default.xml
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11422) Specification of scope is missing for certain Hadoop dependencies
[
https://issues.apache.org/jira/browse/HBASE-11422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049626#comment-14049626
]
Konstantin Boudnik commented on HBASE-11422:
Thanks for the comment [~stack]. And yes - the initial patch had classifiers in
it, hence was my comment.
I will have the patches for both trunk and 0.98 (if different) for everything
but hbase-testing-util, so we can solve the most pressing issue before 0.98.4
After that I can have a bit more type fixing it correctly for the
hbase-testing-util module. Does it sound ok?
> Specification of scope is missing for certain Hadoop dependencies
> -
>
> Key: HBASE-11422
> URL: https://issues.apache.org/jira/browse/HBASE-11422
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Konstantin Boudnik
> Fix For: 0.99.0, 1.0.0, 0.98.4
>
> Attachments: HBASE-11422.0.98.patch
>
>
> [~cos] reported in the mailing thread, 'HBase 0.98.x dependency problems',
> that specifying hadoop-two.version with value other than 2.2.0 pulls in
> incorrect dependencies such as:
> {code}
> [INFO] | \- org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.2.0:compile
> {code}
> This is due to missing specification of scope in the pom.xml files.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10885) Support visibility expressions on Deletes
[ https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049628#comment-14049628 ] ramkrishna.s.vasudevan commented on HBASE-10885: Sure. Its in the final stages of review. We got lot of new scenarios while reviewing and also while fixing one we got new types of combinations. Some small minor logical changes were suggested by Anoop on the final review. Just discussing them if the changes (if we make it or not) are fine then we can commit it later today EOD my time. Also Anoop is on travel now so once he finalises the patch we can be ready for commit. > Support visibility expressions on Deletes > - > > Key: HBASE-10885 > URL: https://issues.apache.org/jira/browse/HBASE-10885 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: > 10885-org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes-output.txt, > HBASE-10885_1.patch, HBASE-10885_2.patch, HBASE-10885_new_tag_type_1.patch, > HBASE-10885_new_tag_type_2.patch, HBASE-10885_v1.patch, > HBASE-10885_v12.patch, HBASE-10885_v12.patch, HBASE-10885_v13.patch, > HBASE-10885_v15.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, > HBASE-10885_v2.patch, HBASE-10885_v3.patch, HBASE-10885_v4.patch, > HBASE-10885_v5.patch, HBASE-10885_v7.patch, HBASE-10885_v8.patch, > HBASE-10885_v9.patch > > > Accumulo can specify visibility expressions for delete markers. During > compaction the cells covered by the tombstone are determined in part by > matching the visibility expression. This is useful for the use case of data > set coalescing, where entries from multiple data sets carrying different > labels are combined into one common large table. Later, a subset of entries > can be conveniently removed using visibility expressions. > Currently doing the same in HBase would only be possible with a custom > coprocessor. Otherwise, a Delete will affect all cells covered by the > tombstone regardless of any visibility expression scoping. This is correct > behavior in that no data spill is possible, but certainly could be > surprising, and is only meant to be transitional. We decided not to support > visibility expressions on Deletes to control the complexity of the initial > implementation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049623#comment-14049623
]
Hadoop QA commented on HBASE-11316:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653533/HBASE-11316-6.patch
against trunk revision .
ATTACHMENT ID: 12653533
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ can try raising this value. Default: the value of LONG.MAX_VALUE,
expressed in bytes.
+A StoreFile is a facade of
HFile. In terms of compaction, use of
+
xlink:href="https://issues.apache.org/jira/browse/HBASE-11316";>HBASE-11316.
+drop (filter out) deletes or expired versions, because of
potential side effects. See hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java.
+ will compact larger StoreFiles. However, during
reads, HBase will need to seek
+ through fewer StpreFo;es to accomplish the read.
Consider this approach if you
+
xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836";>Store
+alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class'
=> 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine',
'hbase.hstore.blockingStoreFiles' => '100'}
+alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION =>
{'hbase.hstore.engine.class' =>
'org.apache.hadoop.hbase.regionserver.StripeStoreEngine',
'hbase.hstore.blockingStoreFiles' => '100'}}
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//console
This message is automatically generated.
> Expand info about compactions beyond HBASE-11120
>
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
> Issue Type: Bug
> Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch,
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch,
> HBASE-11316-6.patch, HBASE-113
[jira] [Commented] (HBASE-11452) add userPermission feature in AccessControlClient as client API
[
https://issues.apache.org/jira/browse/HBASE-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049615#comment-14049615
]
Demai Ni commented on HBASE-11452:
--
the failed replication Testcases should not related with this patch. I also
can't find the artifact of findingbugs warnings. The link will hit 404 error,
which is odd.
[~apurtell], thanks for the comments.
bq. can we consider calling this getUserPermission?
sure. I will change the method name.
bq. Do I have a Java variant of Stockholm Syndrome?
would you please elaborate a little bit? do you feel that we have too many ways
to retrieve the same userPermission information? or we should use 'tableName'
directly instead of 'tableRegex'? Thanks
Demai
> add userPermission feature in AccessControlClient as client API
>
>
> Key: HBASE-11452
> URL: https://issues.apache.org/jira/browse/HBASE-11452
> Project: HBase
> Issue Type: Improvement
> Components: Client, security
>Affects Versions: 0.98.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 2.0.0
>
> Attachments: HBASE-11452-master-v0.patch
>
>
> Currently user can 'grant','revoke' and show 'user_permission' through hbase
> shell. And there are client api implemented in AccessControlClient.java for
> 'grant' and 'revoke'. This jira is to add the 'user_permission' feature.
> To keep interface consistant, this jira will also update user_permission.rb
> to use this API directly. The test result is
> {code}
> hbase(main):001:0> user_permission
> User
> Table,Family,Qualifier:Permission
>
> hbase dn:t1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadminetest,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> hive t1_dn,,: [Permission:
> actions=READ,WRITE]
>
> biadmintable1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintable2,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintest_dn,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> 6 row(s) in 1.6220 seconds
> hbase(main):002:0> user_permission 't.*'
> User
> Table,Family,Qualifier:Permission
>
> hive t1_dn,,: [Permission:
> actions=READ,WRITE]
>
> biadmintable1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintable2,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintest_dn,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> 4 row(s) in 0.2130 seconds
> hbase(main):003:0> user_permission 'dn:t1'
> User
> Table,Family,Qualifier:Permission
>
> hbase dn:t1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> 1 row(s) in 0.0790 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11413) [findbugs] RV: Negating the result of compareTo()/compare()
[
https://issues.apache.org/jira/browse/HBASE-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049613#comment-14049613
]
Hadoop QA commented on HBASE-11413:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653525/HBASE-11413.patch
against trunk revision .
ATTACHMENT ID: 12653525
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 10 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9935//console
This message is automatically generated.
> [findbugs] RV: Negating the result of compareTo()/compare()
> ---
>
> Key: HBASE-11413
> URL: https://issues.apache.org/jira/browse/HBASE-11413
> Project: HBase
> Issue Type: Bug
>Reporter: Mike Drob
>Priority: Trivial
> Attachments: HBASE-11413.patch, HBASE-11413.patch
>
>
> Finbugs warns that {{CatalogJanitor.java}}, {{NamespaceUpgrade.java}} and
> {{FSTableDescriptors.java}} have the following warning:
> {quote}
> RV: Negating the result of compareTo()/compare()
> (RV_NEGATING_RESULT_OF_COMPARETO)
> This code negatives the return value of a compareTo or compare method. This
> is a questionable or bad programming practice, since if the return value is
> Integer.MIN_VALUE, negating the return value won't negate the sign of the
> result. You can achieve the same intended result by reversing the order of
> the operands rather than by negating the results.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-4495:
---
Status: Patch Available (was: Open)
> CatalogTracker has an identity crisis; needs to be cut-back in scope
>
>
> Key: HBASE-4495
> URL: https://issues.apache.org/jira/browse/HBASE-4495
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: 1.0.0
> Fix For: 0.99.0
>
> Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
> HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
> HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
> HBASE-4495.patch, HBASE-4495.patch
>
>
> CT needs a good reworking. I'd suggest its scope be cut way down to only
> deal in zk transactions rather than zk and reading meta location in hbase
> (over an HConnection) and being a purveyor of HRegionInterfaces on meta and
> root servers and being an Abortable and a verifier of catalog locations.
> Once this is done, I would suggest it then better belongs over under the zk
> package and that the Meta* classes then move to client package.
> Here's some messy notes I added to head of CT class in hbase-3446 where I
> spent some time trying to make out what it was CT did.
> {code}
> // TODO: This class needs a rethink. The original intent was that it would
> be
> // the one-stop-shop for root and meta locations and that it would get this
> // info from reading and watching zk state. The class was to be used by
> // servers when they needed to know of root and meta movement but also by
> // client-side (inside in HTable) so rather than figure root and meta
> // locations on fault, the client would instead get notifications out of zk.
> //
> // But this original intent is frustrated by the fact that this class has to
> // read an hbase table, the -ROOT- table, to figure out the .META. region
> // location which means we depend on an HConnection. HConnection will do
> // retrying but also, it has its own mechanism for finding root and meta
> // locations (and for 'verifying'; it tries the location and if it fails,
> does
> // new lookup, etc.). So, at least for now, HConnection (or HTable) can't
> // have a CT since CT needs a HConnection (Even then, do want HT to have a
> CT?
> // For HT keep up a session with ZK? Rather, shouldn't we do like
> asynchbase
> // where we'd open a connection to zk, read what we need then let the
> // connection go?). The 'fix' is make it so both root and meta addresses
> // are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
> //
> // But even then, this class does 'verification' of the location and it does
> // this by making a call over an HConnection (which will do its own root
> // and meta lookups). Isn't this verification 'useless' since when we
> // return, whatever is dependent on the result of this call then needs to
> // use HConnection; what we have verified may change in meantime
> (HConnection
> // uses the CT primitives, the root and meta trackers finding root
> locations).
> //
> // When meta is moved to zk, this class may make more sense. In the
> // meantime, it does not cohere. It should just watch meta and root and
> // NOT do verification -- let that be out in HConnection since its going to
> // be done there ultimately anyways.
> //
> // This class has spread throughout the codebase. It needs to be reigned
> in.
> // This class should be used server-side only, even if we move meta location
> // up into zk. Currently its used over in the client package. Its used in
> // MetaReader and MetaEditor classes usually just to get the Configuration
> // its using (It does this indirectly by asking its HConnection for its
> // Configuration and even then this is just used to get an HConnection out
> on
> // the other end). St.Ack 10/23/2011.
> //
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-4495:
---
Status: Open (was: Patch Available)
> CatalogTracker has an identity crisis; needs to be cut-back in scope
>
>
> Key: HBASE-4495
> URL: https://issues.apache.org/jira/browse/HBASE-4495
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: 1.0.0
> Fix For: 0.99.0
>
> Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
> HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
> HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
> HBASE-4495.patch, HBASE-4495.patch
>
>
> CT needs a good reworking. I'd suggest its scope be cut way down to only
> deal in zk transactions rather than zk and reading meta location in hbase
> (over an HConnection) and being a purveyor of HRegionInterfaces on meta and
> root servers and being an Abortable and a verifier of catalog locations.
> Once this is done, I would suggest it then better belongs over under the zk
> package and that the Meta* classes then move to client package.
> Here's some messy notes I added to head of CT class in hbase-3446 where I
> spent some time trying to make out what it was CT did.
> {code}
> // TODO: This class needs a rethink. The original intent was that it would
> be
> // the one-stop-shop for root and meta locations and that it would get this
> // info from reading and watching zk state. The class was to be used by
> // servers when they needed to know of root and meta movement but also by
> // client-side (inside in HTable) so rather than figure root and meta
> // locations on fault, the client would instead get notifications out of zk.
> //
> // But this original intent is frustrated by the fact that this class has to
> // read an hbase table, the -ROOT- table, to figure out the .META. region
> // location which means we depend on an HConnection. HConnection will do
> // retrying but also, it has its own mechanism for finding root and meta
> // locations (and for 'verifying'; it tries the location and if it fails,
> does
> // new lookup, etc.). So, at least for now, HConnection (or HTable) can't
> // have a CT since CT needs a HConnection (Even then, do want HT to have a
> CT?
> // For HT keep up a session with ZK? Rather, shouldn't we do like
> asynchbase
> // where we'd open a connection to zk, read what we need then let the
> // connection go?). The 'fix' is make it so both root and meta addresses
> // are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
> //
> // But even then, this class does 'verification' of the location and it does
> // this by making a call over an HConnection (which will do its own root
> // and meta lookups). Isn't this verification 'useless' since when we
> // return, whatever is dependent on the result of this call then needs to
> // use HConnection; what we have verified may change in meantime
> (HConnection
> // uses the CT primitives, the root and meta trackers finding root
> locations).
> //
> // When meta is moved to zk, this class may make more sense. In the
> // meantime, it does not cohere. It should just watch meta and root and
> // NOT do verification -- let that be out in HConnection since its going to
> // be done there ultimately anyways.
> //
> // This class has spread throughout the codebase. It needs to be reigned
> in.
> // This class should be used server-side only, even if we move meta location
> // up into zk. Currently its used over in the client package. Its used in
> // MetaReader and MetaEditor classes usually just to get the Configuration
> // its using (It does this indirectly by asking its HConnection for its
> // Configuration and even then this is just used to get an HConnection out
> on
> // the other end). St.Ack 10/23/2011.
> //
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049601#comment-14049601
]
ramkrishna.s.vasudevan commented on HBASE-11440:
Also changing the codec should not fail any replication behaviour also. So
this is interesting.
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11316: Attachment: HBASE-11316-6.patch Added a little explanation of StoreFile / HFile and Store/ColumnFamily at the beginning of the section on compaction! > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316-6.patch, HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[
https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049595#comment-14049595
]
Hadoop QA commented on HBASE-10202:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653512/HBASE-10202.patch
against trunk revision .
ATTACHMENT ID: 12653512
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+Usage: rolling-restart.sh [--config ] [--rs-only]
[--master-only] [--graceful] [--maxthreads xx]
+$ for i in `cat conf/regionservers|sort`; do
./bin/graceful_stop.sh --restart --reload --debug $i; done &> /tmp/log.txt
&
+ $ ./bin/hbase-daemon.sh stop master;
./bin/hbase-daemon.sh start master
+$ for i in `cat conf/regionservers|sort`; do
./bin/graceful_stop.sh --restart --reload --debug $i; done &> /tmp/log.txt
&
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileScanning(TestHRegion.java:3299)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9934//console
This message is automatically generated.
> Documentation is lacking information about rolling-restart.sh script.
> -
>
> Key: HBASE-10202
> URL: https://issues.apache.org/jira/browse/HBASE-10202
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Affects Versions: 0.98.0, 0.94.14, 0.99.0, 0.96.1.1
>Reporter: Jean-Marc Spaggiari
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-10202.patch
>
>
> Current documentation is talking about graceful_stop.sh and how to do a
> rolling restart but is not talking about the rolling-restart.sh script. We
> need to document that.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049594#comment-14049594
]
ramkrishna.s.vasudevan commented on HBASE-11440:
Sorry for the mess. I should have seen the failure before commit. My bad. I
generally used to ensure running locally any testcase that fails. Sorry about
that once agian.
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049588#comment-14049588 ] Misty Stanley-Jones commented on HBASE-11316: - Well, it seems like you and JD are in disagreement on this so I am unsure how to proceed. :) > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow
[ https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049581#comment-14049581 ] Hudson commented on HBASE-11240: FAILURE: Integrated in HBase-TRUNK #5258 (See [https://builds.apache.org/job/HBase-TRUNK/5258/]) HBASE-11240 Print hdfs pipeline when hlog's sync is slow (Liu Shaohui) (stack: rev ad78a9cfded3d65f83f4015120746e5c29436b18) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java > Print hdfs pipeline when hlog's sync is slow > > > Key: HBASE-11240 > URL: https://issues.apache.org/jira/browse/HBASE-11240 > Project: HBase > Issue Type: Improvement > Components: Operability, wal >Reporter: Liu Shaohui >Assignee: Liu Shaohui > Fix For: 2.0.0 > > Attachments: HBASE-11240-trunk-v1.diff, HBASE-11240-trunk-v2.diff > > > Sometimes the slow sync of hlog writer is because there is an abnormal > datanode in the pipeline. So it will be helpful to print the pipeline of slow > sync to diagnose those problems. > The ultimate solution is to join the trace of HBase and HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049574#comment-14049574 ] stack commented on HBASE-11316: --- Misty you have enough to go on? Just ask if not. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11452) add userPermission feature in AccessControlClient as client API
[
https://issues.apache.org/jira/browse/HBASE-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049569#comment-14049569
]
Hadoop QA commented on HBASE-11452:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653503/HBASE-11452-master-v0.patch
against trunk revision .
ATTACHMENT ID: 12653503
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testChangeTable(TestReplicaWithCluster.java:217)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9933//console
This message is automatically generated.
> add userPermission feature in AccessControlClient as client API
>
>
> Key: HBASE-11452
> URL: https://issues.apache.org/jira/browse/HBASE-11452
> Project: HBase
> Issue Type: Improvement
> Components: Client, security
>Affects Versions: 0.98.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 2.0.0
>
> Attachments: HBASE-11452-master-v0.patch
>
>
> Currently user can 'grant','revoke' and show 'user_permission' through hbase
> shell. And there are client api implemented in AccessControlClient.java for
> 'grant' and 'revoke'. This jira is to add the 'user_permission' feature.
> To keep interface consistant, this jira will also update user_permission.rb
> to use this API directly. The test result is
> {code}
> hbase(main):001:0> user_permission
> User
> Table,Family,Qualifier:Permission
>
> hbase dn:t1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadminetest,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> hive t1_dn,,: [Permission:
> actions=READ,WRITE]
[jira] [Updated] (HBASE-11413) [findbugs] RV: Negating the result of compareTo()/compare()
[
https://issues.apache.org/jira/browse/HBASE-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11413:
--
Attachment: HBASE-11413.patch
Failures look unrelated. Retrying.
> [findbugs] RV: Negating the result of compareTo()/compare()
> ---
>
> Key: HBASE-11413
> URL: https://issues.apache.org/jira/browse/HBASE-11413
> Project: HBase
> Issue Type: Bug
>Reporter: Mike Drob
>Priority: Trivial
> Attachments: HBASE-11413.patch, HBASE-11413.patch
>
>
> Finbugs warns that {{CatalogJanitor.java}}, {{NamespaceUpgrade.java}} and
> {{FSTableDescriptors.java}} have the following warning:
> {quote}
> RV: Negating the result of compareTo()/compare()
> (RV_NEGATING_RESULT_OF_COMPARETO)
> This code negatives the return value of a compareTo or compare method. This
> is a questionable or bad programming practice, since if the return value is
> Integer.MIN_VALUE, negating the return value won't negate the sign of the
> result. You can achieve the same intended result by reversing the order of
> the operands rather than by negating the results.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-10204) Offheap efforts
[ https://issues.apache.org/jira/browse/HBASE-10204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-10204. --- Resolution: Duplicate Resolving as duplicate. There are enough offheap JIRAs as it is. > Offheap efforts > --- > > Key: HBASE-10204 > URL: https://issues.apache.org/jira/browse/HBASE-10204 > Project: HBase > Issue Type: Umbrella >Reporter: stack > > This is an umbrella issue under which we tie all efforts at offheaping > whether [~apurtell] and crews' effort at single off-heap allocation arena for > blockcache and memstore (includes reading from dfsclient into DBBs which are > then put into an offheap block cache) through @nkeywal reading queries into > direct bytebuffers. > This issue should include links to discussion and design. It also should > host targets such as that posited by [~vrodionov] in HBASE-10191 where going > end-to-end offheap would imply a new read/write pipeline. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11413) [findbugs] RV: Negating the result of compareTo()/compare()
[
https://issues.apache.org/jira/browse/HBASE-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049543#comment-14049543
]
Hadoop QA commented on HBASE-11413:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652492/HBASE-11413.patch
against trunk revision .
ATTACHMENT ID: 12652492
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 10 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestAssignmentManager
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.master.TestTableLockManager
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testChangeTable(TestReplicaWithCluster.java:217)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9932//console
This message is automatically generated.
> [findbugs] RV: Negating the result of compareTo()/compare()
> ---
>
> Key: HBASE-11413
> URL: https://issues.apache.org/jira/browse/HBASE-11413
> Project: HBase
> Issue Type: Bug
>Reporter: Mike Drob
>Priority: Trivial
> Attachments: HBASE-11413.patch
>
>
> Finbugs warns that {{CatalogJanitor.java}}, {{NamespaceUpgrade.java}} and
> {{FSTableDescriptors.java}} have the following warning:
> {quote}
> RV: Negating the result of compareTo()/compare()
> (RV_NEGATING_RESULT_OF_COMPARETO)
> This code negatives the return value of a compareTo or compare method. This
> is a questionable or bad programming practice, since if the return value is
> Integer.MIN_VALUE, negating the return value won't negate the sign of the
> result. You can achieve the same intended result by reversing the order of
> the operands rather than by negating the results.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11455) New region assignmen algorith by scoring
Yi Deng created HBASE-11455: --- Summary: New region assignmen algorith by scoring Key: HBASE-11455 URL: https://issues.apache.org/jira/browse/HBASE-11455 Project: HBase Issue Type: Improvement Reporter: Yi Deng Instead of applying solid rules for selecting secondary and tertiary region servers, we scoring every server and choose the best one. This is a much flexible framework and guarantee to get a valid assignment plan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11283) [0.89-fb] Roll HLog if sync time is anomalous
[ https://issues.apache.org/jira/browse/HBASE-11283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark resolved HBASE-11283. --- Resolution: Fixed > [0.89-fb] Roll HLog if sync time is anomalous > -- > > Key: HBASE-11283 > URL: https://issues.apache.org/jira/browse/HBASE-11283 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.89-fb > Environment: Try and get a new write pipeline if the HLog sync time > is anomalous. >Reporter: Elliott Clark >Assignee: Elliott Clark > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11383) [0.89-fb] Add jitter to HLog rolling period
[ https://issues.apache.org/jira/browse/HBASE-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark resolved HBASE-11383. --- Resolution: Fixed > [0.89-fb] Add jitter to HLog rolling period > --- > > Key: HBASE-11383 > URL: https://issues.apache.org/jira/browse/HBASE-11383 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > > Just in case things go wrong all at once make sure not to ddos the nn by > making the rate limiting jittered. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10303) Have snappy support properly documented would be helpful to hadoop and hbase users
[ https://issues.apache.org/jira/browse/HBASE-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049511#comment-14049511 ] Misty Stanley-Jones commented on HBASE-10303: - I rewrote that whole chapter and I'm awaiting review of it at HBASE-11400 so closing this one since Stack already did most of the work (years ago). > Have snappy support properly documented would be helpful to hadoop and hbase > users > -- > > Key: HBASE-10303 > URL: https://issues.apache.org/jira/browse/HBASE-10303 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Rural Hunter >Assignee: Misty Stanley-Jones > > The currentl document for configuring snappy > support(http://hbase.apache.org/book/snappy.compression.html) is not complete > and it's a bit obscure. IMO, there are several improvments can be made: > 1. Describe the relationship among hadoop,hbase,snappy. Is the snappy > actually needed by hadoop hdfs or hbase itself? That's to make clear if you > need to configure snappy support in hbase or hadoop. > 2. It didn't mention the default hadoop binary package is compiled without > snappy support and you need to compile it with snappy option manually. > Actually it didn't work with any native libs on 64 bits OS as the > libhadoop.so in the binary package is only for 32 bits OS(this of course is a > hadoop issue not hbase. but it's good to mention it.). > 3. In my experience, I actually need to install both snappy and > hadoop-snappy. So the doc lack of the steps to install hadoop-snappy. > 4. During my set up, I found difference where hadoop and hbase to pick up the > native lib files. hadoop picks those files in ./lib while hbase picks in > ./lib/[PLATFORM]. If it's correct, it can also be mentioned. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-10303) Have snappy support properly documented would be helpful to hadoop and hbase users
[ https://issues.apache.org/jira/browse/HBASE-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones resolved HBASE-10303. - Resolution: Fixed > Have snappy support properly documented would be helpful to hadoop and hbase > users > -- > > Key: HBASE-10303 > URL: https://issues.apache.org/jira/browse/HBASE-10303 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Rural Hunter >Assignee: Misty Stanley-Jones > > The currentl document for configuring snappy > support(http://hbase.apache.org/book/snappy.compression.html) is not complete > and it's a bit obscure. IMO, there are several improvments can be made: > 1. Describe the relationship among hadoop,hbase,snappy. Is the snappy > actually needed by hadoop hdfs or hbase itself? That's to make clear if you > need to configure snappy support in hbase or hadoop. > 2. It didn't mention the default hadoop binary package is compiled without > snappy support and you need to compile it with snappy option manually. > Actually it didn't work with any native libs on 64 bits OS as the > libhadoop.so in the binary package is only for 32 bits OS(this of course is a > hadoop issue not hbase. but it's good to mention it.). > 3. In my experience, I actually need to install both snappy and > hadoop-snappy. So the doc lack of the steps to install hadoop-snappy. > 4. During my set up, I found difference where hadoop and hbase to pick up the > native lib files. hadoop picks those files in ./lib while hbase picks in > ./lib/[PLATFORM]. If it's correct, it can also be mentioned. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-2328) Make important configurations more obvious to new users
[ https://issues.apache.org/jira/browse/HBASE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones resolved HBASE-2328. Resolution: Implemented > Make important configurations more obvious to new users > --- > > Key: HBASE-2328 > URL: https://issues.apache.org/jira/browse/HBASE-2328 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Jean-Daniel Cryans >Assignee: Misty Stanley-Jones > Labels: moved_from_0_20_5 > Attachments: HBASE-2328.txt, notsoquick.html > > > Over the last 2 weeks, I encountered many situations where people didn't set > file descriptors and xcievers higher and that was causing a ton of problems > that are hard to debug if you're not used to them. To improve that we should: > - Refuse to start HBase if ulimit -n returns some small number smaller than > 2048, or at least print out in big red blinking letters that the current > configuration is bad and then link to a simple troubleshooting entry on the > wiki. > - Write a clearer Getting Started document where we don't give as much > explanations but add more stuff like "this is what your > hbase-site.xml/hdfs-site/xml should look like now" and give a complete file > example. At this point we don't even give a number for xcievers and we expect > new users to come up with one. > Any other low hanging fruit others can think of? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-2328) Make important configurations more obvious to new users
[ https://issues.apache.org/jira/browse/HBASE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049508#comment-14049508 ] Misty Stanley-Jones commented on HBASE-2328: These things appear to all be addressed now. Please reopen or open a new JIRA if this is not the case. > Make important configurations more obvious to new users > --- > > Key: HBASE-2328 > URL: https://issues.apache.org/jira/browse/HBASE-2328 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Jean-Daniel Cryans > Labels: moved_from_0_20_5 > Attachments: HBASE-2328.txt, notsoquick.html > > > Over the last 2 weeks, I encountered many situations where people didn't set > file descriptors and xcievers higher and that was causing a ton of problems > that are hard to debug if you're not used to them. To improve that we should: > - Refuse to start HBase if ulimit -n returns some small number smaller than > 2048, or at least print out in big red blinking letters that the current > configuration is bad and then link to a simple troubleshooting entry on the > wiki. > - Write a clearer Getting Started document where we don't give as much > explanations but add more stuff like "this is what your > hbase-site.xml/hdfs-site/xml should look like now" and give a complete file > example. At this point we don't even give a number for xcievers and we expect > new users to come up with one. > Any other low hanging fruit others can think of? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-2328) Make important configurations more obvious to new users
[ https://issues.apache.org/jira/browse/HBASE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones reassigned HBASE-2328: -- Assignee: Misty Stanley-Jones > Make important configurations more obvious to new users > --- > > Key: HBASE-2328 > URL: https://issues.apache.org/jira/browse/HBASE-2328 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Jean-Daniel Cryans >Assignee: Misty Stanley-Jones > Labels: moved_from_0_20_5 > Attachments: HBASE-2328.txt, notsoquick.html > > > Over the last 2 weeks, I encountered many situations where people didn't set > file descriptors and xcievers higher and that was causing a ton of problems > that are hard to debug if you're not used to them. To improve that we should: > - Refuse to start HBase if ulimit -n returns some small number smaller than > 2048, or at least print out in big red blinking letters that the current > configuration is bad and then link to a simple troubleshooting entry on the > wiki. > - Write a clearer Getting Started document where we don't give as much > explanations but add more stuff like "this is what your > hbase-site.xml/hdfs-site/xml should look like now" and give a complete file > example. At this point we don't even give a number for xcievers and we expect > new users to come up with one. > Any other low hanging fruit others can think of? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11454) [0.89-fb] HBCK shouldn't check meta and root region info against meta
Elliott Clark created HBASE-11454: - Summary: [0.89-fb] HBCK shouldn't check meta and root region info against meta Key: HBASE-11454 URL: https://issues.apache.org/jira/browse/HBASE-11454 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Meta doesn't contain region info for META and ROOT so the on disk region info can't be checked agains the meta one. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-10303) Have snappy support properly documented would be helpful to hadoop and hbase users
[ https://issues.apache.org/jira/browse/HBASE-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones reassigned HBASE-10303: --- Assignee: Misty Stanley-Jones > Have snappy support properly documented would be helpful to hadoop and hbase > users > -- > > Key: HBASE-10303 > URL: https://issues.apache.org/jira/browse/HBASE-10303 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Rural Hunter >Assignee: Misty Stanley-Jones > > The currentl document for configuring snappy > support(http://hbase.apache.org/book/snappy.compression.html) is not complete > and it's a bit obscure. IMO, there are several improvments can be made: > 1. Describe the relationship among hadoop,hbase,snappy. Is the snappy > actually needed by hadoop hdfs or hbase itself? That's to make clear if you > need to configure snappy support in hbase or hadoop. > 2. It didn't mention the default hadoop binary package is compiled without > snappy support and you need to compile it with snappy option manually. > Actually it didn't work with any native libs on 64 bits OS as the > libhadoop.so in the binary package is only for 32 bits OS(this of course is a > hadoop issue not hbase. but it's good to mention it.). > 3. In my experience, I actually need to install both snappy and > hadoop-snappy. So the doc lack of the steps to install hadoop-snappy. > 4. During my set up, I found difference where hadoop and hbase to pick up the > native lib files. hadoop picks those files in ./lib while hbase picks in > ./lib/[PLATFORM]. If it's correct, it can also be mentioned. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-10202: Status: Patch Available (was: Open) Let me know if this is sufficient. > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.96.1.1, 0.94.14, 0.98.0, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > Attachments: HBASE-10202.patch > > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-10202: Attachment: HBASE-10202.patch > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.98.0, 0.94.14, 0.99.0, 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > Attachments: HBASE-10202.patch > > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-4495:
---
Status: Patch Available (was: Open)
> CatalogTracker has an identity crisis; needs to be cut-back in scope
>
>
> Key: HBASE-4495
> URL: https://issues.apache.org/jira/browse/HBASE-4495
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: 1.0.0
> Fix For: 0.99.0
>
> Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
> HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
> HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
> HBASE-4495.patch, HBASE-4495.patch
>
>
> CT needs a good reworking. I'd suggest its scope be cut way down to only
> deal in zk transactions rather than zk and reading meta location in hbase
> (over an HConnection) and being a purveyor of HRegionInterfaces on meta and
> root servers and being an Abortable and a verifier of catalog locations.
> Once this is done, I would suggest it then better belongs over under the zk
> package and that the Meta* classes then move to client package.
> Here's some messy notes I added to head of CT class in hbase-3446 where I
> spent some time trying to make out what it was CT did.
> {code}
> // TODO: This class needs a rethink. The original intent was that it would
> be
> // the one-stop-shop for root and meta locations and that it would get this
> // info from reading and watching zk state. The class was to be used by
> // servers when they needed to know of root and meta movement but also by
> // client-side (inside in HTable) so rather than figure root and meta
> // locations on fault, the client would instead get notifications out of zk.
> //
> // But this original intent is frustrated by the fact that this class has to
> // read an hbase table, the -ROOT- table, to figure out the .META. region
> // location which means we depend on an HConnection. HConnection will do
> // retrying but also, it has its own mechanism for finding root and meta
> // locations (and for 'verifying'; it tries the location and if it fails,
> does
> // new lookup, etc.). So, at least for now, HConnection (or HTable) can't
> // have a CT since CT needs a HConnection (Even then, do want HT to have a
> CT?
> // For HT keep up a session with ZK? Rather, shouldn't we do like
> asynchbase
> // where we'd open a connection to zk, read what we need then let the
> // connection go?). The 'fix' is make it so both root and meta addresses
> // are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
> //
> // But even then, this class does 'verification' of the location and it does
> // this by making a call over an HConnection (which will do its own root
> // and meta lookups). Isn't this verification 'useless' since when we
> // return, whatever is dependent on the result of this call then needs to
> // use HConnection; what we have verified may change in meantime
> (HConnection
> // uses the CT primitives, the root and meta trackers finding root
> locations).
> //
> // When meta is moved to zk, this class may make more sense. In the
> // meantime, it does not cohere. It should just watch meta and root and
> // NOT do verification -- let that be out in HConnection since its going to
> // be done there ultimately anyways.
> //
> // This class has spread throughout the codebase. It needs to be reigned
> in.
> // This class should be used server-side only, even if we move meta location
> // up into zk. Currently its used over in the client package. Its used in
> // MetaReader and MetaEditor classes usually just to get the Configuration
> // its using (It does this indirectly by asking its HConnection for its
> // Configuration and even then this is just used to get an HConnection out
> on
> // the other end). St.Ack 10/23/2011.
> //
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-4495:
---
Status: Open (was: Patch Available)
> CatalogTracker has an identity crisis; needs to be cut-back in scope
>
>
> Key: HBASE-4495
> URL: https://issues.apache.org/jira/browse/HBASE-4495
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: 1.0.0
> Fix For: 0.99.0
>
> Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
> HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
> HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
> HBASE-4495.patch, HBASE-4495.patch
>
>
> CT needs a good reworking. I'd suggest its scope be cut way down to only
> deal in zk transactions rather than zk and reading meta location in hbase
> (over an HConnection) and being a purveyor of HRegionInterfaces on meta and
> root servers and being an Abortable and a verifier of catalog locations.
> Once this is done, I would suggest it then better belongs over under the zk
> package and that the Meta* classes then move to client package.
> Here's some messy notes I added to head of CT class in hbase-3446 where I
> spent some time trying to make out what it was CT did.
> {code}
> // TODO: This class needs a rethink. The original intent was that it would
> be
> // the one-stop-shop for root and meta locations and that it would get this
> // info from reading and watching zk state. The class was to be used by
> // servers when they needed to know of root and meta movement but also by
> // client-side (inside in HTable) so rather than figure root and meta
> // locations on fault, the client would instead get notifications out of zk.
> //
> // But this original intent is frustrated by the fact that this class has to
> // read an hbase table, the -ROOT- table, to figure out the .META. region
> // location which means we depend on an HConnection. HConnection will do
> // retrying but also, it has its own mechanism for finding root and meta
> // locations (and for 'verifying'; it tries the location and if it fails,
> does
> // new lookup, etc.). So, at least for now, HConnection (or HTable) can't
> // have a CT since CT needs a HConnection (Even then, do want HT to have a
> CT?
> // For HT keep up a session with ZK? Rather, shouldn't we do like
> asynchbase
> // where we'd open a connection to zk, read what we need then let the
> // connection go?). The 'fix' is make it so both root and meta addresses
> // are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
> //
> // But even then, this class does 'verification' of the location and it does
> // this by making a call over an HConnection (which will do its own root
> // and meta lookups). Isn't this verification 'useless' since when we
> // return, whatever is dependent on the result of this call then needs to
> // use HConnection; what we have verified may change in meantime
> (HConnection
> // uses the CT primitives, the root and meta trackers finding root
> locations).
> //
> // When meta is moved to zk, this class may make more sense. In the
> // meantime, it does not cohere. It should just watch meta and root and
> // NOT do verification -- let that be out in HConnection since its going to
> // be done there ultimately anyways.
> //
> // This class has spread throughout the codebase. It needs to be reigned
> in.
> // This class should be used server-side only, even if we move meta location
> // up into zk. Currently its used over in the client package. Its used in
> // MetaReader and MetaEditor classes usually just to get the Configuration
> // its using (It does this indirectly by asking its HConnection for its
> // Configuration and even then this is just used to get an HConnection out
> on
> // the other end). St.Ack 10/23/2011.
> //
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11434) [AccessController] Disallow inbound cells with reserved tags
[ https://issues.apache.org/jira/browse/HBASE-11434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11434: --- Status: Patch Available (was: Open) > [AccessController] Disallow inbound cells with reserved tags > > > Key: HBASE-11434 > URL: https://issues.apache.org/jira/browse/HBASE-11434 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11434.patch, HBASE-11434.patch, HBASE-11434.patch, > HBASE-11434.patch > > > The AccessController allows users to store cells with ACL tags encoded by the > client. This isn't a security issue currently, because in order to store the > cell the user must have a relevant WRITE grant, and the user is allowed to > specify whatever ACL for the cell they'd like. However it could become a > correctness problem in the future, if we introduce format sanity checking or > the like, so let's disallow inbound mutations containing cells with reserved > tags like the VisibilityController does. > The check is skipped if the active user is a superuser. First, superusers are > allowed to do anything. Second, replication (as superuser) must be able to > store incoming cells with ACL tags. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11434) [AccessController] Disallow inbound cells with reserved tags
[ https://issues.apache.org/jira/browse/HBASE-11434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049499#comment-14049499 ] Andrew Purtell commented on HBASE-11434: Replication tests in precommit were broken due to HBASE-11440. Resubmitting for QA anyhow. > [AccessController] Disallow inbound cells with reserved tags > > > Key: HBASE-11434 > URL: https://issues.apache.org/jira/browse/HBASE-11434 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11434.patch, HBASE-11434.patch, HBASE-11434.patch, > HBASE-11434.patch > > > The AccessController allows users to store cells with ACL tags encoded by the > client. This isn't a security issue currently, because in order to store the > cell the user must have a relevant WRITE grant, and the user is allowed to > specify whatever ACL for the cell they'd like. However it could become a > correctness problem in the future, if we introduce format sanity checking or > the like, so let's disallow inbound mutations containing cells with reserved > tags like the VisibilityController does. > The check is skipped if the active user is a superuser. First, superusers are > allowed to do anything. Second, replication (as superuser) must be able to > store incoming cells with ACL tags. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11434) [AccessController] Disallow inbound cells with reserved tags
[ https://issues.apache.org/jira/browse/HBASE-11434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11434: --- Status: Open (was: Patch Available) > [AccessController] Disallow inbound cells with reserved tags > > > Key: HBASE-11434 > URL: https://issues.apache.org/jira/browse/HBASE-11434 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11434.patch, HBASE-11434.patch, HBASE-11434.patch, > HBASE-11434.patch > > > The AccessController allows users to store cells with ACL tags encoded by the > client. This isn't a security issue currently, because in order to store the > cell the user must have a relevant WRITE grant, and the user is allowed to > specify whatever ACL for the cell they'd like. However it could become a > correctness problem in the future, if we introduce format sanity checking or > the like, so let's disallow inbound mutations containing cells with reserved > tags like the VisibilityController does. > The check is skipped if the active user is a superuser. First, superusers are > allowed to do anything. Second, replication (as superuser) must be able to > store incoming cells with ACL tags. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11446) Reduce the frequency of RNG calls in SecureWALCellCodec#EncryptedKvEncoder
[ https://issues.apache.org/jira/browse/HBASE-11446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049498#comment-14049498 ] Andrew Purtell commented on HBASE-11446: Ping [~enis] for branch-1 > Reduce the frequency of RNG calls in SecureWALCellCodec#EncryptedKvEncoder > -- > > Key: HBASE-11446 > URL: https://issues.apache.org/jira/browse/HBASE-11446 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11446.patch > > > By reducing the frequency of RNG calls in > SecureWALCellCodec#EncryptedKvEncoder we can save 37% of on CPU time in that > method and 3% of total on CPU time during an ingest test. WAL processing is a > critical latency sensitive area. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049493#comment-14049493
]
Mikhail Antonov commented on HBASE-11440:
-
Brw, TestReplicaWithCluster also seems to be affected. Fails / hangs on current
master (meaning before revert), passes if I revert.
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11440:
---
Status: Open (was: Patch Available)
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049491#comment-14049491
]
Andrew Purtell commented on HBASE-11440:
I reverted 1d65d5d on master. Let's find out what's going on with this test
before proceeding with further commits.
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11453) TestReplicationDisableInactivePeer test is failing on master
[ https://issues.apache.org/jira/browse/HBASE-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049488#comment-14049488 ] Mikhail Antonov commented on HBASE-11453: - Thanks [~apurtell] ! > TestReplicationDisableInactivePeer test is failing on master > > > Key: HBASE-11453 > URL: https://issues.apache.org/jira/browse/HBASE-11453 > Project: HBase > Issue Type: Bug > Components: Replication, test >Affects Versions: 0.99.0 >Reporter: Mikhail Antonov > > -- > T E S T S > --- > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; > support was removed in 8.0 > Running org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 87.652 sec > <<< FAILURE! > Results : > Failed tests: > testDisableInactivePeer(org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer): > Waited too much time for put replication > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11452) add userPermission feature in AccessControlClient as client API
[
https://issues.apache.org/jira/browse/HBASE-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049486#comment-14049486
]
Andrew Purtell commented on HBASE-11452:
I don't have a strong preference, but can we consider calling this
getUserPermission? Do I have a Java variant of Stockholm Syndrome?
{noformat}
+
+ /**
+ * List all the userPermissions matching the given pattern.
+ * @param conf
+ * @param tableRegex The regular expression string to match against
+ * @return - returns an array of UserPermissions
+ * @throws Throwable
+ */
+ public static List userPermission(Configuration conf, String
tableRegex)
+ throws Throwable {
{noformat}
Patch looks good
> add userPermission feature in AccessControlClient as client API
>
>
> Key: HBASE-11452
> URL: https://issues.apache.org/jira/browse/HBASE-11452
> Project: HBase
> Issue Type: Improvement
> Components: Client, security
>Affects Versions: 0.98.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 2.0.0
>
> Attachments: HBASE-11452-master-v0.patch
>
>
> Currently user can 'grant','revoke' and show 'user_permission' through hbase
> shell. And there are client api implemented in AccessControlClient.java for
> 'grant' and 'revoke'. This jira is to add the 'user_permission' feature.
> To keep interface consistant, this jira will also update user_permission.rb
> to use this API directly. The test result is
> {code}
> hbase(main):001:0> user_permission
> User
> Table,Family,Qualifier:Permission
>
> hbase dn:t1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadminetest,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> hive t1_dn,,: [Permission:
> actions=READ,WRITE]
>
> biadmintable1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintable2,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintest_dn,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> 6 row(s) in 1.6220 seconds
> hbase(main):002:0> user_permission 't.*'
> User
> Table,Family,Qualifier:Permission
>
> hive t1_dn,,: [Permission:
> actions=READ,WRITE]
>
> biadmintable1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintable2,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintest_dn,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> 4 row(s) in 0.2130 seconds
> hbase(main):003:0> user_permission 'dn:t1'
> User
> Table,Family,Qualifier:Permission
>
> hbase dn:t1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> 1 row(s) in 0.0790 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-11453) TestReplicationDisableInactivePeer test is failing on master
[ https://issues.apache.org/jira/browse/HBASE-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-11453. Resolution: Duplicate Fix Version/s: (was: 0.99.0) Technically not a dup but we are going to fix the problem on HBASE-11440 > TestReplicationDisableInactivePeer test is failing on master > > > Key: HBASE-11453 > URL: https://issues.apache.org/jira/browse/HBASE-11453 > Project: HBase > Issue Type: Bug > Components: Replication, test >Affects Versions: 0.99.0 >Reporter: Mikhail Antonov > > -- > T E S T S > --- > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; > support was removed in 8.0 > Running org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 87.652 sec > <<< FAILURE! > Results : > Failed tests: > testDisableInactivePeer(org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer): > Waited too much time for put replication > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049481#comment-14049481
]
Andrew Purtell commented on HBASE-11440:
Looks like this was committed before a failed unit test reported by HadoopQA
was fixed:
{noformat}
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
{noformat}
I am going to revert now.
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11453) TestReplicationDisableInactivePeer test is failing on master
[ https://issues.apache.org/jira/browse/HBASE-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049480#comment-14049480 ] Alex Newman commented on HBASE-11453: - Once I backout HBASE-11440 this seems to pass, interestingly. Trying to figure out if this is happenstance. > TestReplicationDisableInactivePeer test is failing on master > > > Key: HBASE-11453 > URL: https://issues.apache.org/jira/browse/HBASE-11453 > Project: HBase > Issue Type: Bug > Components: Replication, test >Affects Versions: 0.99.0 >Reporter: Mikhail Antonov > > -- > T E S T S > --- > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; > support was removed in 8.0 > Running org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 87.652 sec > <<< FAILURE! > Results : > Failed tests: > testDisableInactivePeer(org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer): > Waited too much time for put replication > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11434) [AccessController] Disallow inbound cells with reserved tags
[ https://issues.apache.org/jira/browse/HBASE-11434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049474#comment-14049474 ] Andrew Purtell commented on HBASE-11434: Ping [~enis] for branch-1 > [AccessController] Disallow inbound cells with reserved tags > > > Key: HBASE-11434 > URL: https://issues.apache.org/jira/browse/HBASE-11434 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11434.patch, HBASE-11434.patch, HBASE-11434.patch, > HBASE-11434.patch > > > The AccessController allows users to store cells with ACL tags encoded by the > client. This isn't a security issue currently, because in order to store the > cell the user must have a relevant WRITE grant, and the user is allowed to > specify whatever ACL for the cell they'd like. However it could become a > correctness problem in the future, if we introduce format sanity checking or > the like, so let's disallow inbound mutations containing cells with reserved > tags like the VisibilityController does. > The check is skipped if the active user is a superuser. First, superusers are > allowed to do anything. Second, replication (as superuser) must be able to > store incoming cells with ACL tags. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9531) a command line (hbase shell) interface to retreive the replication metrics and show replication lag
[
https://issues.apache.org/jira/browse/HBASE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049475#comment-14049475
]
Demai Ni commented on HBASE-9531:
-
strange that the patch submitted a few days ago didn't trigger HadoopQA. I must
miss a procedure step. Can someone give me a hand? thanks a lot... Demai
> a command line (hbase shell) interface to retreive the replication metrics
> and show replication lag
> ---
>
> Key: HBASE-9531
> URL: https://issues.apache.org/jira/browse/HBASE-9531
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Affects Versions: 0.99.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 0.99.0, 0.98.5
>
> Attachments: HBASE-9531-master-v1.patch, HBASE-9531-trunk-v0.patch,
> HBASE-9531-trunk-v0.patch
>
>
> This jira is to provide a command line (hbase shell) interface to retreive
> the replication metrics info such as:ageOfLastShippedOp,
> timeStampsOfLastShippedOp, sizeOfLogQueue ageOfLastAppliedOp, and
> timeStampsOfLastAppliedOp. And also to provide a point of time info of the
> lag of replication(source only)
> Understand that hbase is using Hadoop
> metrics(http://hbase.apache.org/metrics.html), which is a common way to
> monitor metric info. This Jira is to serve as a light-weight client
> interface, comparing to a completed(certainly better, but heavier)GUI
> monitoring package. I made the code works on 0.94.9 now, and like to use this
> jira to get opinions about whether the feature is valuable to other
> users/workshop. If so, I will build a trunk patch.
> All inputs are greatly appreciated. Thank you!
> The overall design is to reuse the existing logic which supports hbase shell
> command 'status', and invent a new module, called ReplicationLoad. In
> HRegionServer.buildServerLoad() , use the local replication service objects
> to get their loads which could be wrapped in a ReplicationLoad object and
> then simply pass it to the ServerLoad. In ReplicationSourceMetrics and
> ReplicationSinkMetrics, a few getters and setters will be created, and ask
> Replication to build a "ReplicationLoad". (many thanks to Jean-Daniel for
> his kindly suggestions through dev email list)
> the replication lag will be calculated for source only, and use this formula:
> {code:title=Replication lag|borderStyle=solid}
> if sizeOfLogQueue != 0 then max(ageOfLastShippedOp, (current time -
> timeStampsOfLastShippedOp)) //err on the large side
> else if (current time - timeStampsOfLastShippedOp) < 2*
> ageOfLastShippedOp then lag = ageOfLastShippedOp // last shipped happen
> recently
> else lag = 0 // last shipped may happens last night, so NO real lag
> although ageOfLastShippedOp is non-zero
> {code}
> External will look something like:
> {code:title=status 'replication'|borderStyle=solid}
> hbase(main):001:0> status 'replication'
> version 0.94.9
> 3 live servers
> Â Â Â hdtest017.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=14, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:49:48 PDT 2013
> Â Â Â SINKÂ :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:48:48 PDT 2013
> Â Â Â hdtest018.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
> Â Â Â SINKÂ :AgeOfLastAppliedOp=14, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:50:59 PDT 2013
> Â Â Â hdtest015.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
> Â Â Â SINKÂ :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:48:48 PDT 2013
> hbase(main):002:0> status 'replication','source'
> version 0.94.9
> 3 live servers
> Â Â Â hdtest017.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=14, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:49:48 PDT 2013
> Â Â Â hdtest018.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
> Â Â Â hdtest015.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
> hbase(main):003:0> status 'replication','sink'
> version 0.94.9
> 3 live servers
> Â Â Â hdtest017.svl.ibm.com:
> Â Â Â SINKÂ :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:48:48 PDT 2013
> Â Â Â hdtest018.svl.ibm.com:
> Â Â Â SINKÂ :AgeOfLastAppliedOp=14, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:50:59 PDT 2013
> Â Â Â hdtest015.svl.ibm.com:
> Â Â Â SINKÂ :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:48:48 PDT 2013
> hbase(main):003:0> status 'replication',
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049476#comment-14049476 ] Jean-Daniel Cryans commented on HBASE-11316: IMO, unless we specifically refer to something related to the HFile implementation, we should always use StoreFile. That's also how we present in the web UI when list the number of Stores and StoreFiles in a region. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11452) add userPermission feature in AccessControlClient as client API
[
https://issues.apache.org/jira/browse/HBASE-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Demai Ni updated HBASE-11452:
-
Status: Patch Available (was: Open)
patch is attached for master branch.
> add userPermission feature in AccessControlClient as client API
>
>
> Key: HBASE-11452
> URL: https://issues.apache.org/jira/browse/HBASE-11452
> Project: HBase
> Issue Type: Improvement
> Components: Client, security
>Affects Versions: 0.98.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 2.0.0
>
> Attachments: HBASE-11452-master-v0.patch
>
>
> Currently user can 'grant','revoke' and show 'user_permission' through hbase
> shell. And there are client api implemented in AccessControlClient.java for
> 'grant' and 'revoke'. This jira is to add the 'user_permission' feature.
> To keep interface consistant, this jira will also update user_permission.rb
> to use this API directly. The test result is
> {code}
> hbase(main):001:0> user_permission
> User
> Table,Family,Qualifier:Permission
>
> hbase dn:t1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadminetest,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> hive t1_dn,,: [Permission:
> actions=READ,WRITE]
>
> biadmintable1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintable2,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintest_dn,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> 6 row(s) in 1.6220 seconds
> hbase(main):002:0> user_permission 't.*'
> User
> Table,Family,Qualifier:Permission
>
> hive t1_dn,,: [Permission:
> actions=READ,WRITE]
>
> biadmintable1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintable2,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> biadmintest_dn,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> 4 row(s) in 0.2130 seconds
> hbase(main):003:0> user_permission 'dn:t1'
> User
> Table,Family,Qualifier:Permission
>
> hbase dn:t1,,: [Permission:
> actions=READ,WRITE,EXEC,CREATE,ADMIN]
>
> 1 row(s) in 0.0790 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11452) add userPermission feature in AccessControlClient as client API
[
https://issues.apache.org/jira/browse/HBASE-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Demai Ni updated HBASE-11452:
-
Description:
Currently user can 'grant','revoke' and show 'user_permission' through hbase
shell. And there are client api implemented in AccessControlClient.java for
'grant' and 'revoke'. This jira is to add the 'user_permission' feature.
To keep interface consistant, this jira will also update user_permission.rb to
use this API directly. The test result is
{code}
hbase(main):001:0> user_permission
User
Table,Family,Qualifier:Permission
hbase dn:t1,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
biadminetest,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
hive t1_dn,,: [Permission:
actions=READ,WRITE]
biadmintable1,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
biadmintable2,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
biadmintest_dn,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
6 row(s) in 1.6220 seconds
hbase(main):002:0> user_permission 't.*'
User
Table,Family,Qualifier:Permission
hive t1_dn,,: [Permission:
actions=READ,WRITE]
biadmintable1,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
biadmintable2,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
biadmintest_dn,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
4 row(s) in 0.2130 seconds
hbase(main):003:0> user_permission 'dn:t1'
User
Table,Family,Qualifier:Permission
hbase dn:t1,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
1 row(s) in 0.0790 seconds
{code}
was:
Currently user can 'grant','revoke' and show 'user_permission' through hbase
shell. And there are client api implemented in AccessControlClient.java for
'grant' and 'revoke'. This jira is to add the 'user_permission' feature.
To keep interface consistant, this jira will also update user_permission.rb to
use this API directly
> add userPermission feature in AccessControlClient as client API
>
>
> Key: HBASE-11452
> URL: https://issues.apache.org/jira/browse/HBASE-11452
> Project: HBase
> Issue Type: Improvement
> Components: Client, security
>Affects Versions: 0.98.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 2.0.0
>
> Attachments: HBASE-11452-master-v0.patch
>
>
> Currently user can 'grant','revoke' and show 'user_permission' through hbase
> shell. And there are client api implemented in AccessControlClient.java for
> 'grant' and 'revoke'. This jira is to add the 'user_permission' feature.
> To keep interface consistant, this jira will also update user_permission.rb
> to use this API directly. The test result is
> {code}
> hbase(main):001:0> user_permission
> User
> Tab
[jira] [Updated] (HBASE-11452) add userPermission feature in AccessControlClient as client API
[ https://issues.apache.org/jira/browse/HBASE-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Demai Ni updated HBASE-11452: - Attachment: HBASE-11452-master-v0.patch > add userPermission feature in AccessControlClient as client API > > > Key: HBASE-11452 > URL: https://issues.apache.org/jira/browse/HBASE-11452 > Project: HBase > Issue Type: Improvement > Components: Client, security >Affects Versions: 0.98.0 >Reporter: Demai Ni >Assignee: Demai Ni > Fix For: 2.0.0 > > Attachments: HBASE-11452-master-v0.patch > > > Currently user can 'grant','revoke' and show 'user_permission' through hbase > shell. And there are client api implemented in AccessControlClient.java for > 'grant' and 'revoke'. This jira is to add the 'user_permission' feature. > To keep interface consistant, this jira will also update user_permission.rb > to use this API directly -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049470#comment-14049470
]
Alex Newman commented on HBASE-11440:
-
This seems to break TestReplicationDisableInactivePeer
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11452) add userPermission feature in AccessControlClient as client API
[ https://issues.apache.org/jira/browse/HBASE-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11452: --- Fix Version/s: (was: 0.99.0) 2.0.0 > add userPermission feature in AccessControlClient as client API > > > Key: HBASE-11452 > URL: https://issues.apache.org/jira/browse/HBASE-11452 > Project: HBase > Issue Type: Improvement > Components: Client, security >Affects Versions: 0.98.0 >Reporter: Demai Ni >Assignee: Demai Ni > Fix For: 2.0.0 > > > Currently user can 'grant','revoke' and show 'user_permission' through hbase > shell. And there are client api implemented in AccessControlClient.java for > 'grant' and 'revoke'. This jira is to add the 'user_permission' feature. > To keep interface consistant, this jira will also update user_permission.rb > to use this API directly -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11452) add userPermission feature in AccessControlClient as client API
[ https://issues.apache.org/jira/browse/HBASE-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11452: --- Affects Version/s: (was: 0.99.0) 0.98.0 > add userPermission feature in AccessControlClient as client API > > > Key: HBASE-11452 > URL: https://issues.apache.org/jira/browse/HBASE-11452 > Project: HBase > Issue Type: Improvement > Components: Client, security >Affects Versions: 0.98.0 >Reporter: Demai Ni >Assignee: Demai Ni > Fix For: 2.0.0 > > > Currently user can 'grant','revoke' and show 'user_permission' through hbase > shell. And there are client api implemented in AccessControlClient.java for > 'grant' and 'revoke'. This jira is to add the 'user_permission' feature. > To keep interface consistant, this jira will also update user_permission.rb > to use this API directly -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11452) add userPermission feature in AccessControlClient as client API
[ https://issues.apache.org/jira/browse/HBASE-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11452: --- Component/s: Client > add userPermission feature in AccessControlClient as client API > > > Key: HBASE-11452 > URL: https://issues.apache.org/jira/browse/HBASE-11452 > Project: HBase > Issue Type: Improvement > Components: Client, security >Affects Versions: 0.98.0 >Reporter: Demai Ni >Assignee: Demai Ni > Fix For: 2.0.0 > > > Currently user can 'grant','revoke' and show 'user_permission' through hbase > shell. And there are client api implemented in AccessControlClient.java for > 'grant' and 'revoke'. This jira is to add the 'user_permission' feature. > To keep interface consistant, this jira will also update user_permission.rb > to use this API directly -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049467#comment-14049467
]
Mikhail Antonov commented on HBASE-4495:
reported failing test in HBASE-11453
> CatalogTracker has an identity crisis; needs to be cut-back in scope
>
>
> Key: HBASE-4495
> URL: https://issues.apache.org/jira/browse/HBASE-4495
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: 1.0.0
> Fix For: 0.99.0
>
> Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
> HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
> HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
> HBASE-4495.patch, HBASE-4495.patch
>
>
> CT needs a good reworking. I'd suggest its scope be cut way down to only
> deal in zk transactions rather than zk and reading meta location in hbase
> (over an HConnection) and being a purveyor of HRegionInterfaces on meta and
> root servers and being an Abortable and a verifier of catalog locations.
> Once this is done, I would suggest it then better belongs over under the zk
> package and that the Meta* classes then move to client package.
> Here's some messy notes I added to head of CT class in hbase-3446 where I
> spent some time trying to make out what it was CT did.
> {code}
> // TODO: This class needs a rethink. The original intent was that it would
> be
> // the one-stop-shop for root and meta locations and that it would get this
> // info from reading and watching zk state. The class was to be used by
> // servers when they needed to know of root and meta movement but also by
> // client-side (inside in HTable) so rather than figure root and meta
> // locations on fault, the client would instead get notifications out of zk.
> //
> // But this original intent is frustrated by the fact that this class has to
> // read an hbase table, the -ROOT- table, to figure out the .META. region
> // location which means we depend on an HConnection. HConnection will do
> // retrying but also, it has its own mechanism for finding root and meta
> // locations (and for 'verifying'; it tries the location and if it fails,
> does
> // new lookup, etc.). So, at least for now, HConnection (or HTable) can't
> // have a CT since CT needs a HConnection (Even then, do want HT to have a
> CT?
> // For HT keep up a session with ZK? Rather, shouldn't we do like
> asynchbase
> // where we'd open a connection to zk, read what we need then let the
> // connection go?). The 'fix' is make it so both root and meta addresses
> // are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
> //
> // But even then, this class does 'verification' of the location and it does
> // this by making a call over an HConnection (which will do its own root
> // and meta lookups). Isn't this verification 'useless' since when we
> // return, whatever is dependent on the result of this call then needs to
> // use HConnection; what we have verified may change in meantime
> (HConnection
> // uses the CT primitives, the root and meta trackers finding root
> locations).
> //
> // When meta is moved to zk, this class may make more sense. In the
> // meantime, it does not cohere. It should just watch meta and root and
> // NOT do verification -- let that be out in HConnection since its going to
> // be done there ultimately anyways.
> //
> // This class has spread throughout the codebase. It needs to be reigned
> in.
> // This class should be used server-side only, even if we move meta location
> // up into zk. Currently its used over in the client package. Its used in
> // MetaReader and MetaEditor classes usually just to get the Configuration
> // its using (It does this indirectly by asking its HConnection for its
> // Configuration and even then this is just used to get an HConnection out
> on
> // the other end). St.Ack 10/23/2011.
> //
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11453) TestReplicationDisableInactivePeer test is failing on master
[ https://issues.apache.org/jira/browse/HBASE-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-11453: Description: -- T E S T S --- Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0 Running org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 87.652 sec <<< FAILURE! Results : Failed tests: testDisableInactivePeer(org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer): Waited too much time for put replication Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 > TestReplicationDisableInactivePeer test is failing on master > > > Key: HBASE-11453 > URL: https://issues.apache.org/jira/browse/HBASE-11453 > Project: HBase > Issue Type: Bug > Components: Replication, test >Affects Versions: 0.99.0 >Reporter: Mikhail Antonov > Fix For: 0.99.0 > > > -- > T E S T S > --- > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; > support was removed in 8.0 > Running org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 87.652 sec > <<< FAILURE! > Results : > Failed tests: > testDisableInactivePeer(org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer): > Waited too much time for put replication > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11434) [AccessController] Disallow inbound cells with reserved tags
[ https://issues.apache.org/jira/browse/HBASE-11434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049466#comment-14049466 ] Anoop Sam John commented on HBASE-11434: +1 > [AccessController] Disallow inbound cells with reserved tags > > > Key: HBASE-11434 > URL: https://issues.apache.org/jira/browse/HBASE-11434 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11434.patch, HBASE-11434.patch, HBASE-11434.patch, > HBASE-11434.patch > > > The AccessController allows users to store cells with ACL tags encoded by the > client. This isn't a security issue currently, because in order to store the > cell the user must have a relevant WRITE grant, and the user is allowed to > specify whatever ACL for the cell they'd like. However it could become a > correctness problem in the future, if we introduce format sanity checking or > the like, so let's disallow inbound mutations containing cells with reserved > tags like the VisibilityController does. > The check is skipped if the active user is a superuser. First, superusers are > allowed to do anything. Second, replication (as superuser) must be able to > store incoming cells with ACL tags. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11453) TestReplicationDisableInactivePeer test is failing on master
Mikhail Antonov created HBASE-11453: --- Summary: TestReplicationDisableInactivePeer test is failing on master Key: HBASE-11453 URL: https://issues.apache.org/jira/browse/HBASE-11453 Project: HBase Issue Type: Bug Components: Replication, test Affects Versions: 0.99.0 Reporter: Mikhail Antonov Fix For: 0.99.0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11321) Restructure Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049464#comment-14049464 ] stack commented on HBASE-11321: --- Looks great. Would move anitpatterns later than Introduction unless AntiPatterns is "When NOT to use HBase". > Restructure Ref Guide > - > > Key: HBASE-11321 > URL: https://issues.apache.org/jira/browse/HBASE-11321 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBase Reference Guide 2.0 -- Proposed Structure, HBase > Reference Guide 2.0 -- Proposed Structure > > > Reorganize Ref Guide to make the important things more prominent and easier > to find. This will be a big multi-step task. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11315) Keeping MVCC for configurable longer time
[ https://issues.apache.org/jira/browse/HBASE-11315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049460#comment-14049460 ] stack commented on HBASE-11315: --- bq. Should we suggest to keep MVCC forever in dev list? I suppose these will occupy space in the bc but in the scheme of things not much? They occupy space anyways since we have a placeholder in the KV/Cell class? If so, could keep them forever. Purging them means we take up less space in the fs but not in bc? If so, I suppose if we are looking at the keys at compaction time anyways, no harm letting them go either. if the cost of keeping them in fs is small and we KNOW they are permanent, we might be able to simplify in a few places and exploit this facility in others. That'd be the argument for keeping them forever. > Keeping MVCC for configurable longer time > -- > > Key: HBASE-11315 > URL: https://issues.apache.org/jira/browse/HBASE-11315 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.99.0 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Attachments: hbase-11315.patch > > > After hbase-8763, we need keep mvcc number longer in hfile so that it can be > used to order changes during writes. For example, the known > put,delete,put,... scenario, cross region server scan, out of order puts(in > recovery case). > Current thinking is that we make the retention period configurable(below > we're using 1 day to explain). During major compaction, we check hfile's > creation time if a hfile creation time is older than 1 day then all mvcc of > KVs in that hfile will be removed. If a hfile is created within 1 day, then > all mvccs of KVs in that hfile will be kept. > In case there are time clock skew, we can firstly sort hfiles based on its > seqId in ascending order and find the first hfile's creation time stamp less > than 1 day. Then mvcc of all hfiles before the found file will be removed > during compaction. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11321) Restructure Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11321: Attachment: HBase Reference Guide 2.0 -- Proposed Structure Version 2 - incorporating feedback so far. > Restructure Ref Guide > - > > Key: HBASE-11321 > URL: https://issues.apache.org/jira/browse/HBASE-11321 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBase Reference Guide 2.0 -- Proposed Structure, HBase > Reference Guide 2.0 -- Proposed Structure > > > Reorganize Ref Guide to make the important things more prominent and easier > to find. This will be a big multi-step task. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11344) Hide row keys and such from the web UIs
[ https://issues.apache.org/jira/browse/HBASE-11344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049455#comment-14049455 ] stack commented on HBASE-11344: --- Why not just put these utility methods into HRI? A dedicated class at the top level strikes me as assigning this little little facility more import than it warrants. Otherwise looks good. Needs release note. > Hide row keys and such from the web UIs > --- > > Key: HBASE-11344 > URL: https://issues.apache.org/jira/browse/HBASE-11344 > Project: HBase > Issue Type: Improvement >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.99.0 > > Attachments: 11344-1.txt, 11344-2.txt > > > The table details on the master UI lists the start row keys of the regions. > The row keys might have sensitive data. We should hide them based on whether > or not the user accessing has the required authorization to view the table.. > To start with, we could make the display of row keys and such based on a > configuration being true or false. If it is false, such potentially sensitive > data is never displayed on the web UI. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049450#comment-14049450
]
stack commented on HBASE-4495:
--
Ok. Then unrelated to this patch. I'm +1 on commit. [~enis] Was going to
apply to 0.99 too... if good by you.
> CatalogTracker has an identity crisis; needs to be cut-back in scope
>
>
> Key: HBASE-4495
> URL: https://issues.apache.org/jira/browse/HBASE-4495
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: 1.0.0
> Fix For: 0.99.0
>
> Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
> HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
> HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
> HBASE-4495.patch, HBASE-4495.patch
>
>
> CT needs a good reworking. I'd suggest its scope be cut way down to only
> deal in zk transactions rather than zk and reading meta location in hbase
> (over an HConnection) and being a purveyor of HRegionInterfaces on meta and
> root servers and being an Abortable and a verifier of catalog locations.
> Once this is done, I would suggest it then better belongs over under the zk
> package and that the Meta* classes then move to client package.
> Here's some messy notes I added to head of CT class in hbase-3446 where I
> spent some time trying to make out what it was CT did.
> {code}
> // TODO: This class needs a rethink. The original intent was that it would
> be
> // the one-stop-shop for root and meta locations and that it would get this
> // info from reading and watching zk state. The class was to be used by
> // servers when they needed to know of root and meta movement but also by
> // client-side (inside in HTable) so rather than figure root and meta
> // locations on fault, the client would instead get notifications out of zk.
> //
> // But this original intent is frustrated by the fact that this class has to
> // read an hbase table, the -ROOT- table, to figure out the .META. region
> // location which means we depend on an HConnection. HConnection will do
> // retrying but also, it has its own mechanism for finding root and meta
> // locations (and for 'verifying'; it tries the location and if it fails,
> does
> // new lookup, etc.). So, at least for now, HConnection (or HTable) can't
> // have a CT since CT needs a HConnection (Even then, do want HT to have a
> CT?
> // For HT keep up a session with ZK? Rather, shouldn't we do like
> asynchbase
> // where we'd open a connection to zk, read what we need then let the
> // connection go?). The 'fix' is make it so both root and meta addresses
> // are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
> //
> // But even then, this class does 'verification' of the location and it does
> // this by making a call over an HConnection (which will do its own root
> // and meta lookups). Isn't this verification 'useless' since when we
> // return, whatever is dependent on the result of this call then needs to
> // use HConnection; what we have verified may change in meantime
> (HConnection
> // uses the CT primitives, the root and meta trackers finding root
> locations).
> //
> // When meta is moved to zk, this class may make more sense. In the
> // meantime, it does not cohere. It should just watch meta and root and
> // NOT do verification -- let that be out in HConnection since its going to
> // be done there ultimately anyways.
> //
> // This class has spread throughout the codebase. It needs to be reigned
> in.
> // This class should be used server-side only, even if we move meta location
> // up into zk. Currently its used over in the client package. Its used in
> // MetaReader and MetaEditor classes usually just to get the Configuration
> // its using (It does this indirectly by asking its HConnection for its
> // Configuration and even then this is just used to get an HConnection out
> on
> // the other end). St.Ack 10/23/2011.
> //
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11344) Hide row keys and such from the web UIs
[
https://issues.apache.org/jira/browse/HBASE-11344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049448#comment-14049448
]
Hadoop QA commented on HBASE-11344:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653464/11344-2.txt
against trunk revision .
ATTACHMENT ID: 12653464
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 15 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ <%=
escapeXml(Bytes.toStringBinary(RegionDetailsForDisplay.getRegionNameForDisplay(regionInfo,
conf))) %>
+ <%=
escapeXml(Bytes.toStringBinary(RegionDetailsForDisplay.getStartKey(regionInfo,
conf))) %>
+ <%=
escapeXml(Bytes.toStringBinary(RegionDetailsForDisplay.getEndKey(regionInfo,
conf))) %>
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testChangeTable(TestReplicaWithCluster.java:217)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9929//console
This message is automatically generated.
> Hide row keys and such from the web UIs
> ---
>
> Key: HBASE-11344
> URL: https://issues.apache.org/jira/browse/HBASE-11344
> Project: HBase
> Issue Type: Improvement
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Fix For: 0.99.0
>
> Attachments: 11344-1.txt, 11344-2.txt
>
>
> The table details on the master UI lists the start row keys of the regions.
> The row keys might have sensitive data. We should hide them based on whether
> or not the user accessing has the required authorization to view the table..
> To start with, we could make the display of row keys and such based on a
> configuration being true or false. If it is false, such potentially sensitive
> data is never displayed on the web UI.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049438#comment-14049438
]
Alex Newman commented on HBASE-4495:
Same fails for me on master
--
T E S T S
---
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
support was removed in 8.0
Running org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 87.652 sec <<<
FAILURE!
Results :
Failed tests:
testDisableInactivePeer(org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer):
Waited too much time for put replication
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
> CatalogTracker has an identity crisis; needs to be cut-back in scope
>
>
> Key: HBASE-4495
> URL: https://issues.apache.org/jira/browse/HBASE-4495
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: 1.0.0
> Fix For: 0.99.0
>
> Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
> HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
> HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
> HBASE-4495.patch, HBASE-4495.patch
>
>
> CT needs a good reworking. I'd suggest its scope be cut way down to only
> deal in zk transactions rather than zk and reading meta location in hbase
> (over an HConnection) and being a purveyor of HRegionInterfaces on meta and
> root servers and being an Abortable and a verifier of catalog locations.
> Once this is done, I would suggest it then better belongs over under the zk
> package and that the Meta* classes then move to client package.
> Here's some messy notes I added to head of CT class in hbase-3446 where I
> spent some time trying to make out what it was CT did.
> {code}
> // TODO: This class needs a rethink. The original intent was that it would
> be
> // the one-stop-shop for root and meta locations and that it would get this
> // info from reading and watching zk state. The class was to be used by
> // servers when they needed to know of root and meta movement but also by
> // client-side (inside in HTable) so rather than figure root and meta
> // locations on fault, the client would instead get notifications out of zk.
> //
> // But this original intent is frustrated by the fact that this class has to
> // read an hbase table, the -ROOT- table, to figure out the .META. region
> // location which means we depend on an HConnection. HConnection will do
> // retrying but also, it has its own mechanism for finding root and meta
> // locations (and for 'verifying'; it tries the location and if it fails,
> does
> // new lookup, etc.). So, at least for now, HConnection (or HTable) can't
> // have a CT since CT needs a HConnection (Even then, do want HT to have a
> CT?
> // For HT keep up a session with ZK? Rather, shouldn't we do like
> asynchbase
> // where we'd open a connection to zk, read what we need then let the
> // connection go?). The 'fix' is make it so both root and meta addresses
> // are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
> //
> // But even then, this class does 'verification' of the location and it does
> // this by making a call over an HConnection (which will do its own root
> // and meta lookups). Isn't this verification 'useless' since when we
> // return, whatever is dependent on the result of this call then needs to
> // use HConnection; what we have verified may change in meantime
> (HConnection
> // uses the CT primitives, the root and meta trackers finding root
> locations).
> //
> // When meta is moved to zk, this class may make more sense. In the
> // meantime, it does not cohere. It should just watch meta and root and
> // NOT do verification -- let that be out in HConnection since its going to
> // be done there ultimately anyways.
> //
> // This class has spread throughout the codebase. It needs to be reigned
> in.
> // This class should be used server-side only, even if we move meta location
> // up into zk. Currently its used over in the client package. Its used in
> // MetaReader and MetaEditor classes usually just to get the Configuration
> // its using (It does this indirectly by asking its HConnection for its
> // Configuration and even then this is just used to get an HConnection out
> on
> // the other end). St.Ack 10/23/2011.
> //
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11452) add userPermission feature in AccessControlClient as client API
Demai Ni created HBASE-11452: Summary: add userPermission feature in AccessControlClient as client API Key: HBASE-11452 URL: https://issues.apache.org/jira/browse/HBASE-11452 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.99.0 Reporter: Demai Ni Assignee: Demai Ni Fix For: 0.99.0 Currently user can 'grant','revoke' and show 'user_permission' through hbase shell. And there are client api implemented in AccessControlClient.java for 'grant' and 'revoke'. This jira is to add the 'user_permission' feature. To keep interface consistant, this jira will also update user_permission.rb to use this API directly -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049430#comment-14049430
]
Mikhail Antonov commented on HBASE-4495:
On my local TestSplitTransactionOnCluster passes w/ and w/ this pach. However,
TestReplicationDisableInactivePeer TestReplicaWithCluster fail even without
this patch on master. Anybody seen that?
> CatalogTracker has an identity crisis; needs to be cut-back in scope
>
>
> Key: HBASE-4495
> URL: https://issues.apache.org/jira/browse/HBASE-4495
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: 1.0.0
> Fix For: 0.99.0
>
> Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
> HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
> HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
> HBASE-4495.patch, HBASE-4495.patch
>
>
> CT needs a good reworking. I'd suggest its scope be cut way down to only
> deal in zk transactions rather than zk and reading meta location in hbase
> (over an HConnection) and being a purveyor of HRegionInterfaces on meta and
> root servers and being an Abortable and a verifier of catalog locations.
> Once this is done, I would suggest it then better belongs over under the zk
> package and that the Meta* classes then move to client package.
> Here's some messy notes I added to head of CT class in hbase-3446 where I
> spent some time trying to make out what it was CT did.
> {code}
> // TODO: This class needs a rethink. The original intent was that it would
> be
> // the one-stop-shop for root and meta locations and that it would get this
> // info from reading and watching zk state. The class was to be used by
> // servers when they needed to know of root and meta movement but also by
> // client-side (inside in HTable) so rather than figure root and meta
> // locations on fault, the client would instead get notifications out of zk.
> //
> // But this original intent is frustrated by the fact that this class has to
> // read an hbase table, the -ROOT- table, to figure out the .META. region
> // location which means we depend on an HConnection. HConnection will do
> // retrying but also, it has its own mechanism for finding root and meta
> // locations (and for 'verifying'; it tries the location and if it fails,
> does
> // new lookup, etc.). So, at least for now, HConnection (or HTable) can't
> // have a CT since CT needs a HConnection (Even then, do want HT to have a
> CT?
> // For HT keep up a session with ZK? Rather, shouldn't we do like
> asynchbase
> // where we'd open a connection to zk, read what we need then let the
> // connection go?). The 'fix' is make it so both root and meta addresses
> // are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
> //
> // But even then, this class does 'verification' of the location and it does
> // this by making a call over an HConnection (which will do its own root
> // and meta lookups). Isn't this verification 'useless' since when we
> // return, whatever is dependent on the result of this call then needs to
> // use HConnection; what we have verified may change in meantime
> (HConnection
> // uses the CT primitives, the root and meta trackers finding root
> locations).
> //
> // When meta is moved to zk, this class may make more sense. In the
> // meantime, it does not cohere. It should just watch meta and root and
> // NOT do verification -- let that be out in HConnection since its going to
> // be done there ultimately anyways.
> //
> // This class has spread throughout the codebase. It needs to be reigned
> in.
> // This class should be used server-side only, even if we move meta location
> // up into zk. Currently its used over in the client package. Its used in
> // MetaReader and MetaEditor classes usually just to get the Configuration
> // its using (It does this indirectly by asking its HConnection for its
> // Configuration and even then this is just used to get an HConnection out
> on
> // the other end). St.Ack 10/23/2011.
> //
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11315) Keeping MVCC for configurable longer time
[
https://issues.apache.org/jira/browse/HBASE-11315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049427#comment-14049427
]
Jeffrey Zhong commented on HBASE-11315:
---
{quote}
isAllFiles is bad name for a param. Should be allFiles
{quote}
Ok, I'll change the name to allFiles upon checkin.
{quote}
Would we ever want to keep sequenceid forever?
{quote}
Yes, personally I want to keep mvcc forever. I'm afraid people want a way to GC
them so I come up the patch to make it configurable. Should we suggest to keep
MVCC forever in dev list?
> Keeping MVCC for configurable longer time
> --
>
> Key: HBASE-11315
> URL: https://issues.apache.org/jira/browse/HBASE-11315
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Attachments: hbase-11315.patch
>
>
> After hbase-8763, we need keep mvcc number longer in hfile so that it can be
> used to order changes during writes. For example, the known
> put,delete,put,... scenario, cross region server scan, out of order puts(in
> recovery case).
> Current thinking is that we make the retention period configurable(below
> we're using 1 day to explain). During major compaction, we check hfile's
> creation time if a hfile creation time is older than 1 day then all mvcc of
> KVs in that hfile will be removed. If a hfile is created within 1 day, then
> all mvccs of KVs in that hfile will be kept.
> In case there are time clock skew, we can firstly sort hfiles based on its
> seqId in ascending order and find the first hfile's creation time stamp less
> than 1 day. Then mvcc of all hfiles before the found file will be removed
> during compaction.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[
https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049424#comment-14049424
]
Hadoop QA commented on HBASE-11367:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653465/hbase-11367_v3.patch
against trunk revision .
ATTACHMENT ID: 12653465
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 22 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterNoCluster
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testChangeTable(TestReplicaWithCluster.java:217)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//console
This message is automatically generated.
> Pluggable replication endpoint
> --
>
> Key: HBASE-11367
> URL: https://issues.apache.org/jira/browse/HBASE-11367
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Blocker
> Fix For: 0.99.0
>
> Attachments: 0001-11367.patch, hbase-11367_v1.patch,
> hbase-11367_v2.patch, hbase-11367_v3.patch
>
>
> We need a pluggable endpoint for replication for more flexibility. See parent
> jira for more context.
> ReplicationSource tails the logs for each peer. This jira introduces
> ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is
> run in the same RS process and instantiated per replication peer per region
> server. Implementations of this interface handle the actual shipping of WAL
> edits to the remote cluster.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11344) Hide row keys and such from the web UIs
[ https://issues.apache.org/jira/browse/HBASE-11344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049413#comment-14049413 ] Ted Yu commented on HBASE-11344: lgtm > Hide row keys and such from the web UIs > --- > > Key: HBASE-11344 > URL: https://issues.apache.org/jira/browse/HBASE-11344 > Project: HBase > Issue Type: Improvement >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.99.0 > > Attachments: 11344-1.txt, 11344-2.txt > > > The table details on the master UI lists the start row keys of the regions. > The row keys might have sensitive data. We should hide them based on whether > or not the user accessing has the required authorization to view the table.. > To start with, we could make the display of row keys and such based on a > configuration being true or false. If it is false, such potentially sensitive > data is never displayed on the web UI. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049410#comment-14049410 ] stack commented on HBASE-11316: --- In the code, StoreFile is a facade over our file implementation HFile. Store is how we implement the modeling concept column family. We should have named Stores 'ColumnFamily' rather than Store. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-11367: Attachment: 0001-11367.patch > Pluggable replication endpoint > -- > > Key: HBASE-11367 > URL: https://issues.apache.org/jira/browse/HBASE-11367 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Blocker > Fix For: 0.99.0 > > Attachments: 0001-11367.patch, hbase-11367_v1.patch, > hbase-11367_v2.patch, hbase-11367_v3.patch > > > We need a pluggable endpoint for replication for more flexibility. See parent > jira for more context. > ReplicationSource tails the logs for each peer. This jira introduces > ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is > run in the same RS process and instantiated per replication peer per region > server. Implementations of this interface handle the actual shipping of WAL > edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049409#comment-14049409 ] Alex Newman commented on HBASE-11367: - I found some small nits while looking over review board. Nothing that should hold anything up, I attached a patch. > Pluggable replication endpoint > -- > > Key: HBASE-11367 > URL: https://issues.apache.org/jira/browse/HBASE-11367 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Blocker > Fix For: 0.99.0 > > Attachments: 0001-11367.patch, hbase-11367_v1.patch, > hbase-11367_v2.patch, hbase-11367_v3.patch > > > We need a pluggable endpoint for replication for more flexibility. See parent > jira for more context. > ReplicationSource tails the logs for each peer. This jira introduces > ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is > run in the same RS process and instantiated per replication peer per region > server. Implementations of this interface handle the actual shipping of WAL > edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11118) non environment variable solution for "IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.Literal
[
https://issues.apache.org/jira/browse/HBASE-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049408#comment-14049408
]
Hadoop QA commented on HBASE-8:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653477/8.098.txt
against trunk revision .
ATTACHMENT ID: 12653477
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 21 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9930//console
This message is automatically generated.
> non environment variable solution for "IllegalAccessError: class
> com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass
> com.google.protobuf.LiteralByteString"
> --
>
> Key: HBASE-8
> URL: https://issues.apache.org/jira/browse/HBASE-8
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: André Kelpe
>Priority: Blocker
> Fix For: 0.99.0
>
> Attachments: 8.098.txt, 8.bytestringer.txt,
> 1118.suggested.undoing.optimization.on.clientside.txt,
> 1118.suggested.undoing.optimization.on.clientside.txt,
> HBASE-8-0.98.patch.gz, HBASE-8-trunk.patch.gz, shade_attempt.patch
>
>
> I am running into the problem described in
> https://issues.apache.org/jira/browse/HBASE-10304, while trying to use a
> newer version within cascading.hbase
> (https://github.com/cascading/cascading.hbase).
> One of the features of cascading.hbase is that you can use it from lingual
> (http://www.cascading.org/projects/lingual/), our SQL layer for hadoop.
> lingual has a notion of providers, which are fat jars that we pull down
> dynamically at runtime. Those jars give users the ability to talk to any
> system or format from SQL. They are added to the classpath programmatically
> before we submit jobs to a hadoop cluster.
> Since lingual does not know upfront , which providers are going to be used in
> a given run, the HADOOP_CLASSPATH trick proposed in the JIRA above is really
> clunky and breaks the ease of use we had before. No other provider requires
> this right now.
> It would be great to have a programmatical way to fix this, when using fat
> jars.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11434) [AccessController] Disallow inbound cells with reserved tags
[
https://issues.apache.org/jira/browse/HBASE-11434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049406#comment-14049406
]
Hadoop QA commented on HBASE-11434:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653463/HBASE-11434.patch
against trunk revision .
ATTACHMENT ID: 12653463
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.replication.TestMasterReplication
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testChangeTable(TestReplicaWithCluster.java:217)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9927//console
This message is automatically generated.
> [AccessController] Disallow inbound cells with reserved tags
>
>
> Key: HBASE-11434
> URL: https://issues.apache.org/jira/browse/HBASE-11434
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 0.99.0, 1.0.0, 0.98.4
>
> Attachments: HBASE-11434.patch, HBASE-11434.patch, HBASE-11434.patch,
> HBASE-11434.patch
>
>
> The AccessController allows users to store cells with ACL tags encoded by the
> client. This isn't a security issue currently, because in order to store the
> cell the user must have a relevant WRITE grant, and the user is allowed to
> specify whatever ACL for the cell they'd like. However it could become a
> correctness problem in the future, if we introduce format sanity checking or
> the like, so let's disallow inbound mutations containing cells with reserved
> tags like the VisibilityController does.
> The check is skipped if the active user is a superuser. First, superusers are
> allowed to do anything. Second, replication (as superuser) must be able to
> store incoming cells with ACL tags.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049405#comment-14049405 ] Misty Stanley-Jones commented on HBASE-11316: - OK, then how do they relate? Especially Store / Column Family. It seems really strange to me to talk about an HFile being "in" a Column Family. Can you help me out? > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11057) Improve TestShell coverage of grant and revoke comamnds
[ https://issues.apache.org/jira/browse/HBASE-11057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11057: --- Assignee: Srikanth Srungarapu > Improve TestShell coverage of grant and revoke comamnds > --- > > Key: HBASE-11057 > URL: https://issues.apache.org/jira/browse/HBASE-11057 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Srikanth Srungarapu >Priority: Minor > Fix For: 0.99.0, 0.98.5 > > > The TestShell coverage of grant and revoke commands doesn't seem sufficient > to catch a botch that prevented global grants. Also cover the alternative > grant syntax introduced in HBASE-11001. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049402#comment-14049402 ] stack commented on HBASE-11316: --- Folks seem to update hfile rather than put in place a new version (it is on version 3 at the moment). hfile seems to be known outside hbase (we bulk import hfiles, not storefiles). hfile is used more often in our doc currently: kalashnikov:docbkx stack$ grep hfile *|wc 97 980 11524 kalashnikov:docbkx stack$ grep -i hfile * |wc 2524413 35218 kalashnikov:docbkx stack$ grep -i storefile * |wc 1882386 22833 If you just use storefile, folks will wonder where hfile fits. You are the boss. Pick one. Just explain how the two relate on first use. Good on you misty. Ditto with column family/Store. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11057) Improve TestShell coverage of grant and revoke comamnds
[ https://issues.apache.org/jira/browse/HBASE-11057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049399#comment-14049399 ] Srikanth Srungarapu commented on HBASE-11057: - [~andrew.purt...@gmail.com] Can you please let me know if it is okay for me to pick this up? > Improve TestShell coverage of grant and revoke comamnds > --- > > Key: HBASE-11057 > URL: https://issues.apache.org/jira/browse/HBASE-11057 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 0.98.5 > > > The TestShell coverage of grant and revoke commands doesn't seem sufficient > to catch a botch that prevented global grants. Also cover the alternative > grant syntax introduced in HBASE-11001. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11440) Make KeyValueCodecWithTags as the default codec for replication in trunk
[
https://issues.apache.org/jira/browse/HBASE-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049396#comment-14049396
]
Hudson commented on HBASE-11440:
FAILURE: Integrated in HBase-TRUNK #5257 (See
[https://builds.apache.org/job/HBase-TRUNK/5257/])
HBASE-11440 Make KeyValueCodecWithTags as the default codec for (ramkrishna:
rev 1d65d5dfa61c3a7c5767e8ce01552bcf16303f8b)
* hbase-common/src/main/resources/hbase-default.xml
> Make KeyValueCodecWithTags as the default codec for replication in trunk
>
>
> Key: HBASE-11440
> URL: https://issues.apache.org/jira/browse/HBASE-11440
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-11440.patch, HBASE-11440_1.patch
>
>
> Set
> {code}
>
> hbase.replication.rpc.codec
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags
>
> {code}
> in hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11315) Keeping MVCC for configurable longer time
[ https://issues.apache.org/jira/browse/HBASE-11315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049390#comment-14049390 ] stack commented on HBASE-11315: --- isAllFiles is bad name for a param. Should be allFiles. Or majorCompaction? Would we ever want to keep sequenceid forever? Patch looks good to me. > Keeping MVCC for configurable longer time > -- > > Key: HBASE-11315 > URL: https://issues.apache.org/jira/browse/HBASE-11315 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.99.0 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Attachments: hbase-11315.patch > > > After hbase-8763, we need keep mvcc number longer in hfile so that it can be > used to order changes during writes. For example, the known > put,delete,put,... scenario, cross region server scan, out of order puts(in > recovery case). > Current thinking is that we make the retention period configurable(below > we're using 1 day to explain). During major compaction, we check hfile's > creation time if a hfile creation time is older than 1 day then all mvcc of > KVs in that hfile will be removed. If a hfile is created within 1 day, then > all mvccs of KVs in that hfile will be kept. > In case there are time clock skew, we can firstly sort hfiles based on its > seqId in ascending order and find the first hfile's creation time stamp less > than 1 day. Then mvcc of all hfiles before the found file will be removed > during compaction. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049388#comment-14049388
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653459/HBASE-4495-v7.patch
against trunk revision .
ATTACHMENT ID: 12653459
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 145
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//console
This message is automatically generated.
> CatalogTracker has an identity crisis; needs to be cut-back in scope
>
>
> Key: HBASE-4495
> URL: https://issues.apache.org/jira/browse/HBASE-4495
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: 1.0.0
> Fix For: 0.99.0
>
> Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
> HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
> HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
> HBASE-4495.patch, HBASE-4495.patch
>
>
> CT needs a good reworking. I'd suggest its scope be cut way down to only
> deal in zk transactions rather than zk and reading meta location in hbase
> (over an HConnection) and being a purveyor of HRegionInterfaces on meta and
> root servers and being an Abortable and a verifier of catalog locations.
> Once this is done, I would suggest it then better belongs over under the zk
> package and that the Meta* classes then move to client package.
> Here's some messy notes I added to head of CT class in hbase-3446 where I
> spent some time trying to make out what it was CT did.
> {code}
> // TODO: This class needs a rethink. The original intent was that it would
> be
> // the one-stop-shop for root and meta locations and that it would get this
> // info from reading and watching zk state. The class was to be used by
> // servers when the
[jira] [Resolved] (HBASE-11432) [AccessController] Remove cell first strategy
[ https://issues.apache.org/jira/browse/HBASE-11432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-11432. Resolution: Fixed Pardon, grep missed it, already in 1.0 > [AccessController] Remove cell first strategy > - > > Key: HBASE-11432 > URL: https://issues.apache.org/jira/browse/HBASE-11432 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11432-0.98.patch, HBASE-11432.patch, > HBASE-11432.patch > > > The cell first evaluation strategy for cell ACLs has been a problem since > introduction. It was an afterthought and a mistake. It's not possible to use > at all with the default config on trunk after HBASE-11077. Fairly certain > there are no users. Remove. Deprecate related client API methods on Query. > (Remove on trunk?) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (HBASE-11432) [AccessController] Remove cell first strategy
[ https://issues.apache.org/jira/browse/HBASE-11432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-11432: Reopen for commit to branch-1. Ping [~enis] > [AccessController] Remove cell first strategy > - > > Key: HBASE-11432 > URL: https://issues.apache.org/jira/browse/HBASE-11432 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11432-0.98.patch, HBASE-11432.patch, > HBASE-11432.patch > > > The cell first evaluation strategy for cell ACLs has been a problem since > introduction. It was an afterthought and a mistake. It's not possible to use > at all with the default config on trunk after HBASE-11077. Fairly certain > there are no users. Remove. Deprecate related client API methods on Query. > (Remove on trunk?) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11432) [AccessController] Remove cell first strategy
[ https://issues.apache.org/jira/browse/HBASE-11432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11432: --- Fix Version/s: 1.0.0 > [AccessController] Remove cell first strategy > - > > Key: HBASE-11432 > URL: https://issues.apache.org/jira/browse/HBASE-11432 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11432-0.98.patch, HBASE-11432.patch, > HBASE-11432.patch > > > The cell first evaluation strategy for cell ACLs has been a problem since > introduction. It was an afterthought and a mistake. It's not possible to use > at all with the default config on trunk after HBASE-11077. Fairly certain > there are no users. Remove. Deprecate related client API methods on Query. > (Remove on trunk?) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11064) Odd behaviors of TableName for empty namespace
[
https://issues.apache.org/jira/browse/HBASE-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11064:
---
Fix Version/s: 1.0.0
0.99.0
> Odd behaviors of TableName for empty namespace
> --
>
> Key: HBASE-11064
> URL: https://issues.apache.org/jira/browse/HBASE-11064
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.3
>Reporter: Hiroshi Ikeda
>Assignee: Rekha Joshi
>Priority: Trivial
> Fix For: 0.99.0, 1.0.0, 0.98.4
>
> Attachments: HBASE-11064.1.patch
>
>
> In the class TableName,
> {code}
> public static byte [] isLegalFullyQualifiedTableName(final byte[] tableName) {
> ...
> int namespaceDelimIndex = ...
> if (namespaceDelimIndex == 0 || namespaceDelimIndex == -1){
> isLegalTableQualifierName(tableName);
> } else {
> ...
> {code}
> That means, for example, giving ":a" as the argument throws an exception
> which says invalid qualifier, instead of invalid namespace.
> Also, TableName.valueOf(String) and valueOf(byte[]) can create an instance
> with empty namespace, which is inconsistent.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11039) [VisibilityController] Integration test for labeled data set mixing and filtered excise
[ https://issues.apache.org/jira/browse/HBASE-11039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11039: --- Fix Version/s: 1.0.0 > [VisibilityController] Integration test for labeled data set mixing and > filtered excise > --- > > Key: HBASE-11039 > URL: https://issues.apache.org/jira/browse/HBASE-11039 > Project: HBase > Issue Type: Test >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11039_ITBLL_v1.patch > > > Create an integration test for the VisibilityController that: > 1. Create several tables of test data > 2. Assign a set of auths to each table. Label all entries in the table with > appropriate visibility expressions. Insure that some data in every table > overlaps with data in other tables at common row/family/qualifier > coordinates. Generate data like ITBLL so we can verify all data present later. > 3. Mix the data from the different tables into a new common table > 4. Verify for each set of auths defined in step #2 that all entries found in > the source table can be found in the common table. Like the ITBLL > verification step but done N times for each set of auths defined in step #2. > 5. Choose one of the source tables. Get its set of auths. Perform a deletion > with visibility expression from the common table using those auths. > 6. Verify that no data in the common table with the auth set chosen in #5 > remains. A simple row count with the set of auths chosen in #5 that should > return 0. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10885) Support visibility expressions on Deletes
[ https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-10885: --- Fix Version/s: 1.0.0 This is a blocker for 0.98 but should not go in here before commit to 1.0. Please ping [~enis] when ready. > Support visibility expressions on Deletes > - > > Key: HBASE-10885 > URL: https://issues.apache.org/jira/browse/HBASE-10885 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: > 10885-org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes-output.txt, > HBASE-10885_1.patch, HBASE-10885_2.patch, HBASE-10885_new_tag_type_1.patch, > HBASE-10885_new_tag_type_2.patch, HBASE-10885_v1.patch, > HBASE-10885_v12.patch, HBASE-10885_v12.patch, HBASE-10885_v13.patch, > HBASE-10885_v15.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, > HBASE-10885_v2.patch, HBASE-10885_v3.patch, HBASE-10885_v4.patch, > HBASE-10885_v5.patch, HBASE-10885_v7.patch, HBASE-10885_v8.patch, > HBASE-10885_v9.patch > > > Accumulo can specify visibility expressions for delete markers. During > compaction the cells covered by the tombstone are determined in part by > matching the visibility expression. This is useful for the use case of data > set coalescing, where entries from multiple data sets carrying different > labels are combined into one common large table. Later, a subset of entries > can be conveniently removed using visibility expressions. > Currently doing the same in HBase would only be possible with a custom > coprocessor. Otherwise, a Delete will affect all cells covered by the > tombstone regardless of any visibility expression scoping. This is correct > behavior in that no data spill is possible, but certainly could be > surprising, and is only meant to be transitional. We decided not to support > visibility expressions on Deletes to control the complexity of the initial > implementation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11256) [Visibility Controller] Enhance IntegrationTestWithCellVisibilityLoadAndVerify to verify delete behaviour
[ https://issues.apache.org/jira/browse/HBASE-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11256: --- Fix Version/s: 1.0.0 > [Visibility Controller] Enhance > IntegrationTestWithCellVisibilityLoadAndVerify to verify delete behaviour > - > > Key: HBASE-11256 > URL: https://issues.apache.org/jira/browse/HBASE-11256 > Project: HBase > Issue Type: Test >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11039_1.patch, HBASE-11256_2.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11446) Reduce the frequency of RNG calls in SecureWALCellCodec#EncryptedKvEncoder
[ https://issues.apache.org/jira/browse/HBASE-11446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11446: --- Fix Version/s: 1.0.0 > Reduce the frequency of RNG calls in SecureWALCellCodec#EncryptedKvEncoder > -- > > Key: HBASE-11446 > URL: https://issues.apache.org/jira/browse/HBASE-11446 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11446.patch > > > By reducing the frequency of RNG calls in > SecureWALCellCodec#EncryptedKvEncoder we can save 37% of on CPU time in that > method and 3% of total on CPU time during an ingest test. WAL processing is a > critical latency sensitive area. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11423) Visibility label and per cell ACL feature not working with HTable#mutateRow() and MultiRowMutationEndpoint
[ https://issues.apache.org/jira/browse/HBASE-11423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11423: --- Fix Version/s: 1.0.0 > Visibility label and per cell ACL feature not working with HTable#mutateRow() > and MultiRowMutationEndpoint > -- > > Key: HBASE-11423 > URL: https://issues.apache.org/jira/browse/HBASE-11423 > Project: HBase > Issue Type: Bug > Components: Coprocessors, security >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11423.patch > > > This is because pre/postBatchMutate() APIs are not getting called from > HRegion#processRowsWithLocks() -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11422) Specification of scope is missing for certain Hadoop dependencies
[
https://issues.apache.org/jira/browse/HBASE-11422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11422:
---
Fix Version/s: 1.0.0
> Specification of scope is missing for certain Hadoop dependencies
> -
>
> Key: HBASE-11422
> URL: https://issues.apache.org/jira/browse/HBASE-11422
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Konstantin Boudnik
> Fix For: 0.99.0, 1.0.0, 0.98.4
>
> Attachments: HBASE-11422.0.98.patch
>
>
> [~cos] reported in the mailing thread, 'HBase 0.98.x dependency problems',
> that specifying hadoop-two.version with value other than 2.2.0 pulls in
> incorrect dependencies such as:
> {code}
> [INFO] | \- org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.2.0:compile
> {code}
> This is due to missing specification of scope in the pom.xml files.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11318) Classes in security subpackages missing @InterfaceAudience annotations.
[ https://issues.apache.org/jira/browse/HBASE-11318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11318: --- Fix Version/s: 1.0.0 > Classes in security subpackages missing @InterfaceAudience annotations. > --- > > Key: HBASE-11318 > URL: https://issues.apache.org/jira/browse/HBASE-11318 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.99.0, 0.96.1.1, 0.98.3 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: hbase-11318.patch > > > I was reading some of the security related code and noticed that many of the > security related classes lack @InterfaceAudience markings. > WIth the current api I believe all but Permission should be Private. With > the introduction of cell level ACL's Permission must be public because it is > now exposed in the Mutation setACL calls[1]. > There is an inconsistency with the Mutation ACL -- the acl setters take > Permission instances but the getter returns byte[]'s. As a follow on issue > we could change the signature of Mutation.setACL so we don't have to expose > the Permission class and convert it to be byte[], or change the getter to > return an exposed Permission instance. > [1] > http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Mutation.html#setACL(java.util.Map) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11434) [AccessController] Disallow inbound cells with reserved tags
[ https://issues.apache.org/jira/browse/HBASE-11434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11434: --- Fix Version/s: 1.0.0 > [AccessController] Disallow inbound cells with reserved tags > > > Key: HBASE-11434 > URL: https://issues.apache.org/jira/browse/HBASE-11434 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11434.patch, HBASE-11434.patch, HBASE-11434.patch, > HBASE-11434.patch > > > The AccessController allows users to store cells with ACL tags encoded by the > client. This isn't a security issue currently, because in order to store the > cell the user must have a relevant WRITE grant, and the user is allowed to > specify whatever ACL for the cell they'd like. However it could become a > correctness problem in the future, if we introduce format sanity checking or > the like, so let's disallow inbound mutations containing cells with reserved > tags like the VisibilityController does. > The check is skipped if the active user is a superuser. First, superusers are > allowed to do anything. Second, replication (as superuser) must be able to > store incoming cells with ACL tags. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11088) Support Visibility Expression Deletes in Shell
[ https://issues.apache.org/jira/browse/HBASE-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11088: --- Fix Version/s: 1.0.0 > Support Visibility Expression Deletes in Shell > -- > > Key: HBASE-11088 > URL: https://issues.apache.org/jira/browse/HBASE-11088 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11058.patch, HBASE-11058_2.patch, > HBASE-11088_3.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
