[jira] [Commented] (HBASE-17627) Active workers metric for thrift
[ https://issues.apache.org/jira/browse/HBASE-17627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869105#comment-15869105 ] Hudson commented on HBASE-17627: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2512 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2512/]) HBASE-17627 Active workers metric for thrift (Ashu Pachauri) (garyh: rev d2c083d21ce2b9c6cafe3e30b6093419cacdf118) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/TBoundedThreadPoolServer.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java * (add) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/THBaseThreadPoolExecutor.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSource.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftMetrics.java > Active workers metric for thrift > > > Key: HBASE-17627 > URL: https://issues.apache.org/jira/browse/HBASE-17627 > Project: HBase > Issue Type: Improvement > Components: Thrift >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: HBASE-17627.branch-1.001.patch, > HBASE-17627.master.001.patch > > > It would be good to have a metric for number of active handlers on thrift > servers, which gives a good indication of business of a thrift server. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17312) [JDK8] Use default method for Observer Coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869282#comment-15869282 ] stack commented on HBASE-17312: --- [~appy] Want to describe what CPs will have to do to move to hbase-2.0.0, the one line change (or [~zghaobac]?) > [JDK8] Use default method for Observer Coprocessors > --- > > Key: HBASE-17312 > URL: https://issues.apache.org/jira/browse/HBASE-17312 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17312.master.001.patch, > HBASE-17312.master.001.patch > > > Use default method in MasterObserver, RegionObserver, RegionServerObserver > and WALObserver. And mark the BaseRegionObserver, > BaseMasterAndRegionObserver, BaseRegionServerObserver and BaseWALObserver. > User can implement the interface directly and will not break compatibility > when add new default methods. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869415#comment-15869415
]
huzheng commented on HBASE-17472:
-
Patch for branch-1 will be different, I'll upload it later.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.master.v6.patch, HBASE-17472.v1.patch,
> HBASE-17472.v2.patch, HBASE-17472.v3.patch, HBASE-17472.v4.patch,
> HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17627) Active workers metric for thrift
[ https://issues.apache.org/jira/browse/HBASE-17627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869130#comment-15869130 ] Hudson commented on HBASE-17627: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #106 (See [https://builds.apache.org/job/HBase-1.3-JDK7/106/]) HBASE-17627 Active workers metric for thrift (Ashu Pachauri) (garyh: rev f2c8b5c66e0b607c09cc5b8bf3fef33fa8e468df) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSource.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/TBoundedThreadPoolServer.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftMetrics.java * (add) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/THBaseThreadPoolExecutor.java > Active workers metric for thrift > > > Key: HBASE-17627 > URL: https://issues.apache.org/jira/browse/HBASE-17627 > Project: HBase > Issue Type: Improvement > Components: Thrift >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: HBASE-17627.branch-1.001.patch, > HBASE-17627.master.001.patch > > > It would be good to have a metric for number of active handlers on thrift > servers, which gives a good indication of business of a thrift server. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869140#comment-15869140
]
Hadoop QA commented on HBASE-17652:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 52 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 22s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
3s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 11s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 31s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 20s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 38s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 39s {color}
| {color:red} hbase-thrift in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 48s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 4m 28s {color}
| {color:red} hbase-endpoint in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 28s
{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s
{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 133m 8s {color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
51s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black}
[jira] [Updated] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17069:
--
Affects Version/s: 2.0.0
Fix Version/s: 2.0.0
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,717 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869323#comment-15869323
]
Duo Zhang commented on HBASE-17069:
---
Will commit shortly if no objections.
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,717 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869412#comment-15869412
]
huzheng commented on HBASE-17472:
-
Upload patch v6 for master branch :
1. remove grant2 method in AccessControllerClient, and just keep grant method
(merge behavior) for RpcUser.
2. adjust ut.
For master branch, ruby shell command need no modification.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.master.v6.patch, HBASE-17472.v1.patch,
> HBASE-17472.v2.patch, HBASE-17472.v3.patch, HBASE-17472.v4.patch,
> HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17653) HBASE-17624 rsgroup synchronizations will (distributed) deadlock
[
https://issues.apache.org/jira/browse/HBASE-17653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17653:
--
Description:
Follow-on from HBASE-17624. HBASE-17624 made it so one thread only has access
to the rsgroup administrator. In tail of HBASE-17624 [~toffer] describes
scenario under which we may end up in a deadlock (distributed). Let me repeat
[~toffer] comment...
{code}
Both read/write access can't be single threaded. Consider the situation:
1. move_rsgroup_servers is called
2. while #1 is happening rsgroup region is in transition (rpc thread in #1
holds monitor lock)
3. while #2 is happening meta is in transition.
Balancer tries to figure out plan for meta region tries to get monitor lock but
can't. rpc thread task won't release monitor lock since rsgroup region never
gets assigned. rsgroup region never gets assigned because it can't update meta
with new state.
There's a good chance this can be reproduce just by moving both rsgroup and
meta region onto the same RS and call move_rsgoup_servers on the same RS.
A bunch different actors will query from group affiliation so we can't have
writes block reads.
In the code prior to this patch the getter methods that retrieve group
information (getRSGroup, ofTable, OfServer, etc) don't require the monitor lock
so the deadlock cycle is broken.
The methods that does mutations and updates to zk and hbase:rsgroup are
synchronized appropriately. Point me to where the incoherence is?
{code}
This issue is about testing/fixing/restoring rsgroup access. Will be back.
was:Follow-on from HBASE-17624. HBASE-17624 made it so one thread access to
the rsgroup administrator. In tail of HBASE-17624 [~toffer] describes scenario
under which we may end up in a deadlock (distributed). This issue is to
address this problem.
> HBASE-17624 rsgroup synchronizations will (distributed) deadlock
>
>
> Key: HBASE-17653
> URL: https://issues.apache.org/jira/browse/HBASE-17653
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: stack
>Assignee: stack
>
> Follow-on from HBASE-17624. HBASE-17624 made it so one thread only has access
> to the rsgroup administrator. In tail of HBASE-17624 [~toffer] describes
> scenario under which we may end up in a deadlock (distributed). Let me
> repeat [~toffer] comment...
> {code}
> Both read/write access can't be single threaded. Consider the situation:
> 1. move_rsgroup_servers is called
> 2. while #1 is happening rsgroup region is in transition (rpc thread in #1
> holds monitor lock)
> 3. while #2 is happening meta is in transition.
> Balancer tries to figure out plan for meta region tries to get monitor lock
> but can't. rpc thread task won't release monitor lock since rsgroup region
> never gets assigned. rsgroup region never gets assigned because it can't
> update meta with new state.
> There's a good chance this can be reproduce just by moving both rsgroup and
> meta region onto the same RS and call move_rsgoup_servers on the same RS.
> A bunch different actors will query from group affiliation so we can't have
> writes block reads.
>
> In the code prior to this patch the getter methods that retrieve group
> information (getRSGroup, ofTable, OfServer, etc) don't require the monitor
> lock so the deadlock cycle is broken.
>
> The methods that does mutations and updates to zk and hbase:rsgroup are
> synchronized appropriately. Point me to where the incoherence is?
> {code}
> This issue is about testing/fixing/restoring rsgroup access. Will be back.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huzheng updated HBASE-17472:
Attachment: HBASE-17472.master.v6.patch
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.master.v6.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15328) Unvalidated Redirect in HMaster
[
https://issues.apache.org/jira/browse/HBASE-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869158#comment-15869158
]
Hadoop QA commented on HBASE-15328:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 46s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m 51s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 131m 46s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestZKAsyncRegistry |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852950/HBASE-15328.0.patch |
| JIRA Issue | HBASE-15328 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 570aff916351 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d2c083d |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5727/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5727/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5727/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5727/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Unvalidated Redirect in HMaster
> ---
>
> Key: HBASE-15328
> URL:
[jira] [Commented] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869265#comment-15869265
]
stack commented on HBASE-17652:
---
You going to resolve as dupe [~appy]?
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Labels: coprocessors, incompatible
> Attachments: HBASE-17652.master.001.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869266#comment-15869266
]
stack commented on HBASE-17652:
---
You going to resolve as dupe [~appy]?
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Labels: coprocessors, incompatible
> Attachments: HBASE-17652.master.001.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Status: Open (was: Patch Available)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16188-1.3.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Attachment: (was: HBASE-16188.patch)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16188-1.3.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Fix Version/s: (was: 2.0.0)
Status: Patch Available (was: Open)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 1.3.0
>
> Attachments: HBASE-16188-1.3.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869375#comment-15869375
]
Gopi Krishnan Nambiar commented on HBASE-16188:
---
Check HadoopQA
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 1.3.0
>
> Attachments: HBASE-16188-1.3.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17226) [C++] Filter and Query classes
[
https://issues.apache.org/jira/browse/HBASE-17226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-17226:
--
Attachment: hbase-17226_v1.patch
v1 implements all Comparators and all Filters.
> [C++] Filter and Query classes
> --
>
> Key: HBASE-17226
> URL: https://issues.apache.org/jira/browse/HBASE-17226
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17226_v0.patch, hbase-17226_v1.patch
>
>
> Implement {{Query}} and {{Filter}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Fix Version/s: 1.3.0
2.0.0
Status: Patch Available (was: In Progress)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16188-1.3.patch, HBASE-16188.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-15328) Unvalidated Redirect in HMaster
[
https://issues.apache.org/jira/browse/HBASE-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-15328:
Attachment: HBASE-15328.0.patch
-00
- check if the request hostname is local for our node
- provide override config in case deployment doesn't allow for above
- small refactor to cleanup how we get the needed info to the redirect servlet
> Unvalidated Redirect in HMaster
> ---
>
> Key: HBASE-15328
> URL: https://issues.apache.org/jira/browse/HBASE-15328
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-15328.0.patch
>
>
> See OWASP page on why we should clean it up someday:
> https://www.owasp.org/index.php/Unvalidated_Redirects_and_Forwards_Cheat_Sheet
> Here is where we do the redirect:
> {code}
> @Override
> public void doGet(HttpServletRequest request,
> HttpServletResponse response) throws ServletException, IOException {
> String redirectUrl = request.getScheme() + "://"
> + request.getServerName() + ":" + regionServerInfoPort
> + request.getRequestURI();
> response.sendRedirect(redirectUrl);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-15328) Unvalidated Redirect in HMaster
[
https://issues.apache.org/jira/browse/HBASE-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-15328:
Status: Patch Available (was: In Progress)
> Unvalidated Redirect in HMaster
> ---
>
> Key: HBASE-15328
> URL: https://issues.apache.org/jira/browse/HBASE-15328
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-15328.0.patch
>
>
> See OWASP page on why we should clean it up someday:
> https://www.owasp.org/index.php/Unvalidated_Redirects_and_Forwards_Cheat_Sheet
> Here is where we do the redirect:
> {code}
> @Override
> public void doGet(HttpServletRequest request,
> HttpServletResponse response) throws ServletException, IOException {
> String redirectUrl = request.getScheme() + "://"
> + request.getServerName() + ":" + regionServerInfoPort
> + request.getRequestURI();
> response.sendRedirect(redirectUrl);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17608) Add suspend support for RawScanResultConsumer
[ https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869077#comment-15869077 ] Duo Zhang commented on HBASE-17608: --- [~stack] What do you think of the new comments? Does it help? Thanks. > Add suspend support for RawScanResultConsumer > - > > Key: HBASE-17608 > URL: https://issues.apache.org/jira/browse/HBASE-17608 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17608.patch, HBASE-17608-v1.patch, > HBASE-17608-v2.patch, HBASE-17608-v3.patch > > > Now for the AsyncResultScanner, we can only close the scanner if we reach the > cache size limit and open a new scanner later. This will breaks the region > level consistency. > For example, you put 10 rows to the region, and open a scanner to scan it. > The scanner returns 5 rows at the first time and the cache is full, so it > closes the background scanner. And before you reopens the scanner to fetch > data, the remaining 5 rows has been deleted and a compaction makes them gone > for ever. Then after you reopen the scanner you can not see the remaining 5 > rows. > So here we should keep the scanner open at RS side to prevent the data below > the mvcc read point of this scanner being deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869345#comment-15869345
]
Hadoop QA commented on HBASE-16188:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s {color}
| {color:red} HBASE-16188 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852988/HBASE-16188-1.3.patch
|
| JIRA Issue | HBASE-16188 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5728/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16188-1.3.patch, HBASE-16188.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huzheng updated HBASE-17472:
Attachment: HBASE-17472.master.v6.patch
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.master.v6.patch, HBASE-17472.v1.patch,
> HBASE-17472.v2.patch, HBASE-17472.v3.patch, HBASE-17472.v4.patch,
> HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17649) REST API for scan should return 410 when table is disabled
[ https://issues.apache.org/jira/browse/HBASE-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869427#comment-15869427 ] Duo Zhang commented on HBASE-17649: --- +1. And seems you have already committed it? > REST API for scan should return 410 when table is disabled > -- > > Key: HBASE-17649 > URL: https://issues.apache.org/jira/browse/HBASE-17649 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17649.branch-1.3.v1.txt, 17649.v1.txt > > > Background: > When scanning using REST API, if the table is disabled in the middle of the > scan, code 410 (Gone) should be returned. > Currently in 1.3 and lower releases, ScannerResultGenerator#next() doesn't > handle TableNotEnabledException, leading to the exception being ignored and > code 204 is returned to caller. > This was first spotted by [~Apache9] when investigating HBASE-17603. > Patch for HBASE-17603 contains fix for this problem. > This issue fixes the bug for 1.3 and lower branches and adds test for > branch-1 and master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17312) [JDK8] Use default method for Observer Coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868999#comment-15868999 ] Sean Busbey commented on HBASE-17312: - ping [~apurtell] and [~enis]: any chance y'all could find someone in Phoenix willing to gut check the impact of this + HBASE-17652 on removing all hte Base* classes in favor of relying on interface default implementations? > [JDK8] Use default method for Observer Coprocessors > --- > > Key: HBASE-17312 > URL: https://issues.apache.org/jira/browse/HBASE-17312 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17312.master.001.patch, > HBASE-17312.master.001.patch > > > Use default method in MasterObserver, RegionObserver, RegionServerObserver > and WALObserver. And mark the BaseRegionObserver, > BaseMasterAndRegionObserver, BaseRegionServerObserver and BaseWALObserver. > User can implement the interface directly and will not break compatibility > when add new default methods. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868997#comment-15868997
]
Sean Busbey commented on HBASE-17472:
-
+1 to making the flag compatible on branch-1 and default to the new behavior in
master. Presumably it will still need to be exposed in the shell?
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Attachment: HBASE-16188-1.3.patch
HBASE-16188.patch
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-16188-1.3.patch, HBASE-16188.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17595) Add partial result support for small/limited scan
[ https://issues.apache.org/jira/browse/HBASE-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867585#comment-15867585 ] Duo Zhang commented on HBASE-17595: --- Yeah there will be some conflicts. But I think you can start your work based on this branch? https://github.com/apache9/hbase/tree/HBASE-17595 > Add partial result support for small/limited scan > - > > Key: HBASE-17595 > URL: https://issues.apache.org/jira/browse/HBASE-17595 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0, 1.4.0 >Reporter: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > > The partial result support is marked as a 'TODO' when implementing > HBASE-17045. And when implementing HBASE-17508, we found that if we make > small scan share the same logic with general scan, the scan request other > than open scanner will not have the small flag so the server may return > partial result to the client and cause some strange behavior. It is solved by > modifying the logic at server side, but this means the 1.4.x client is not > safe to contact with earlier 1.x server. So we'd better address the problem > at client side. Marked as blocker as this issue should be finished before any > 2.x and 1.4.x releases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17595) Add partial result support for small/limited scan
[ https://issues.apache.org/jira/browse/HBASE-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867579#comment-15867579 ] Phil Yang commented on HBASE-17595: --- We can add an UT to assert this feature? And HBASE-15484 should wait this work done? > Add partial result support for small/limited scan > - > > Key: HBASE-17595 > URL: https://issues.apache.org/jira/browse/HBASE-17595 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0, 1.4.0 >Reporter: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > > The partial result support is marked as a 'TODO' when implementing > HBASE-17045. And when implementing HBASE-17508, we found that if we make > small scan share the same logic with general scan, the scan request other > than open scanner will not have the small flag so the server may return > partial result to the client and cause some strange behavior. It is solved by > modifying the logic at server side, but this means the 1.4.x client is not > safe to contact with earlier 1.x server. So we'd better address the problem > at client side. Marked as blocker as this issue should be finished before any > 2.x and 1.4.x releases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17374) ZKPermissionWatcher crashed when grant after region close
[
https://issues.apache.org/jira/browse/HBASE-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867746#comment-15867746
]
Bhupendra Kumar Jain commented on HBASE-17374:
--
As I went thru the JIRA and patch
Jira desc says "The instance of TableAuthManager should not be close
unless regionserver is closing."
1) If Accesscontroller coprocessor is configured for regionserver, then
there will be always one TableAuthManager instance. So the issue should not
happen
2) If Accesscontroller coprocessor is not configured for regionserver, then
the issue will happen. This patch only handles the exception case which occurs
during zk event processing and executor is already shutdown.
But I think , as zk watcher holds the ZKPermissionWatcher listener instance, so
ZKPermissionWatcher will keep getting the events (even though already closed)
and it will lead to memory leak of TableAuthManager instance also.
So to solve this completely, ZKPermissionWatcher should unregister itself from
the watcher in close() method. with that it will not get any events further and
it will avoid memory leak.
Please correct if any other understanding.
> ZKPermissionWatcher crashed when grant after region close
> -
>
> Key: HBASE-17374
> URL: https://issues.apache.org/jira/browse/HBASE-17374
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.15
>Reporter: Liu Junhong
>Assignee: Liu Junhong
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 0001-fix-for-HBASE-17374-20161228.patch,
> 0001-fix-for-HBASE-17374.patch
>
>
> It was occurred many time that I granted some permission, but few of some
> regionservers was not token effect and must be restart . When I look up logs,
> I found that :
> 2016-12-08 15:00:26,238 DEBUG
> [RS_CLOSE_REGION-dx-data-hbase-watcher05:60020-0] handler.CloseRegionHandler
> (CloseRegionHandler.java:process(128)) - Processing close of
> data-probe-test,,1481180420784.5f06cb6447343b602e05996bfd87ce14.
> {color:red} 2016-12-08 15:00:26,242 DEBUG
> [RS_CLOSE_REGION-dx-data-hbase-watcher05:60020-0] regionserver.HRegion
> (HRegion.java:doClose(1163)) - Closing
> data-probe-test,,1481180420784.5f06cb6447343b602e05996bfd87ce14.: disabling
> compactions & flushes {color}
> 2016-12-08 15:00:26,242 DEBUG
> [RS_CLOSE_REGION-dx-data-hbase-watcher05:60020-0] regionserver.HRegion
> (HRegion.java:doClose(1190)) - Updates disabled for region
> data-probe-test,,1481180420784.5f06cb6447343b602e05996bfd87ce14.
> 2016-12-08 15:00:26,242 INFO
> [RS_CLOSE_REGION-dx-data-hbase-watcher05:60020-0] regionserver.HRegion
> (HRegion.java:internalFlushcache(1753)) - Started memstore flush for
> data-probe-test,,1481180420784.5f06cb6447343b602e05996bfd87ce14., current
> region memstore size 160
> 2016-12-08 15:00:26,284 INFO
> [RS_CLOSE_REGION-dx-data-hbase-watcher05:60020-0]
> regionserver.DefaultStoreFlusher (DefaultStoreFlusher.java:flushSnapshot(95))
> - Flushed, sequenceid=6, memsize=160, hasBloomFilter=true, into tmp file
> hdfs://dx-data-hbase-watcher/hbase/data/default/data-probe-test/5f06cb6447343b602e05996bfd87ce14/.tmp/8d734ce3d93e40628d8f82111e754cb3
> 2016-12-08 15:00:26,303 DEBUG
> [RS_CLOSE_REGION-dx-data-hbase-watcher05:60020-0]
> regionserver.HRegionFileSystem (HRegionFileSystem.java:commitStoreFile(370))
> - Committing store file
> hdfs://dx-data-hbase-watcher/hbase/data/default/data-probe-test/5f06cb6447343b602e05996bfd87ce14/.tmp/8d734ce3d93e40628d8f82111e754cb3
> as
> hdfs://dx-data-hbase-watcher/hbase/data/default/data-probe-test/5f06cb6447343b602e05996bfd87ce14/cf2/8d734ce3d93e40628d8f82111e754cb3
> 2016-12-08 15:00:26,318 INFO
> [RS_CLOSE_REGION-dx-data-hbase-watcher05:60020-0] regionserver.HStore
> (HStore.java:commitFile(877)) - Added
> hdfs://dx-data-hbase-watcher/hbase/data/default/data-probe-test/5f06cb6447343b602e05996bfd87ce14/cf2/8d734ce3d93e40628d8f82111e754cb3,
> entries=1, sequenceid=6, filesize=985
> 2016-12-08 15:00:26,319 INFO
> [RS_CLOSE_REGION-dx-data-hbase-watcher05:60020-0] regionserver.HRegion
> (HRegion.java:internalFlushcache(1920)) - Finished memstore flush of
> ~160/160, currentsize=0/0 for region
> data-probe-test,,1481180420784.5f06cb6447343b602e05996bfd87ce14. in 77ms,
> sequenceid=6, compaction requested=false
> 2016-12-08 15:00:26,323 INFO
> [StoreCloserThread-data-probe-test,,1481180420784.5f06cb6447343b602e05996bfd87ce14.-1]
> regionserver.HStore (HStore.java:close(774)) - Closed cf1
> 2016-12-08 15:00:26,325 INFO
> [StoreCloserThread-data-probe-test,,1481180420784.5f06cb6447343b602e05996bfd87ce14.-1]
> regionserver.HStore (HStore.java:close(774)) - Closed cf2
> 2016-12-08 15:00:26,326 DEBUG
>
[jira] [Commented] (HBASE-17608) Add suspend support for RawScanResultConsumer
[
https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867712#comment-15867712
]
Hadoop QA commented on HBASE-17608:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 41s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
31s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 37s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 18s
{color} | {color:red} hbase-client generated 1 new + 13 unchanged - 0 fixed =
14 total (was 13) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 1s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 23s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 113m 10s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 35s
{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
57s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 180m 34s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852770/HBASE-17608-v3.patch |
| JIRA Issue | HBASE-17608 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 43db90d96b22 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 33e9a8c |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| javadoc |
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867762#comment-15867762
]
Abhishek Singh Chouhan commented on HBASE-17069:
[~Apache9] Yep got confused a bit there :) retrowIOException might be better.
Got to the bottom of things. Here's what was happening.
- Region A getting split into B & C. The first request with daughter info is a
multi (with hregion info) while the second one is a put.
In Hregion.processRowsWithLocks we have
{noformat}
// 6. Append no sync
if (!walEdit.isEmpty()) {
// we use HLogKey here instead of WALKey directly to support legacy
coprocessors.
walKey = new HLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
this.htableDescriptor.getTableName(), WALKey.NO_SEQUENCE_ID, now,
processor.getClusterIds(), nonceGroup, nonce, mvcc);
txid = this.wal.append(this.htableDescriptor, this.getRegionInfo(),
walKey, walEdit, false);
}
{noformat}
Since we pass false for inMemstore in append, we mess up the seq id accounting.
In SequenceIdAccounting.update() we pass false for the multirequest (lets say
sequence id here was 1) so lowestunflusedsequenceid is not updated.
Now for the second put that goes through doMiniBatchMutation we pass true
correctly during append(Seq id 2). lowestUnflushedSequenceIds is set to 2 for
the metafamily. The rs sends the report using HRegion.setCompleteSequenceId
where it sets the lastflushedsequence id for this store as 1 (however we still
haven't actually flushed).
- At this point the RS dies
- During the split we receive lastflushedseqid for this store as 1 and filter
out the cells belonging to the multi which had the hregioninfo. The
regionserver thats opening the region now will replay the edits correctly but
we've lost data belonging to the multi and hence the client fails with
"HRegionInfo was null"
However this case is not particular to split or meta but the case where a
region is just opened and we do a number of multi followed by a put, in case
the RS dies before we flush we lose data belonging to the multi. Fix is simply
a line change :)
[~apurtell] [~lhofhansl]
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Critical
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
>
[jira] [Comment Edited] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867762#comment-15867762
]
Abhishek Singh Chouhan edited comment on HBASE-17069 at 2/15/17 12:49 PM:
--
[~Apache9] Yep got confused a bit there :) rethrowIOException might be better.
Got to the bottom of things. Here's what was happening.
- Region A getting split into B & C. The first request with daughter info is a
multi (with hregion info) while the second one is a put.
In Hregion.processRowsWithLocks we have
{noformat}
// 6. Append no sync
if (!walEdit.isEmpty()) {
// we use HLogKey here instead of WALKey directly to support legacy
coprocessors.
walKey = new HLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
this.htableDescriptor.getTableName(), WALKey.NO_SEQUENCE_ID, now,
processor.getClusterIds(), nonceGroup, nonce, mvcc);
txid = this.wal.append(this.htableDescriptor, this.getRegionInfo(),
walKey, walEdit, false);
}
{noformat}
Since we pass false for inMemstore in append, we mess up the seq id accounting.
In SequenceIdAccounting.update() we pass false for the multirequest (lets say
sequence id here was 1) so lowestunflusedsequenceid is not updated.
Now for the second put that goes through doMiniBatchMutation we pass true
correctly during append(Seq id 2). lowestUnflushedSequenceIds is set to 2 for
the metafamily. The rs sends the report using HRegion.setCompleteSequenceId
where it sets the lastflushedsequence id for this store as 1 (however we still
haven't actually flushed).
- At this point the RS dies
- During the split we receive lastflushedseqid for this store as 1 and filter
out the cells belonging to the multi which had the hregioninfo. The
regionserver thats opening the region now will replay the edits correctly but
we've lost data belonging to the multi and hence the client fails with
"HRegionInfo was null"
However this case is not particular to split or meta but the case where a
region is just opened and we do a number of multi followed by a put, in case
the RS dies before we flush we lose data belonging to the multi. Fix is simply
a line change :)
[~apurtell] [~lhofhansl]
was (Author: abhishek.chouhan):
[~Apache9] Yep got confused a bit there :) retrowIOException might be better.
Got to the bottom of things. Here's what was happening.
- Region A getting split into B & C. The first request with daughter info is a
multi (with hregion info) while the second one is a put.
In Hregion.processRowsWithLocks we have
{noformat}
// 6. Append no sync
if (!walEdit.isEmpty()) {
// we use HLogKey here instead of WALKey directly to support legacy
coprocessors.
walKey = new HLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
this.htableDescriptor.getTableName(), WALKey.NO_SEQUENCE_ID, now,
processor.getClusterIds(), nonceGroup, nonce, mvcc);
txid = this.wal.append(this.htableDescriptor, this.getRegionInfo(),
walKey, walEdit, false);
}
{noformat}
Since we pass false for inMemstore in append, we mess up the seq id accounting.
In SequenceIdAccounting.update() we pass false for the multirequest (lets say
sequence id here was 1) so lowestunflusedsequenceid is not updated.
Now for the second put that goes through doMiniBatchMutation we pass true
correctly during append(Seq id 2). lowestUnflushedSequenceIds is set to 2 for
the metafamily. The rs sends the report using HRegion.setCompleteSequenceId
where it sets the lastflushedsequence id for this store as 1 (however we still
haven't actually flushed).
- At this point the RS dies
- During the split we receive lastflushedseqid for this store as 1 and filter
out the cells belonging to the multi which had the hregioninfo. The
regionserver thats opening the region now will replay the edits correctly but
we've lost data belonging to the multi and hence the client fails with
"HRegionInfo was null"
However this case is not particular to split or meta but the case where a
region is just opened and we do a number of multi followed by a put, in case
the RS dies before we flush we lose data belonging to the multi. Fix is simply
a line change :)
[~apurtell] [~lhofhansl]
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Critical
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
>
[jira] [Updated] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Singh Chouhan updated HBASE-17069:
---
Status: Patch Available (was: Open)
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Critical
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,717 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for tiny: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960,
[jira] [Updated] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Singh Chouhan updated HBASE-17069:
---
Attachment: HBASE-17069.branch-1.3.001.patch
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Critical
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,717 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for tiny: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440,
[jira] [Commented] (HBASE-17210) Set timeout on trying rowlock according to client's RPC timeout
[
https://issues.apache.org/jira/browse/HBASE-17210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867669#comment-15867669
]
Hadoop QA commented on HBASE-17210:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 49s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m 51s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 137m 1s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
| | hadoop.hbase.regionserver.TestHRegion |
| Timed out junit tests | org.apache.hadoop.hbase.client.TestFromClientSide3 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852771/HBASE-17210.v02.patch
|
| JIRA Issue | HBASE-17210 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 177306f5950b 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 33e9a8c |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5717/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5717/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5717/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5717/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Set timeout
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867687#comment-15867687
]
Hadoop QA commented on HBASE-17472:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 37s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 21s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 29s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 113m 23s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 181m 38s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852764/HBASE-17472.v5.patch |
| JIRA Issue | HBASE-17472 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile cc hbaseprotoc |
| uname | Linux 1565ca18427c 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 33e9a8c |
| Default
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867469#comment-15867469
]
huzheng commented on HBASE-17472:
-
Upload patch v5.
1. set default value to false for new fields (mergeExistingPermissions);
2. using LOG.error(message, t) to print grant(2)/revoke exception , instead of
e.printStraceTrace().
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17210) Set timeout on trying rowlock according to client's RPC timeout
[ https://issues.apache.org/jira/browse/HBASE-17210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-17210: -- Attachment: HBASE-17210.v02.patch > Set timeout on trying rowlock according to client's RPC timeout > --- > > Key: HBASE-17210 > URL: https://issues.apache.org/jira/browse/HBASE-17210 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-17120.v1.patch, HBASE-17210.v02.patch > > > Now when we want to get a row lock, the timeout is fixed and default is 30s. > But the requests from client have different RPC timeout setting. We can use > the client's deadline to set timeout on tryLock. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17210) Set timeout on trying rowlock according to client's RPC timeout
[ https://issues.apache.org/jira/browse/HBASE-17210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867489#comment-15867489 ] Phil Yang commented on HBASE-17210: --- Upload a rebased patch, let's see QA first > Set timeout on trying rowlock according to client's RPC timeout > --- > > Key: HBASE-17210 > URL: https://issues.apache.org/jira/browse/HBASE-17210 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-17120.v1.patch, HBASE-17210.v02.patch > > > Now when we want to get a row lock, the timeout is fixed and default is 30s. > But the requests from client have different RPC timeout setting. We can use > the client's deadline to set timeout on tryLock. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867533#comment-15867533
]
Hadoop QA commented on HBASE-17561:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 45s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 56s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 125m 2s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852756/HBASE-17561.0.patch |
| JIRA Issue | HBASE-17561 |
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux 7e7b98e79c97 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 33e9a8c |
| Default Java | 1.8.0_121 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5715/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5715/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5715/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huzheng updated HBASE-17472:
Attachment: HBASE-17472.v5.patch
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17608) Add suspend support for RawScanResultConsumer
[ https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17608: -- Attachment: HBASE-17608-v3.patch > Add suspend support for RawScanResultConsumer > - > > Key: HBASE-17608 > URL: https://issues.apache.org/jira/browse/HBASE-17608 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17608.patch, HBASE-17608-v1.patch, > HBASE-17608-v2.patch, HBASE-17608-v3.patch > > > Now for the AsyncResultScanner, we can only close the scanner if we reach the > cache size limit and open a new scanner later. This will breaks the region > level consistency. > For example, you put 10 rows to the region, and open a scanner to scan it. > The scanner returns 5 rows at the first time and the cache is full, so it > closes the background scanner. And before you reopens the scanner to fetch > data, the remaining 5 rows has been deleted and a compaction makes them gone > for ever. Then after you reopen the scanner you can not see the remaining 5 > rows. > So here we should keep the scanner open at RS side to prevent the data below > the mvcc read point of this scanner being deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867469#comment-15867469
]
huzheng edited comment on HBASE-17472 at 2/15/17 8:43 AM:
--
Upload patch v5.
1. set default value to false for new protobuf field
(mergeExistingPermissions);
2. using LOG.error(message, t) to print grant(2)/revoke exception , instead of
e.printStraceTrace().
was (Author: openinx):
Upload patch v5.
1. set default value to false for new fields (mergeExistingPermissions);
2. using LOG.error(message, t) to print grant(2)/revoke exception , instead of
e.printStraceTrace().
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17646) Implement Async getRegion method
[ https://issues.apache.org/jira/browse/HBASE-17646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867801#comment-15867801 ] huzheng commented on HBASE-17646: - Upload patch v1. > Implement Async getRegion method > > > Key: HBASE-17646 > URL: https://issues.apache.org/jira/browse/HBASE-17646 > Project: HBase > Issue Type: Sub-task > Components: asyncclient >Affects Versions: 2.0.0 >Reporter: huzheng >Assignee: huzheng > Labels: asynchronous > Fix For: 2.0.0 > > Attachments: HBASE-17646.v1.patch > > > There are some async admin APIs which depends on async getRegion method. > Such as : > 1. closeRegion. > 2. flushRegion. > 3. compactRegion. > 4. mergeRegion. > 5. splitRegion. > and so on . > So, implement async getRegion method first. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14043) Syntax error in Section 26.2 of Reference Guide
[ https://issues.apache.org/jira/browse/HBASE-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867859#comment-15867859 ] Jan Hentschel commented on HBASE-14043: --- Is this still an issue? It seems that the link in the reference guide was simplified (http://hbase.apache.org/book.html#_data_model_operations). > Syntax error in Section 26.2 of Reference Guide > --- > > Key: HBASE-14043 > URL: https://issues.apache.org/jira/browse/HBASE-14043 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Joe McCarthy >Priority: Trivial > > The following string does not appear rendered as the preceding string > describing Table.put: > "link:http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Table.html#batch(java.util.List, > java.lang.Object[])[Table.batch] (non-writeBuffer)" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17651) Implement grant2 ruby shell command.
huzheng created HBASE-17651: --- Summary: Implement grant2 ruby shell command. Key: HBASE-17651 URL: https://issues.apache.org/jira/browse/HBASE-17651 Project: HBase Issue Type: Sub-task Components: Admin Reporter: huzheng Assignee: huzheng Fix For: 2.0.0 As HBASE-17472 said, we'll implement a new shell command grant2 to add one or more action(s) for user, and deprecated previous grant shell command. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867782#comment-15867782
]
Ted Yu commented on HBASE-17069:
Lgtm
Please add license header to the new test.
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Critical
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,717 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for tiny: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440,
[jira] [Updated] (HBASE-17646) Implement Async getRegion method
[ https://issues.apache.org/jira/browse/HBASE-17646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huzheng updated HBASE-17646: Attachment: HBASE-17646.v1.patch > Implement Async getRegion method > > > Key: HBASE-17646 > URL: https://issues.apache.org/jira/browse/HBASE-17646 > Project: HBase > Issue Type: Sub-task > Components: asyncclient >Affects Versions: 2.0.0 >Reporter: huzheng >Assignee: huzheng > Labels: asynchronous > Fix For: 2.0.0 > > Attachments: HBASE-17646.v1.patch > > > There are some async admin APIs which depends on async getRegion method. > Such as : > 1. closeRegion. > 2. flushRegion. > 3. compactRegion. > 4. mergeRegion. > 5. splitRegion. > and so on . > So, implement async getRegion method first. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867813#comment-15867813
]
Duo Zhang commented on HBASE-17069:
---
Nice catch! This is ridiculous why here we set inMemstore to false when calling
wal.append...
It is introduced by HBASE-14465, which is the backport of HBASE-12751. But on
master it is fixed by HBASE-15158 where we did some refactoring and change to
use doWALAppend instead of call wal.append directly, and in doWALAppend,
inMemstore is set to true...
HBASE-14465 only effects branch-1, branch-1.3 and branch-1.2. So that's why
branch-1.1 can pass ITBLL and all newer 1.x releases can not.
Well done!
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960,
[jira] [Updated] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17069:
--
Affects Version/s: 1.4.0
1.3.0
Priority: Blocker (was: Critical)
Fix Version/s: 1.2.5
1.3.1
1.4.0
Component/s: wal
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,717 INFO
>
[jira] [Comment Edited] (HBASE-17646) Implement Async getRegion method
[ https://issues.apache.org/jira/browse/HBASE-17646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867801#comment-15867801 ] huzheng edited comment on HBASE-17646 at 2/15/17 1:19 PM: -- Upload patch v1. And this patch depends on HBASE-17619, so I will put it to ReviewBoard after HBASE-17619 committed to hbase-git. was (Author: openinx): Upload patch v1. > Implement Async getRegion method > > > Key: HBASE-17646 > URL: https://issues.apache.org/jira/browse/HBASE-17646 > Project: HBase > Issue Type: Sub-task > Components: asyncclient >Affects Versions: 2.0.0 >Reporter: huzheng >Assignee: huzheng > Labels: asynchronous > Fix For: 2.0.0 > > Attachments: HBASE-17646.v1.patch > > > There are some async admin APIs which depends on async getRegion method. > Such as : > 1. closeRegion. > 2. flushRegion. > 3. compactRegion. > 4. mergeRegion. > 5. splitRegion. > and so on . > So, implement async getRegion method first. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17646) Implement Async getRegion method
[
https://issues.apache.org/jira/browse/HBASE-17646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867816#comment-15867816
]
Hadoop QA commented on HBASE-17646:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HBASE-17646 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852812/HBASE-17646.v1.patch |
| JIRA Issue | HBASE-17646 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5720/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Implement Async getRegion method
>
>
> Key: HBASE-17646
> URL: https://issues.apache.org/jira/browse/HBASE-17646
> Project: HBase
> Issue Type: Sub-task
> Components: asyncclient
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Labels: asynchronous
> Fix For: 2.0.0
>
> Attachments: HBASE-17646.v1.patch
>
>
> There are some async admin APIs which depends on async getRegion method.
> Such as :
> 1. closeRegion.
> 2. flushRegion.
> 3. compactRegion.
> 4. mergeRegion.
> 5. splitRegion.
> and so on .
> So, implement async getRegion method first.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867837#comment-15867837
]
Duo Zhang commented on HBASE-17069:
---
Minor nits.
{code}
import static org.junit.Assert.*;
{code}
Avoid star import.
{code}
@Test(timeout = 6)
public void MutateRowsAndCheckPostKill() throws IOException,
InterruptedException {
{code}
I think we already have a CatogaryBasedTimeout which assign timeout
automatically based on the annotation for the test classes(LargeTests,
MediumTests, etc), so we do not need to specify a timeout here again?
And I think we could also commit the UT to master to prevent we do the stupid
thing again in the future?
Thanks.
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440,
[jira] [Updated] (HBASE-17646) Implement Async getRegion method
[ https://issues.apache.org/jira/browse/HBASE-17646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huzheng updated HBASE-17646: Status: Patch Available (was: Open) > Implement Async getRegion method > > > Key: HBASE-17646 > URL: https://issues.apache.org/jira/browse/HBASE-17646 > Project: HBase > Issue Type: Sub-task > Components: asyncclient >Affects Versions: 2.0.0 >Reporter: huzheng >Assignee: huzheng > Labels: asynchronous > Fix For: 2.0.0 > > Attachments: HBASE-17646.v1.patch > > > There are some async admin APIs which depends on async getRegion method. > Such as : > 1. closeRegion. > 2. flushRegion. > 3. compactRegion. > 4. mergeRegion. > 5. splitRegion. > and so on . > So, implement async getRegion method first. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867850#comment-15867850
]
Duo Zhang commented on HBASE-17472:
---
Will commit tomorrow if no objections. And [~openinx] please open the issue for
adding shell command for grant2. Thanks.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-14044) #keyvalue / #keyvalue.example anchor(s) in Reference Guide
[ https://issues.apache.org/jira/browse/HBASE-14044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867857#comment-15867857 ] Jan Hentschel commented on HBASE-14044: --- Is this one still valid? It seems that there are currently no references to the mentioned anchor, which seems to work (http://hbase.apache.org/book.html#keyvalue.example). > #keyvalue / #keyvalue.example anchor(s) in Reference Guide > -- > > Key: HBASE-14044 > URL: https://issues.apache.org/jira/browse/HBASE-14044 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Joe McCarthy >Priority: Trivial > > There are several references to a #keyvalue anchor in the Reference Guide, > but I do not see any definition of that anchor. There is a #keyvalue.example > definition. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867884#comment-15867884
]
huzheng commented on HBASE-17472:
-
Hi [~Apache9], I've created HBASE-17651 for it.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868939#comment-15868939
]
Duo Zhang commented on HBASE-17472:
---
Oh sorry, in the patch here we already have a mergeExistingPermissions flag...
So let's just remove the grant2 method.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868937#comment-15868937
]
Duo Zhang commented on HBASE-17472:
---
So let's introduce a flag called overwrite? in branch-1 it is default to true,
and in master we change it to false and mark this issue as an incompatible
change?
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17627) Active workers metric for thrift
[ https://issues.apache.org/jira/browse/HBASE-17627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868983#comment-15868983 ] Hudson commented on HBASE-17627: FAILURE: Integrated in Jenkins build HBase-1.4 #631 (See [https://builds.apache.org/job/HBase-1.4/631/]) HBASE-17627 Active workers metric for thrift (Ashu Pachauri) (garyh: rev 2eb810d0f7dd79e30176e9c8fbd1a69905c9bb63) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftMetrics.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSource.java * (add) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/THBaseThreadPoolExecutor.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/TBoundedThreadPoolServer.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java > Active workers metric for thrift > > > Key: HBASE-17627 > URL: https://issues.apache.org/jira/browse/HBASE-17627 > Project: HBase > Issue Type: Improvement > Components: Thrift >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: HBASE-17627.branch-1.001.patch, > HBASE-17627.master.001.patch > > > It would be good to have a metric for number of active handlers on thrift > servers, which gives a good indication of business of a thrift server. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868996#comment-15868996
]
huzheng commented on HBASE-17472:
-
Thanks [~busbey], [~enis], [~Apache9], new patch with a flag will come soon.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17649) REST API for scan should return 410 when table is disabled
[ https://issues.apache.org/jira/browse/HBASE-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868066#comment-15868066 ] Ted Yu commented on HBASE-17649: [~Apache9]: This fixes the problem of scanning thru REST API returning 204. > REST API for scan should return 410 when table is disabled > -- > > Key: HBASE-17649 > URL: https://issues.apache.org/jira/browse/HBASE-17649 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17649.branch-1.3.v1.txt, 17649.v1.txt > > > Background: > When scanning using REST API, if the table is disabled in the middle of the > scan, code 410 (Gone) should be returned. > Currently in 1.3 and lower releases, ScannerResultGenerator#next() doesn't > handle TableNotEnabledException, leading to the exception being ignored and > code 204 is returned to caller. > This was first spotted by [~Apache9] when investigating HBASE-17603. > Patch for HBASE-17603 contains fix for this problem. > This issue fixes the bug for 1.3 and lower branches and adds test for > branch-1 and master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868115#comment-15868115
]
Sean Busbey commented on HBASE-17472:
-
I'd even go so far as having the flag indicate the user wants the old behavior
so that the natural choice in HBase 2 can be what I think most folks expect
when you use grant, but I don't feel strongly about that part.
Major versions are made for this sort of change, so long as we flag it as
incompatible and write a release note it's okay to correct things in a way that
downstream folks have to pay attention to.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17561) table status page should escape values that may contain arbitrary characters.
[
https://issues.apache.org/jira/browse/HBASE-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868128#comment-15868128
]
Sean Busbey commented on HBASE-17561:
-
looks like yetus failed to parse the failures:
{code}
Results :
Tests in error:
TestRegionReplicaFailover.testPrimaryRegionKill:200->verifyNumericRowsWithTimeout:227->Object.wait:460->Object.wait:-2
? TestTimedOut
Flaked tests:
org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet(org.apache.hadoop.hbase.client.TestReplicasClient)
Run 1: TestReplicasClient.testCancelOfMultiGet:588 null
Run 2: PASS
org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenSlaveClusterDoesntHaveTable(org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters)
Run 1:
TestReplicationAdminWithClusters.testEnableReplicationWhenSlaveClusterDoesntHaveTable:83
? IllegalArgument
Run 2: PASS
{code}
I don't think these are related.
> table status page should escape values that may contain arbitrary characters.
> -
>
> Key: HBASE-17561
> URL: https://issues.apache.org/jira/browse/HBASE-17561
> Project: HBase
> Issue Type: Sub-task
> Components: master, UI
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Attachments: HBASE-17561.0.patch
>
>
> We write out table names to an html document without escaping html entities
> e.g. in this case it even comes directly from the request
> {code}
>
> <% if ( !readOnly && action != null ) { %>
> HBase Master: <%= master.getServerName() %>
> <% } else { %>
> Table: <%= fqtn %>
> <% } %>
> {code}
> in
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-17623:
--
Attachment: after(snappy_hfilesize=755MB).png
before(snappy_hfilesize=755MB).png
The attachment is the GC activity. It illustrates that the lower number of GC
is due to memory reusing.
The size of memory allocation is presented below.
||before||after||
|35GB|22GB|
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: after(snappy_hfilesize=755MB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.v0.patch,
> HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (HBASE-17493) TestAsyncTableGetMultiThreadedWithEagerCompaction intermittently fails due to NotServingRegionException
[
https://issues.apache.org/jira/browse/HBASE-17493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-17493.
Resolution: Cannot Reproduce
> TestAsyncTableGetMultiThreadedWithEagerCompaction intermittently fails due to
> NotServingRegionException
> ---
>
> Key: HBASE-17493
> URL: https://issues.apache.org/jira/browse/HBASE-17493
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Priority: Minor
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/5322/artifact/patchprocess/patch-unit-hbase-server.txt
> :
> {code}
> test(org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreadedWithEagerCompaction)
> Time elapsed: 90.51 sec <<< ERROR!
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region
> async,222,1484797745093.f9b23b061b6cc56bc801f6d962fc5313. is not online on
> 089e51c5fdc1,37640,1484797689531
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3161)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1239)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.splitRegion(RSRpcServices.java:2044)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:25093)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:1140)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:257)
> at sun.reflect.GeneratedConstructorAccessor34.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:95)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:85)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:357)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:334)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.split(ProtobufUtil.java:1948)
> at org.apache.hadoop.hbase.client.HBaseAdmin.split(HBaseAdmin.java:1685)
> at org.apache.hadoop.hbase.client.HBaseAdmin.split(HBaseAdmin.java:1646)
> at
> org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded.test(TestAsyncTableGetMultiThreaded.java:136)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at
[jira] [Commented] (HBASE-17649) REST API for scan should return 410 when table is disabled
[ https://issues.apache.org/jira/browse/HBASE-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868480#comment-15868480 ] Hudson commented on HBASE-17649: FAILURE: Integrated in Jenkins build HBase-1.4 #630 (See [https://builds.apache.org/job/HBase-1.4/630/]) HBASE-17649 REST API for scan should return 410 when table is disabled (tedyu: rev e3d70db4af2e02288bf25327dd37bc2b628ee31d) * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestScannerResource.java > REST API for scan should return 410 when table is disabled > -- > > Key: HBASE-17649 > URL: https://issues.apache.org/jira/browse/HBASE-17649 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17649.branch-1.3.v1.txt, 17649.v1.txt > > > Background: > When scanning using REST API, if the table is disabled in the middle of the > scan, code 410 (Gone) should be returned. > Currently in 1.3 and lower releases, ScannerResultGenerator#next() doesn't > handle TableNotEnabledException, leading to the exception being ignored and > code 204 is returned to caller. > This was first spotted by [~Apache9] when investigating HBASE-17603. > Patch for HBASE-17603 contains fix for this problem. > This issue fixes the bug for 1.3 and lower branches and adds test for > branch-1 and master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17649) REST API for scan should return 410 when table is disabled
[ https://issues.apache.org/jira/browse/HBASE-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868490#comment-15868490 ] Hudson commented on HBASE-17649: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #105 (See [https://builds.apache.org/job/HBase-1.3-JDK7/105/]) HBASE-17649 REST API for scan should return 410 when table is disabled (tedyu: rev a3d021aec5583456ee2bf306f796ba89fbe95f71) * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerResultGenerator.java * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestScannerResource.java > REST API for scan should return 410 when table is disabled > -- > > Key: HBASE-17649 > URL: https://issues.apache.org/jira/browse/HBASE-17649 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17649.branch-1.3.v1.txt, 17649.v1.txt > > > Background: > When scanning using REST API, if the table is disabled in the middle of the > scan, code 410 (Gone) should be returned. > Currently in 1.3 and lower releases, ScannerResultGenerator#next() doesn't > handle TableNotEnabledException, leading to the exception being ignored and > code 204 is returned to caller. > This was first spotted by [~Apache9] when investigating HBASE-17603. > Patch for HBASE-17603 contains fix for this problem. > This issue fixes the bug for 1.3 and lower branches and adds test for > branch-1 and master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17649) REST API for scan should return 410 when table is disabled
[ https://issues.apache.org/jira/browse/HBASE-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868498#comment-15868498 ] Hudson commented on HBASE-17649: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2510 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2510/]) HBASE-17649 REST API for scan should return 410 when table is disabled (tedyu: rev de23d306ebd1ab4135fcfca6a682f8c167911d05) * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestScannerResource.java > REST API for scan should return 410 when table is disabled > -- > > Key: HBASE-17649 > URL: https://issues.apache.org/jira/browse/HBASE-17649 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17649.branch-1.3.v1.txt, 17649.v1.txt > > > Background: > When scanning using REST API, if the table is disabled in the middle of the > scan, code 410 (Gone) should be returned. > Currently in 1.3 and lower releases, ScannerResultGenerator#next() doesn't > handle TableNotEnabledException, leading to the exception being ignored and > code 204 is returned to caller. > This was first spotted by [~Apache9] when investigating HBASE-17603. > Patch for HBASE-17603 contains fix for this problem. > This issue fixes the bug for 1.3 and lower branches and adds test for > branch-1 and master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Singh Chouhan updated HBASE-17069:
---
Attachment: HBASE-17069.branch-1.3.002.patch
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> logs.tar.gz, parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,717 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for tiny:
[jira] [Comment Edited] (HBASE-17624) Address late review of HBASE-6721, rsgroups feature
[
https://issues.apache.org/jira/browse/HBASE-17624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868334#comment-15868334
]
Francis Liu edited comment on HBASE-17624 at 2/15/17 6:58 PM:
--
{quote}
Tell me more Francis Liu about deadlock. How? The concurrency issue addressed
was just making it so all modification to rsgroup state was single-threaded;
{quote}
Both read/write access can't be single threaded. Consider the situation:
1. move_rsgroup_servers is called
2. while #1 is happening rsgroup region is in transition (rpc thread in #1
holds monitor lock)
3. while #2 is happening meta is in transition.
Balancer tries to figure out plan for meta region tries to get monitor lock but
can't. rpc thread task won't release monitor lock since rsgroup region never
gets assigned. rsgroup region never gets assigned because it can't update meta
with new state.
There's a good chance this can be reproduce just by moving both rsgroup and
meta region onto the same RS and call move_rsgoup_servers on the same RS.
A bunch different actors will query from group affiliation so we can't have
writes block reads.
{quote}
previous it seemed loosey-goosey – least I couldn't figure out the regime.
{quote}
Looks like the first patch already had these changes so I can't really respond
to what may or may not be loosey goosey. I'll try to review the changes today
and see what other issues could be there.
Regarding Guava classes, sounds like this is a new policy. The HostAndPort
change was one of the requirements to get a '-1' withdrawn. Prior to that had a
similar class with different name. Not having Guava classes exposed sounds
better to me tho.
was (Author: toffer):
{quote}
Tell me more Francis Liu about deadlock. How? The concurrency issue addressed
was just making it so all modification to rsgroup state was single-threaded;
{quote}
Both read/write access can't be single threaded. Consider the situation:
1. move_rsgroup_servers is called
2. while #1 is happening rsgroup region is in transition (rpc thread in #1
holds monitor lock)
3. while #2 is happening meta is in transition.
Balancer tries to figure out plan for meta region tries to get monitor lock but
can't. rpc thread task won't release monitor lock since rsgroup region never
gets assigned. rsgroup region never gets assigned because it can't update meta
with new state.
There's a good chance this can be reproduce just by moving both rsgroup and
meta region onto the same RS and call move_rsgoup_servers on the same RS.
A bunch of threads will query from group affiliation so we can't have writes
block reads.
{quote}
previous it seemed loosey-goosey – least I couldn't figure out the regime.
{quote}
Looks like the first patch already had these changes so I can't really respond
to what may or may not be loosey goosey. I'll try to review the changes today
and see what other issues could be there.
Regarding Guava classes, sounds like this is a new policy. The HostAndPort
change was one of the requirements to get a '-1' withdrawn. Prior to that had a
similar class with different name. Not having Guava classes exposed sounds
better to me tho.
> Address late review of HBASE-6721, rsgroups feature
> ---
>
> Key: HBASE-17624
> URL: https://issues.apache.org/jira/browse/HBASE-17624
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: HBASE-17624.master.001.patch,
> HBASE-17624.master.002.patch, HBASE-17624.master.003.patch,
> HBASE-17624.master.004.patch, HBASE-17624.master.005.patch,
> HBASE-17624.master.006.patch, HBASE-17624.master.007.patch,
> HBASE-17624.master.008.patch, HBASE-17624.master.009.patch,
> HBASE-17624.master.010.patch
>
>
> An internal review by [~busbey] and [~appy] turned up a bunch of good
> findings going over HBASE-6721. They found some really good stuff a guava
> type is part of our public API and concurrency in a few core classes is
> inconsistent.
> Patch coming.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17590) Drop cache hint should work for StoreFile write path
[
https://issues.apache.org/jira/browse/HBASE-17590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868428#comment-15868428
]
Ashu Pachauri commented on HBASE-17590:
---
I checked the failing unit test, it's unrelated to the patch.
> Drop cache hint should work for StoreFile write path
>
>
> Key: HBASE-17590
> URL: https://issues.apache.org/jira/browse/HBASE-17590
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Ashu Pachauri
> Attachments: HBASE-17590.master.001.patch
>
>
> We have this in the code right now.
> {noformat}
> public Builder withShouldDropCacheBehind(boolean
> shouldDropCacheBehind/*NOT USED!!*/) {
> // TODO: HAS NO EFFECT!!! FIX!!
> return this;
> }
> {noformat}
> Creating jira to track it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17478) Avoid sending FSUtilization reports to master when quota support is not enabled
[ https://issues.apache.org/jira/browse/HBASE-17478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17478: --- Status: Open (was: Patch Available) > Avoid sending FSUtilization reports to master when quota support is not > enabled > --- > > Key: HBASE-17478 > URL: https://issues.apache.org/jira/browse/HBASE-17478 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Trivial > Fix For: HBASE-16961 > > Attachments: HBASE-17478.001.HBASE-16961.patch, > HBASE-17478.002.HBASE-16961.patch > > > Trivial little change to make sure that the RS's do not send the filesystem > utilization reports to the master when hbase.quota.enabled=false and, > similarly, that the master gracefully handles these reports when the feature > is not enabled. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17478) Avoid sending FSUtilization reports to master when quota support is not enabled
[ https://issues.apache.org/jira/browse/HBASE-17478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17478: --- Status: Patch Available (was: Open) > Avoid sending FSUtilization reports to master when quota support is not > enabled > --- > > Key: HBASE-17478 > URL: https://issues.apache.org/jira/browse/HBASE-17478 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Trivial > Fix For: HBASE-16961 > > Attachments: HBASE-17478.001.HBASE-16961.patch, > HBASE-17478.002.HBASE-16961.patch > > > Trivial little change to make sure that the RS's do not send the filesystem > utilization reports to the master when hbase.quota.enabled=false and, > similarly, that the master gracefully handles these reports when the feature > is not enabled. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17624) Address late review of HBASE-6721, rsgroups feature
[ https://issues.apache.org/jira/browse/HBASE-17624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868280#comment-15868280 ] stack commented on HBASE-17624: --- bq. Thanks stack I have concerns about the added synchronization in the getter methods (in 0.98 it will cause deadlock). Need to read through the reasonings. Are there other concurrency issues you guys addressed? Tell me more [~toffer] about deadlock. How? The concurrency issue addressed was just making it so all modification to rsgroup state was single-threaded; previous it seemed loosey-goosey -- least I couldn't figure out the regime. > Address late review of HBASE-6721, rsgroups feature > --- > > Key: HBASE-17624 > URL: https://issues.apache.org/jira/browse/HBASE-17624 > Project: HBase > Issue Type: Bug > Components: rsgroup >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-17624.master.001.patch, > HBASE-17624.master.002.patch, HBASE-17624.master.003.patch, > HBASE-17624.master.004.patch, HBASE-17624.master.005.patch, > HBASE-17624.master.006.patch, HBASE-17624.master.007.patch, > HBASE-17624.master.008.patch, HBASE-17624.master.009.patch, > HBASE-17624.master.010.patch > > > An internal review by [~busbey] and [~appy] turned up a bunch of good > findings going over HBASE-6721. They found some really good stuff a guava > type is part of our public API and concurrency in a few core classes is > inconsistent. > Patch coming. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17025) [Shell] Support space quota get/set via the shell
[ https://issues.apache.org/jira/browse/HBASE-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868316#comment-15868316 ] Josh Elser commented on HBASE-17025: bq. Is it possible to add some .rb test ? Oh yeah, for sure. I never noticed that we actually had some examples :) > [Shell] Support space quota get/set via the shell > - > > Key: HBASE-17025 > URL: https://issues.apache.org/jira/browse/HBASE-17025 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17025.001.patch, HBASE-17025.002.patch, > HBASE-17025.003.HBASE-16961.patch > > > Need to make sure that admins can use the shell to get/set the new space > quotas. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868193#comment-15868193
]
ChiaPing Tsai commented on HBASE-17623:
---
[~yuzhih...@gmail.com]
I will address your comment in the day. Thanks for your feedback.
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: after(snappy_hfilesize=755MB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.v0.patch,
> HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17651) Implement grant2 ruby shell command.
[ https://issues.apache.org/jira/browse/HBASE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868047#comment-15868047 ] Sean Busbey commented on HBASE-17651: - an additional grant command is going to be confusing. can we please make this a flag on the grant command instead? > Implement grant2 ruby shell command. > > > Key: HBASE-17651 > URL: https://issues.apache.org/jira/browse/HBASE-17651 > Project: HBase > Issue Type: Sub-task > Components: Admin >Reporter: huzheng >Assignee: huzheng > Labels: shell > Fix For: 2.0.0 > > > As HBASE-17472 said, we'll implement a new shell command grant2 to add one > or more action(s) for user, and deprecated previous grant shell command. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868048#comment-15868048
]
Sean Busbey commented on HBASE-17472:
-
Can we do this as a flag on a single grant command please? an additional
command named "grant2" is going to be confusing.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17619) Add async admin Impl which connect to RegionServer and implement close region methods.
[
https://issues.apache.org/jira/browse/HBASE-17619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868203#comment-15868203
]
Hadoop QA commented on HBASE-17619:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 41s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 0s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m 12s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 142m 30s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852825/HBASE-17619.v2.patch |
| JIRA Issue | HBASE-17619 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e1ad7a4908bb 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 33e9a8c |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5721/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5721/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Add async admin Impl which connect to RegionServer
[jira] [Work started] (HBASE-15328) Unvalidated Redirect in HMaster
[
https://issues.apache.org/jira/browse/HBASE-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-15328 started by Sean Busbey.
---
> Unvalidated Redirect in HMaster
> ---
>
> Key: HBASE-15328
> URL: https://issues.apache.org/jira/browse/HBASE-15328
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Minor
>
> See OWASP page on why we should clean it up someday:
> https://www.owasp.org/index.php/Unvalidated_Redirects_and_Forwards_Cheat_Sheet
> Here is where we do the redirect:
> {code}
> @Override
> public void doGet(HttpServletRequest request,
> HttpServletResponse response) throws ServletException, IOException {
> String redirectUrl = request.getScheme() + "://"
> + request.getServerName() + ":" + regionServerInfoPort
> + request.getRequestURI();
> response.sendRedirect(redirectUrl);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868182#comment-15868182
]
Ted Yu commented on HBASE-17623:
The GC activity was monitored for 4 minutes.
Can you capture GC activity for longer period of time ?
Nice improvement.
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: after(snappy_hfilesize=755MB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.v0.patch,
> HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868632#comment-15868632
]
Hadoop QA commented on HBASE-17069:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 20s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 4s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 141m 55s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852889/HBASE-17069.master.001.patch
|
| JIRA Issue | HBASE-17069 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux acd5dc48f38e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / de23d30 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5724/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5724/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5724/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868570#comment-15868570
]
Hadoop QA commented on HBASE-17069:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 49s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
51s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} branch-1.3 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} branch-1.3 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} branch-1.3 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} branch-1.3 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 40s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 9s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 132m 40s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapreduce.TestMultiTableSnapshotInputFormat
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:66fbe99 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852884/HBASE-17069.branch-1.3.002.patch
|
| JIRA Issue | HBASE-17069 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 9b099bc9df10 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/hbase.sh |
| git revision | branch-1.3 /
[jira] [Commented] (HBASE-17649) REST API for scan should return 410 when table is disabled
[ https://issues.apache.org/jira/browse/HBASE-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868615#comment-15868615 ] Hudson commented on HBASE-17649: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #115 (See [https://builds.apache.org/job/HBase-1.3-JDK8/115/]) HBASE-17649 REST API for scan should return 410 when table is disabled (tedyu: rev a3d021aec5583456ee2bf306f796ba89fbe95f71) * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerResultGenerator.java * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestScannerResource.java > REST API for scan should return 410 when table is disabled > -- > > Key: HBASE-17649 > URL: https://issues.apache.org/jira/browse/HBASE-17649 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17649.branch-1.3.v1.txt, 17649.v1.txt > > > Background: > When scanning using REST API, if the table is disabled in the middle of the > scan, code 410 (Gone) should be returned. > Currently in 1.3 and lower releases, ScannerResultGenerator#next() doesn't > handle TableNotEnabledException, leading to the exception being ignored and > code 204 is returned to caller. > This was first spotted by [~Apache9] when investigating HBASE-17603. > Patch for HBASE-17603 contains fix for this problem. > This issue fixes the bug for 1.3 and lower branches and adds test for > branch-1 and master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Work started] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-16188 started by Gopi Krishnan Nambiar.
-
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17652:
-
Status: Patch Available (was: Open)
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17652.master.001.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868787#comment-15868787
]
Guanghao Zhang commented on HBASE-17652:
Please take a look about HBASE-17312.
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Labels: coprocessors, incompatible
> Attachments: HBASE-17652.master.001.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17624) Address late review of HBASE-6721, rsgroups feature
[
https://issues.apache.org/jira/browse/HBASE-17624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868705#comment-15868705
]
Francis Liu commented on HBASE-17624:
-
{quote}
That makes sense. What you describe though seems like a general issue w/ our
assignment (and w/ having to rpc under a synchronization).
{quote}
Yes the problem can be solved either by fixing the synchronization in
assignment or in rsgroup. RSGroup is simple enough that it should be no problem
to make reads concurrent. There could be other actors accessign RSGroup so
better to make it concurrent so consumers don't have to worry about deadlock
(already too many things to worry about).
{quote}
If you can describe how what was originally in place made it so moves would
work though system tables were in transition – hbase:meta and hbase:rsgroup –
that'd help;
{quote}
In the code prior to this patch the getter methods that retrieve group
information (getRSGroup, ofTable, OfServer, etc) don't require the monitor lock
so the deadlock cycle is broken.
{quote}
how was update of zk cache and updates to hbase:rsgroup protected perviously? I
didn't see coherent guard (probably my misunderstanding). Lets fix.
{quote}
The methods that does mutations and updates to zk and hbase:rsgroup are
synchronized appropriately. Point me to where the incoherence is?
> Address late review of HBASE-6721, rsgroups feature
> ---
>
> Key: HBASE-17624
> URL: https://issues.apache.org/jira/browse/HBASE-17624
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: HBASE-17624.master.001.patch,
> HBASE-17624.master.002.patch, HBASE-17624.master.003.patch,
> HBASE-17624.master.004.patch, HBASE-17624.master.005.patch,
> HBASE-17624.master.006.patch, HBASE-17624.master.007.patch,
> HBASE-17624.master.008.patch, HBASE-17624.master.009.patch,
> HBASE-17624.master.010.patch
>
>
> An internal review by [~busbey] and [~appy] turned up a bunch of good
> findings going over HBASE-6721. They found some really good stuff a guava
> type is part of our public API and concurrency in a few core classes is
> inconsistent.
> Patch coming.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17652:
-
Description:
In cases where one might need to use multiple observers, say region, master and
regionserver; and the fact that only one class can be extended, it gives rise
to following pattern:
{noformat}
public class BaseMasterAndRegionObserver
extends BaseRegionObserver
implements MasterObserver
class AccessController
extends BaseMasterAndRegionObserver
implements RegionServerObserver
{noformat}
were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
There is an example of simple case too where the current design fails.
Say only one observer is needed by the coprocessor, but the design doesn't
permit extending even that single observer (see RSGroupAdminEndpoint), that
leads to full copy of Base...Observer class into coprocessor class leading to
1000s of lines of code and this ugly mix of 5 main functions with 100 useless
functions.
was:
In cases where one might need to use multiple observers, say region, master and
regionserver; and the fact that only one class can be extended, it gives rise
to following pattern:
public class BaseMasterAndRegionObserver
extends BaseRegionObserver
implements MasterObserver
class AccessController
extends BaseMasterAndRegionObserver
implements RegionServerObserver
were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
There is an example of simple case too where the current design fails.
Say only one observer is needed by the coprocessor, but the design doesn't
permit extending even that single observer (see RSGroupAdminEndpoint), that
leads to copy of full Bas
e...Observer class into coprocessor class leading to 1000s of lines of code
and this ugly mix of 5 main functions with 100 useless functions.
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17652) Use 'default' keyword in Observer interfaces to get rid of 'Base...Observer' implementations.
Appy created HBASE-17652: Summary: Use 'default' keyword in Observer interfaces to get rid of 'Base...Observer' implementations. Key: HBASE-17652 URL: https://issues.apache.org/jira/browse/HBASE-17652 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy In cases where one might need to use multiple observers, say region, master and regionserver; and the fact that only one class can be extended, it gives rise to following pattern: public class BaseMasterAndRegionObserver extends BaseRegionObserver implements MasterObserver class AccessController extends BaseMasterAndRegionObserver implements RegionServerObserver were BaseMasterAndRegionObserver is full copy of BaseMasterObserver. There is an example of simple case too where the current design fails. Say only one observer is needed by the coprocessor, but the design doesn't permit extending even that single observer (see RSGroupAdminEndpoint), that leads to copy of full Bas e...Observer class into coprocessor class leading to 1000s of lines of code and this ugly mix of 5 main functions with 100 useless functions. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[ https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17652: - Summary: Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations. (was: Use 'default' keyword in Observer interfaces to get rid of 'Base...Observer' implementations.) > Use 'default' keyword in Coprocessor Observer interfaces to get rid of > 'Base...Observer' implementations. > - > > Key: HBASE-17652 > URL: https://issues.apache.org/jira/browse/HBASE-17652 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > > In cases where one might need to use multiple observers, say region, master > and regionserver; and the fact that only one class can be extended, it gives > rise to following pattern: > public class BaseMasterAndRegionObserver > extends BaseRegionObserver > implements MasterObserver > class AccessController > extends BaseMasterAndRegionObserver > implements RegionServerObserver > were BaseMasterAndRegionObserver is full copy of BaseMasterObserver. > There is an example of simple case too where the current design fails. > Say only one observer is needed by the coprocessor, but the design > doesn't permit extending even that single observer (see > RSGroupAdminEndpoint), that leads to copy of full Bas > e...Observer class into coprocessor class leading to 1000s of lines of > code and this ugly mix of 5 main functions with 100 useless functions. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17652:
-
Attachment: HBASE-17652.master.001.patch
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17652.master.001.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868737#comment-15868737
]
Appy commented on HBASE-17652:
--
[~stack], [~busbey] ptal. Thanks.
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17652.master.001.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17652:
-
Labels: coprocessors incompatible (was: coprocessors)
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Labels: coprocessors, incompatible
> Attachments: HBASE-17652.master.001.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17652:
-
Labels: coprocessors (was: )
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Labels: coprocessors, incompatible
> Attachments: HBASE-17652.master.001.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17652) Use 'default' keyword in Coprocessor Observer interfaces to get rid of 'Base...Observer' implementations.
[
https://issues.apache.org/jira/browse/HBASE-17652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17652:
-
Release Note: TODO (update release note)
> Use 'default' keyword in Coprocessor Observer interfaces to get rid of
> 'Base...Observer' implementations.
> -
>
> Key: HBASE-17652
> URL: https://issues.apache.org/jira/browse/HBASE-17652
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Labels: coprocessors, incompatible
> Attachments: HBASE-17652.master.001.patch
>
>
> In cases where one might need to use multiple observers, say region, master
> and regionserver; and the fact that only one class can be extended, it gives
> rise to following pattern:
> {noformat}
> public class BaseMasterAndRegionObserver
> extends BaseRegionObserver
> implements MasterObserver
> class AccessController
> extends BaseMasterAndRegionObserver
> implements RegionServerObserver
> {noformat}
> were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.
> There is an example of simple case too where the current design fails.
> Say only one observer is needed by the coprocessor, but the design
> doesn't permit extending even that single observer (see
> RSGroupAdminEndpoint), that leads to full copy of Base...Observer class into
> coprocessor class leading to 1000s of lines of code and this ugly mix of 5
> main functions with 100 useless functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17624) Address late review of HBASE-6721, rsgroups feature
[
https://issues.apache.org/jira/browse/HBASE-17624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868334#comment-15868334
]
Francis Liu commented on HBASE-17624:
-
{quote}
Tell me more Francis Liu about deadlock. How? The concurrency issue addressed
was just making it so all modification to rsgroup state was single-threaded;
{quote}
Both read/write access can't be single threaded. Consider the situation:
1. move_rsgroup_servers is called
2. while #1 is happening rsgroup region is in transition (rpc thread in #1
holds monitor lock)
3. while #2 is happening meta is in transition.
Balancer tries to figure out plan for meta region tries to get monitor lock but
can't. rpc thread task won't release monitor lock since rsgroup region never
gets assigned. rsgroup region never gets assigned because it can't update meta
with new state.
There's a good chance this can be reproduce just by moving both rsgroup and
meta region onto the same RS and call move_rsgoup_servers on the same region.
A bunch of threads will query from group affiliation so we can't have writes
block reads.
{quote}
previous it seemed loosey-goosey – least I couldn't figure out the regime.
{quote}
Looks like the first patch already had these changes so I can't really respond
to what may or may not be loosey goosey. I'll try to review the changes today
and see what other issues could be there.
Regarding Guava classes, sounds like this is a new policy. The HostAndPort
change was one of the requirements to get a '-1' withdrawn. Prior to that had a
similar class with different name. Not having Guava classes exposed sounds
better to me tho.
> Address late review of HBASE-6721, rsgroups feature
> ---
>
> Key: HBASE-17624
> URL: https://issues.apache.org/jira/browse/HBASE-17624
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: HBASE-17624.master.001.patch,
> HBASE-17624.master.002.patch, HBASE-17624.master.003.patch,
> HBASE-17624.master.004.patch, HBASE-17624.master.005.patch,
> HBASE-17624.master.006.patch, HBASE-17624.master.007.patch,
> HBASE-17624.master.008.patch, HBASE-17624.master.009.patch,
> HBASE-17624.master.010.patch
>
>
> An internal review by [~busbey] and [~appy] turned up a bunch of good
> findings going over HBASE-6721. They found some really good stuff a guava
> type is part of our public API and concurrency in a few core classes is
> inconsistent.
> Patch coming.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
