[jira] [Updated] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17069:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to branch-1.2+.
Thanks [~abhishek.chouhan] for your nice work.
And [~stack] [~apurtell], let's start verifying if branch-1.2 and branch-1.3
can pass ITBLL now? If so, we can safely retire 1.1.x release then?
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false,
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Status: Open (was: Patch Available)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 1.3.0
>
> Attachments: HBASE-16188-1.3.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Attachment: (was: HBASE-16188-1.3.patch)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 1.3.0
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Work started] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-16188 started by Gopi Krishnan Nambiar.
-
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 1.3.0
>
> Attachments: HBASE-16188-1.3.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869459#comment-15869459
]
Hudson commented on HBASE-17069:
FAILURE: Integrated in Jenkins build HBase-1.4 #632 (See
[https://builds.apache.org/job/HBase-1.4/632/])
HBASE-17069 RegionServer writes invalid META entries for split daughters
(zhangduo: rev 4038917239aa34296fbf2371855fae8cd60db34c)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMutateRowsRecovery.java
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960,
[jira] [Updated] (HBASE-16981) Expand Mob Compaction Partition policy from daily to weekly, monthly
[
https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huaxiang sun updated HBASE-16981:
-
Release Note:
Mob compaction partition policy can be set by
hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'weekly'}
or
hbase> alter 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'monthly'}
Available MOB_COMPACT_PARTITION_POLICY options are "daily", "weekly" and
"monthly", the default is "daily".
When it is "weekly" policy, the mob compaction will try to compact files within
one calendar week into one for a specific partition, similar for "daily" and
"monthly".
With "weekly" policy, one mob file normally is compacted twice during its
lifetime, for one region, there normally are 52 files for one year. With
"Monthly" policy, one mob file normally is compacted 3 times during its
lifetime and normally the are 12 files for one year.
> Expand Mob Compaction Partition policy from daily to weekly, monthly
>
>
> Key: HBASE-16981
> URL: https://issues.apache.org/jira/browse/HBASE-16981
> Project: HBase
> Issue Type: New Feature
> Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Fix For: 2.0.0
>
> Attachments: HBASE-16981.master.001.patch,
> HBASE-16981.master.002.patch, HBASE-16981.master.003.patch,
> HBASE-16981.master.004.patch, HBASE-16981.master.005.patch,
> HBASE-16981.master.006.patch, HBASE-16981.master.007.patch,
> HBASE-16981.master.008.patch,
> Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf
>
>
> Today the mob region holds all mob files for all regions. With daily
> partition mob compaction policy, after major mob compaction, there is still
> one file per region daily. Given there is 365 days in one year, at least 365
> files per region. Since HDFS has limitation for number of files under one
> folder, this is not going to scale if there are lots of regions. To reduce
> mob file number, we want to introduce other partition policies such as
> weekly, monthly to compact mob files within one week or month into one file.
> This jira is create to track this effort.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17658) Bookkeeping issue in BaseLoadBalancer for Table Skew
[ https://issues.apache.org/jira/browse/HBASE-17658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870974#comment-15870974 ] Ted Yu commented on HBASE-17658: +1, pending QA. > Bookkeeping issue in BaseLoadBalancer for Table Skew > > > Key: HBASE-17658 > URL: https://issues.apache.org/jira/browse/HBASE-17658 > Project: HBase > Issue Type: Bug > Components: Balancer >Reporter: Tim Brown >Assignee: Tim Brown > Attachments: JIRA-17658.master.001.patch > > > Currently when numMaxRegionsPerTable of a given table increases above the > current maximum, the value is not properly updated. This means that the cost > for Table Skew cannot get worse for a given region move during the balancer > process leading to an imbalanced cluster with respect to Table Skew. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17619) Add async admin Impl which connect to RegionServer and implement close region methods.
[ https://issues.apache.org/jira/browse/HBASE-17619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17619: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master. Thanks [~openinx] for contributing. > Add async admin Impl which connect to RegionServer and implement close region > methods. > -- > > Key: HBASE-17619 > URL: https://issues.apache.org/jira/browse/HBASE-17619 > Project: HBase > Issue Type: Sub-task > Components: asyncclient >Affects Versions: 2.0.0 >Reporter: huzheng >Assignee: huzheng > Labels: asynchronous > Fix For: 2.0.0 > > Attachments: HBASE-17619.v1.patch, HBASE-17619.v1.patch, > HBASE-17619.v2.patch > > > Currently , Async client has async master impl and async htable impl , but > not async admin impl , which connect to RS for managing regions in region > server . > This issue will add async admin which implement AdminService.Interface to > AsyncConnectionImpl, and implement close region method. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17312) [JDK8] Use default method for Observer Coprocessors
[
https://issues.apache.org/jira/browse/HBASE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871048#comment-15871048
]
Appy commented on HBASE-17312:
--
So answering [~stack]'s question above:
Right now we have
{noformat}
public interface Foo {
void getBar1();
void getBar2();
}
public class BaseFoo {
void getBar1() { }
void getBar2() { }
}
---
class ThirdParty extends BaseFoo {
void getBar1() { // override }
}
{noformat}
That's changes to:
{noformat}
public interface Foo {
void getBar1() {}
void getBar2() {}
}
--
class ThirdParty implement Foo { // Only one line fix
void getBar1() { // override }
}
{noformat}
One weird case which i saw in tests was, overrides calling base class'
functions. It doesn't make sense because our base classes have no
functionality. They are there to just prevent redundant empty implementations
(or returning default values when hooks have non-void return type).
> [JDK8] Use default method for Observer Coprocessors
> ---
>
> Key: HBASE-17312
> URL: https://issues.apache.org/jira/browse/HBASE-17312
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-17312.master.001.patch,
> HBASE-17312.master.001.patch
>
>
> Use default method in MasterObserver, RegionObserver, RegionServerObserver
> and WALObserver. And mark the BaseRegionObserver,
> BaseMasterAndRegionObserver, BaseRegionServerObserver and BaseWALObserver.
> User can implement the interface directly and will not break compatibility
> when add new default methods.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16981) Expand Mob Compaction Partition policy from daily to weekly, monthly
[ https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871047#comment-15871047 ] Jingcheng Du commented on HBASE-16981: -- Thanks [~huaxiang], it looks good to me. > Expand Mob Compaction Partition policy from daily to weekly, monthly > > > Key: HBASE-16981 > URL: https://issues.apache.org/jira/browse/HBASE-16981 > Project: HBase > Issue Type: New Feature > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0 > > Attachments: HBASE-16981.master.001.patch, > HBASE-16981.master.002.patch, HBASE-16981.master.003.patch, > HBASE-16981.master.004.patch, HBASE-16981.master.005.patch, > HBASE-16981.master.006.patch, HBASE-16981.master.007.patch, > HBASE-16981.master.008.patch, > Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf > > > Today the mob region holds all mob files for all regions. With daily > partition mob compaction policy, after major mob compaction, there is still > one file per region daily. Given there is 365 days in one year, at least 365 > files per region. Since HDFS has limitation for number of files under one > folder, this is not going to scale if there are lots of regions. To reduce > mob file number, we want to introduce other partition policies such as > weekly, monthly to compact mob files within one week or month into one file. > This jira is create to track this effort. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17583) Add inclusive/exclusive support for startRow and endRow of scan for sync client
[ https://issues.apache.org/jira/browse/HBASE-17583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17583: -- Release Note: Now you can include/exlude the startRow and stopRow for a scan. And the new methods to specify startRow and stopRow are withStartRow and withStopRow. The old methods to specify startRow and Row(include constructors) are marked as deprecated as in the old time if startRow and stopRow are equal then we will consider it as a get scan and include the stopRow implicitly. This is strange after we can set inclusiveness explicitly so we add new methods and depredate the old methods. The deprecated methods will be removed in the future. > Add inclusive/exclusive support for startRow and endRow of scan for sync > client > --- > > Key: HBASE-17583 > URL: https://issues.apache.org/jira/browse/HBASE-17583 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17583-branch-1.patch, > HBASE-17583-branch-1-v1.patch, HBASE-17583.patch, HBASE-17583-v1.patch, > HBASE-17583-v2.patch, HBASE-17583-v3.patch > > > Implement the same feature of HBASE-17320 for sync client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Issue Comment Deleted] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Comment: was deleted
(was: Check HadoopQA)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-16188.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17421) Update refguide w.r.t. MOB Sweeper
[ https://issues.apache.org/jira/browse/HBASE-17421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-17421: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) > Update refguide w.r.t. MOB Sweeper > -- > > Key: HBASE-17421 > URL: https://issues.apache.org/jira/browse/HBASE-17421 > Project: HBase > Issue Type: Bug > Components: documentation, mob >Reporter: Ted Yu >Assignee: Jingcheng Du > Fix For: 2.0.0 > > Attachments: HBASE-17421.patch > > > The http://hbase.apache.org/book.html#_mob_sweeper section is out of date. > It mentions settings such as hbase.mob.sweep.tool.compaction.mergeable.size > which no longer apply in master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871070#comment-15871070
]
huzheng commented on HBASE-17472:
-
Oh... Sorry, Will upload a new patch. Thanks [~enis]
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0, 1.4.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17472.branch-1.3.v6.patch,
> HBASE-17472.branch-1.v6.patch, HBASE-17472.master.v6.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17658) Bookkeeping issue in BaseLoadBalancer for Table Skew
[ https://issues.apache.org/jira/browse/HBASE-17658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Brown updated HBASE-17658: -- Attachment: JIRA-17658.master.001.patch > Bookkeeping issue in BaseLoadBalancer for Table Skew > > > Key: HBASE-17658 > URL: https://issues.apache.org/jira/browse/HBASE-17658 > Project: HBase > Issue Type: Bug > Components: Balancer >Reporter: Tim Brown >Assignee: Tim Brown > Attachments: JIRA-17658.master.001.patch > > > Currently when numMaxRegionsPerTable of a given table increases above the > current maximum, the value is not properly updated. This means that the cost > for Table Skew cannot get worse for a given region move during the balancer > process leading to an imbalanced cluster with respect to Table Skew. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Attachment: HBASE-16188.patch
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 1.3.0
>
> Attachments: HBASE-16188.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-17312) [JDK8] Use default method for Observer Coprocessors
[
https://issues.apache.org/jira/browse/HBASE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871048#comment-15871048
]
Appy edited comment on HBASE-17312 at 2/17/17 2:00 AM:
---
So answering [~stack]'s question above:
Right now we have
{noformat}
public interface Foo {
void getBar1();
void getBar2();
}
public class BaseFoo {
void getBar1() { }
void getBar2() { }
}
---
class ThirdParty extends BaseFoo {
void getBar1() { // override }
}
{noformat}
That's changes to:
{noformat}
public interface Foo {
void getBar1() {}
void getBar2() {}
}
--
class ThirdParty implement Foo { // Only one line fix
void getBar1() { // override }
}
{noformat}
One weird case which i saw in tests was, overrides calling base class'
functions. It doesn't make sense because our base classes have no
functionality. They are there to just prevent redundant empty implementations
(or returning default values when hooks have non-void return type).
But that's just our tests, and we should remove these calls ([~zghaobac]).
was (Author: appy):
So answering [~stack]'s question above:
Right now we have
{noformat}
public interface Foo {
void getBar1();
void getBar2();
}
public class BaseFoo {
void getBar1() { }
void getBar2() { }
}
---
class ThirdParty extends BaseFoo {
void getBar1() { // override }
}
{noformat}
That's changes to:
{noformat}
public interface Foo {
void getBar1() {}
void getBar2() {}
}
--
class ThirdParty implement Foo { // Only one line fix
void getBar1() { // override }
}
{noformat}
One weird case which i saw in tests was, overrides calling base class'
functions. It doesn't make sense because our base classes have no
functionality. They are there to just prevent redundant empty implementations
(or returning default values when hooks have non-void return type).
> [JDK8] Use default method for Observer Coprocessors
> ---
>
> Key: HBASE-17312
> URL: https://issues.apache.org/jira/browse/HBASE-17312
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-17312.master.001.patch,
> HBASE-17312.master.001.patch
>
>
> Use default method in MasterObserver, RegionObserver, RegionServerObserver
> and WALObserver. And mark the BaseRegionObserver,
> BaseMasterAndRegionObserver, BaseRegionServerObserver and BaseWALObserver.
> User can implement the interface directly and will not break compatibility
> when add new default methods.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-17623:
--
Status: Patch Available (was: Open)
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.v0.patch, HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory
> allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-17583) Add inclusive/exclusive support for startRow and endRow of scan for sync client
[
https://issues.apache.org/jira/browse/HBASE-17583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871120#comment-15871120
]
Ted Yu edited comment on HBASE-17583 at 2/17/17 3:32 AM:
-
Looks like the following test failure is reproducible:
{code}
testEnableReplicationWhenSlaveClusterDoesntHaveTable(org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters)
Time elapsed: 1.243 sec <<< ERROR!
java.lang.IllegalArgumentException: Table test has replication already enabled
for atleast one Column Family.
at
org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenSlaveClusterDoesntHaveTable(TestReplicationAdminWithClusters.java:83)
{code}
But the test failure was present before this went in.
was (Author: yuzhih...@gmail.com):
Looks like the following test failure is reproducible:
{code}
testEnableReplicationWhenSlaveClusterDoesntHaveTable(org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters)
Time elapsed: 1.243 sec <<< ERROR!
java.lang.IllegalArgumentException: Table test has replication already enabled
for atleast one Column Family.
at
org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenSlaveClusterDoesntHaveTable(TestReplicationAdminWithClusters.java:83)
{code}
> Add inclusive/exclusive support for startRow and endRow of scan for sync
> client
> ---
>
> Key: HBASE-17583
> URL: https://issues.apache.org/jira/browse/HBASE-17583
> Project: HBase
> Issue Type: Sub-task
> Components: Client, scan
>Affects Versions: 2.0.0, 1.4.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17583-branch-1.patch,
> HBASE-17583-branch-1-v1.patch, HBASE-17583.patch, HBASE-17583-v1.patch,
> HBASE-17583-v2.patch, HBASE-17583-v3.patch
>
>
> Implement the same feature of HBASE-17320 for sync client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17653) HBASE-17624 rsgroup synchronizations will (distributed) deadlock
[
https://issues.apache.org/jira/browse/HBASE-17653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17653:
--
Attachment: HBASE-17653.master.003.patch
> HBASE-17624 rsgroup synchronizations will (distributed) deadlock
>
>
> Key: HBASE-17653
> URL: https://issues.apache.org/jira/browse/HBASE-17653
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-17653.master.001.patch,
> HBASE-17653.master.002.patch, HBASE-17653.master.003.patch
>
>
> Follow-on from HBASE-17624. HBASE-17624 made it so one thread only has access
> to the rsgroup administrator. In tail of HBASE-17624 [~toffer] describes
> scenario under which we may end up in a deadlock (distributed). Let me
> repeat [~toffer] comment...
> {code}
> Both read/write access can't be single threaded. Consider the situation:
> 1. move_rsgroup_servers is called
> 2. while #1 is happening rsgroup region is in transition (rpc thread in #1
> holds monitor lock)
> 3. while #2 is happening meta is in transition.
> Balancer tries to figure out plan for meta region tries to get monitor lock
> but can't. rpc thread task won't release monitor lock since rsgroup region
> never gets assigned. rsgroup region never gets assigned because it can't
> update meta with new state.
> There's a good chance this can be reproduce just by moving both rsgroup and
> meta region onto the same RS and call move_rsgoup_servers on the same RS.
> A bunch different actors will query from group affiliation so we can't have
> writes block reads.
>
> In the code prior to this patch the getter methods that retrieve group
> information (getRSGroup, ofTable, OfServer, etc) don't require the monitor
> lock so the deadlock cycle is broken.
>
> The methods that does mutations and updates to zk and hbase:rsgroup are
> synchronized appropriately. Point me to where the incoherence is?
> {code}
> This issue is about testing/fixing/restoring rsgroup access. Will be back.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopi Krishnan Nambiar updated HBASE-16188:
--
Fix Version/s: (was: 1.3.0)
2.0.0
Status: Patch Available (was: In Progress)
> Add EventCounter information to log4j properties file
> -
>
> Key: HBASE-16188
> URL: https://issues.apache.org/jira/browse/HBASE-16188
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.1
>Reporter: Lars George
>Assignee: Gopi Krishnan Nambiar
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-16188.patch
>
>
> Hadoop's {{JvmMetrics}}, which HBase also is using in Metrics2 and provides
> it as an MBean, has the ability to count log4j log calls. This is tracked by
> a special {{Appender}} class, also provided by Hadoop, called
> {{EventCounter}}.
> We should add some info how to enable this (or maybe even enable it by
> default?).
> The appender needs to be added in two places, shown here:
> {noformat}
> hbase.root.logger=INFO,console
> ...
> # Define the root logger to the system property "hbase.root.logger".
> log4j.rootLogger=${hbase.root.logger}, EventCounter
> log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
> {noformat}
> We could simply add this commented out akin to the {{hbase-env.sh}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17654) RSGroup code refactoring
[ https://issues.apache.org/jira/browse/HBASE-17654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17654: - Attachment: HBASE-17654.master.002.patch > RSGroup code refactoring > > > Key: HBASE-17654 > URL: https://issues.apache.org/jira/browse/HBASE-17654 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-17654.master.001.patch, > HBASE-17654.master.002.patch > > > TBD -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17654) RSGroup code refactoring
[ https://issues.apache.org/jira/browse/HBASE-17654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17654: - Attachment: HBASE-17654.master.003.patch > RSGroup code refactoring > > > Key: HBASE-17654 > URL: https://issues.apache.org/jira/browse/HBASE-17654 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-17654.master.001.patch, > HBASE-17654.master.002.patch, HBASE-17654.master.003.patch > > > TBD -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17608) Add suspend support for RawScanResultConsumer
[ https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17608: -- Attachment: HBASE-17608-v4.patch Fix the javadoc warning. > Add suspend support for RawScanResultConsumer > - > > Key: HBASE-17608 > URL: https://issues.apache.org/jira/browse/HBASE-17608 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17608.patch, HBASE-17608-v1.patch, > HBASE-17608-v2.patch, HBASE-17608-v3.patch, HBASE-17608-v4.patch > > > Now for the AsyncResultScanner, we can only close the scanner if we reach the > cache size limit and open a new scanner later. This will breaks the region > level consistency. > For example, you put 10 rows to the region, and open a scanner to scan it. > The scanner returns 5 rows at the first time and the cache is full, so it > closes the background scanner. And before you reopens the scanner to fetch > data, the remaining 5 rows has been deleted and a compaction makes them gone > for ever. Then after you reopen the scanner you can not see the remaining 5 > rows. > So here we should keep the scanner open at RS side to prevent the data below > the mvcc read point of this scanner being deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17608) Add suspend support for RawScanResultConsumer
[ https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871094#comment-15871094 ] Duo Zhang commented on HBASE-17608: --- Ping [~stack]. > Add suspend support for RawScanResultConsumer > - > > Key: HBASE-17608 > URL: https://issues.apache.org/jira/browse/HBASE-17608 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17608.patch, HBASE-17608-v1.patch, > HBASE-17608-v2.patch, HBASE-17608-v3.patch, HBASE-17608-v4.patch > > > Now for the AsyncResultScanner, we can only close the scanner if we reach the > cache size limit and open a new scanner later. This will breaks the region > level consistency. > For example, you put 10 rows to the region, and open a scanner to scan it. > The scanner returns 5 rows at the first time and the cache is full, so it > closes the background scanner. And before you reopens the scanner to fetch > data, the remaining 5 rows has been deleted and a compaction makes them gone > for ever. Then after you reopen the scanner you can not see the remaining 5 > rows. > So here we should keep the scanner open at RS side to prevent the data below > the mvcc read point of this scanner being deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17583) Add inclusive/exclusive support for startRow and endRow of scan for sync client
[
https://issues.apache.org/jira/browse/HBASE-17583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871120#comment-15871120
]
Ted Yu commented on HBASE-17583:
Looks like the following test failure is reproducible:
{code}
testEnableReplicationWhenSlaveClusterDoesntHaveTable(org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters)
Time elapsed: 1.243 sec <<< ERROR!
java.lang.IllegalArgumentException: Table test has replication already enabled
for atleast one Column Family.
at
org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenSlaveClusterDoesntHaveTable(TestReplicationAdminWithClusters.java:83)
{code}
> Add inclusive/exclusive support for startRow and endRow of scan for sync
> client
> ---
>
> Key: HBASE-17583
> URL: https://issues.apache.org/jira/browse/HBASE-17583
> Project: HBase
> Issue Type: Sub-task
> Components: Client, scan
>Affects Versions: 2.0.0, 1.4.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17583-branch-1.patch,
> HBASE-17583-branch-1-v1.patch, HBASE-17583.patch, HBASE-17583-v1.patch,
> HBASE-17583-v2.patch, HBASE-17583-v3.patch
>
>
> Implement the same feature of HBASE-17320 for sync client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17658) Bookkeeping issue in BaseLoadBalancer for Table Skew
[ https://issues.apache.org/jira/browse/HBASE-17658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Brown updated HBASE-17658: -- Status: Patch Available (was: In Progress) > Bookkeeping issue in BaseLoadBalancer for Table Skew > > > Key: HBASE-17658 > URL: https://issues.apache.org/jira/browse/HBASE-17658 > Project: HBase > Issue Type: Bug > Components: Balancer >Reporter: Tim Brown >Assignee: Tim Brown > Attachments: JIRA-17658.master.001.patch > > > Currently when numMaxRegionsPerTable of a given table increases above the > current maximum, the value is not properly updated. This means that the cost > for Table Skew cannot get worse for a given region move during the balancer > process leading to an imbalanced cluster with respect to Table Skew. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Work started] (HBASE-17658) Bookkeeping issue in BaseLoadBalancer for Table Skew
[ https://issues.apache.org/jira/browse/HBASE-17658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-17658 started by Tim Brown. - > Bookkeeping issue in BaseLoadBalancer for Table Skew > > > Key: HBASE-17658 > URL: https://issues.apache.org/jira/browse/HBASE-17658 > Project: HBase > Issue Type: Bug > Components: Balancer >Reporter: Tim Brown >Assignee: Tim Brown > Attachments: JIRA-17658.master.001.patch > > > Currently when numMaxRegionsPerTable of a given table increases above the > current maximum, the value is not properly updated. This means that the cost > for Table Skew cannot get worse for a given region move during the balancer > process leading to an imbalanced cluster with respect to Table Skew. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17653) HBASE-17624 rsgroup synchronizations will (distributed) deadlock
[
https://issues.apache.org/jira/browse/HBASE-17653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17653:
--
Attachment: HBASE-17653.master.002.patch
> HBASE-17624 rsgroup synchronizations will (distributed) deadlock
>
>
> Key: HBASE-17653
> URL: https://issues.apache.org/jira/browse/HBASE-17653
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-17653.master.001.patch,
> HBASE-17653.master.002.patch
>
>
> Follow-on from HBASE-17624. HBASE-17624 made it so one thread only has access
> to the rsgroup administrator. In tail of HBASE-17624 [~toffer] describes
> scenario under which we may end up in a deadlock (distributed). Let me
> repeat [~toffer] comment...
> {code}
> Both read/write access can't be single threaded. Consider the situation:
> 1. move_rsgroup_servers is called
> 2. while #1 is happening rsgroup region is in transition (rpc thread in #1
> holds monitor lock)
> 3. while #2 is happening meta is in transition.
> Balancer tries to figure out plan for meta region tries to get monitor lock
> but can't. rpc thread task won't release monitor lock since rsgroup region
> never gets assigned. rsgroup region never gets assigned because it can't
> update meta with new state.
> There's a good chance this can be reproduce just by moving both rsgroup and
> meta region onto the same RS and call move_rsgoup_servers on the same RS.
> A bunch different actors will query from group affiliation so we can't have
> writes block reads.
>
> In the code prior to this patch the getter methods that retrieve group
> information (getRSGroup, ofTable, OfServer, etc) don't require the monitor
> lock so the deadlock cycle is broken.
>
> The methods that does mutations and updates to zk and hbase:rsgroup are
> synchronized appropriately. Point me to where the incoherence is?
> {code}
> This issue is about testing/fixing/restoring rsgroup access. Will be back.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17654) RSGroup code refactoring
[
https://issues.apache.org/jira/browse/HBASE-17654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871004#comment-15871004
]
Hadoop QA commented on HBASE-17654:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s {color}
| {color:red} HBASE-17654 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853184/HBASE-17654.master.003.patch
|
| JIRA Issue | HBASE-17654 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5745/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> RSGroup code refactoring
>
>
> Key: HBASE-17654
> URL: https://issues.apache.org/jira/browse/HBASE-17654
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17654.master.001.patch,
> HBASE-17654.master.002.patch, HBASE-17654.master.003.patch
>
>
> TBD
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17654) RSGroup code refactoring
[ https://issues.apache.org/jira/browse/HBASE-17654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871017#comment-15871017 ] stack commented on HBASE-17654: --- Tests failed. I think Closeable is already removed? There is already a Utility class for rsgroup called Utility so RSGroupUtil clashes. Leave as it was? How does RSGroupStartupWorker get an instance of masterServices now it does not take it in constructor. Any reason to change the ServerEventsListenerThread so it listened on events instead? (I'd rather get rid of that thread if we could). There is a bunch of nice cleanup in here. RB is down. Will give you better review when it comes back. > RSGroup code refactoring > > > Key: HBASE-17654 > URL: https://issues.apache.org/jira/browse/HBASE-17654 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-17654.master.001.patch, > HBASE-17654.master.002.patch, HBASE-17654.master.003.patch > > > TBD -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17421) Update refguide w.r.t. MOB Sweeper
[ https://issues.apache.org/jira/browse/HBASE-17421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871063#comment-15871063 ] Jingcheng Du commented on HBASE-17421: -- Pushed to the master branch. Thanks [~tedyu] for the review! > Update refguide w.r.t. MOB Sweeper > -- > > Key: HBASE-17421 > URL: https://issues.apache.org/jira/browse/HBASE-17421 > Project: HBase > Issue Type: Bug > Components: documentation, mob >Reporter: Ted Yu >Assignee: Jingcheng Du > Attachments: HBASE-17421.patch > > > The http://hbase.apache.org/book.html#_mob_sweeper section is out of date. > It mentions settings such as hbase.mob.sweep.tool.compaction.mergeable.size > which no longer apply in master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-17623:
--
Attachment: HBASE-17623.branch-1.v0.patch
I borrow some codes from HBASE-11862 and HBASE-15077 for branch-1.
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.v0.patch, HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory
> allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huzheng updated HBASE-17472:
Attachment: HBASE-17472.branch-1.v7.patch
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0, 1.4.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17472.branch-1.3.v6.patch,
> HBASE-17472.branch-1.v6.patch, HBASE-17472.branch-1.v7.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.master.v6.patch,
> HBASE-17472.v1.patch, HBASE-17472.v2.patch, HBASE-17472.v3.patch,
> HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-17623:
--
Fix Version/s: 1.4.0
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.v0.patch, HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory
> allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-17623:
--
Status: Open (was: Patch Available)
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.v0.patch, HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory
> allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (HBASE-16431) Add missing method in class HTableWrapper
[ https://issues.apache.org/jira/browse/HBASE-16431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-16431. - Resolution: Cannot Reproduce > Add missing method in class HTableWrapper > - > > Key: HBASE-16431 > URL: https://issues.apache.org/jira/browse/HBASE-16431 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.2 >Reporter: Biao Ma > Fix For: 1.2.2 > > > Class `HTableWrapper` must either declared abstract or implement abstract > method 'setRpcTimeout(int)' in 'Table' -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17657) TestZKAsyncRegistry is flaky
[
https://issues.apache.org/jira/browse/HBASE-17657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870805#comment-15870805
]
Hadoop QA commented on HBASE-17657:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 4s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 57s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m 6s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 130m 52s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853123/17657.v1.txt |
| JIRA Issue | HBASE-17657 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 4e5c847f8e1e 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 0446ed0 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5739/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5739/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TestZKAsyncRegistry is flaky
> -
>
> Key: HBASE-17657
> URL: https://issues.apache.org/jira/browse/HBASE-17657
> Project: HBase
> Issue
[jira] [Commented] (HBASE-17312) [JDK8] Use default method for Observer Coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870809#comment-15870809 ] Andrew Purtell commented on HBASE-17312: No discussion needed. My answer is based on a lack of discussion actually. Look at what's supported on master. Note the lack of discussion about 2.0. :-) Sure, Phoenix won't be on 1.x forever. When they move to 2.x there will be a lot of work needed regardless, as you point out [~enis] > [JDK8] Use default method for Observer Coprocessors > --- > > Key: HBASE-17312 > URL: https://issues.apache.org/jira/browse/HBASE-17312 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17312.master.001.patch, > HBASE-17312.master.001.patch > > > Use default method in MasterObserver, RegionObserver, RegionServerObserver > and WALObserver. And mark the BaseRegionObserver, > BaseMasterAndRegionObserver, BaseRegionServerObserver and BaseWALObserver. > User can implement the interface directly and will not break compatibility > when add new default methods. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870847#comment-15870847
]
Vladimir Rodionov commented on HBASE-14123:
---
{quote}
Is the patch here contains all of the jiras linked to HBASE-14414 and resolved?
{quote}
Yes, all of them are in the patch. I should have added HBASE-14030 (Phase 1) to
the list of JIRAs as well.
> HBase Backup/Restore Phase 2
>
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
> Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: HBASE-7912
>
> Attachments: 14123-master.v14.txt, 14123-master.v15.txt,
> 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt,
> 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt,
> 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt,
> 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt,
> 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt,
> 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt,
> 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt,
> 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch,
> 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch,
> 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch,
> 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch,
> 14123.master.v52.patch, 14123.master.v54.patch, 14123.master.v56.patch,
> 14123.master.v57.patch, 14123-master.v5.txt, 14123-master.v6.txt,
> 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt,
> Backup-restoreinHBase2.0.pdf, HBASE-14123-for-7912-v1.patch,
> HBASE-14123-for-7912-v6.patch, HBASE-14123-v10.patch, HBASE-14123-v11.patch,
> HBASE-14123-v12.patch, HBASE-14123-v13.patch, HBASE-14123-v15.patch,
> HBASE-14123-v16.patch, HBASE-14123-v1.patch, HBASE-14123-v2.patch,
> HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch,
> HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17653) HBASE-17624 rsgroup synchronizations will (distributed) deadlock
[
https://issues.apache.org/jira/browse/HBASE-17653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17653:
--
Attachment: HBASE-17653.master.001.patch
> HBASE-17624 rsgroup synchronizations will (distributed) deadlock
>
>
> Key: HBASE-17653
> URL: https://issues.apache.org/jira/browse/HBASE-17653
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-17653.master.001.patch
>
>
> Follow-on from HBASE-17624. HBASE-17624 made it so one thread only has access
> to the rsgroup administrator. In tail of HBASE-17624 [~toffer] describes
> scenario under which we may end up in a deadlock (distributed). Let me
> repeat [~toffer] comment...
> {code}
> Both read/write access can't be single threaded. Consider the situation:
> 1. move_rsgroup_servers is called
> 2. while #1 is happening rsgroup region is in transition (rpc thread in #1
> holds monitor lock)
> 3. while #2 is happening meta is in transition.
> Balancer tries to figure out plan for meta region tries to get monitor lock
> but can't. rpc thread task won't release monitor lock since rsgroup region
> never gets assigned. rsgroup region never gets assigned because it can't
> update meta with new state.
> There's a good chance this can be reproduce just by moving both rsgroup and
> meta region onto the same RS and call move_rsgoup_servers on the same RS.
> A bunch different actors will query from group affiliation so we can't have
> writes block reads.
>
> In the code prior to this patch the getter methods that retrieve group
> information (getRSGroup, ofTable, OfServer, etc) don't require the monitor
> lock so the deadlock cycle is broken.
>
> The methods that does mutations and updates to zk and hbase:rsgroup are
> synchronized appropriately. Point me to where the incoherence is?
> {code}
> This issue is about testing/fixing/restoring rsgroup access. Will be back.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17656) Move new Address class from util to net package
[ https://issues.apache.org/jira/browse/HBASE-17656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17656: -- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed minor issue rename of a class to master branch. > Move new Address class from util to net package > --- > > Key: HBASE-17656 > URL: https://issues.apache.org/jira/browse/HBASE-17656 > Project: HBase > Issue Type: Sub-task > Components: rsgroup >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17656.master.001.patch > > > I added the new Address type in the wrong package committing the parent JIRA > patch. It should be in net, not util. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Reopened] (HBASE-16431) Add missing method in class HTableWrapper
[ https://issues.apache.org/jira/browse/HBASE-16431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-16431: - > Add missing method in class HTableWrapper > - > > Key: HBASE-16431 > URL: https://issues.apache.org/jira/browse/HBASE-16431 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.2 >Reporter: Biao Ma > Fix For: 1.2.2 > > > Class `HTableWrapper` must either declared abstract or implement abstract > method 'setRpcTimeout(int)' in 'Table' -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14123: -- Attachment: Backup-restoreinHBase2.0 (1).pdf > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: HBASE-7912 > > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt, > 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt, > 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt, > 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt, > 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt, > 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt, > 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch, > 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch, > 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch, > 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch, > 14123.master.v52.patch, 14123.master.v54.patch, 14123.master.v56.patch, > 14123.master.v57.patch, 14123-master.v5.txt, 14123-master.v6.txt, > 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, > Backup-restoreinHBase2.0 (1).pdf, Backup-restoreinHBase2.0.pdf, > HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch, > HBASE-14123-v10.patch, HBASE-14123-v11.patch, HBASE-14123-v12.patch, > HBASE-14123-v13.patch, HBASE-14123-v15.patch, HBASE-14123-v16.patch, > HBASE-14123-v1.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch, > HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, > HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17656) Move new Address class from util to net package
[
https://issues.apache.org/jira/browse/HBASE-17656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870855#comment-15870855
]
Hadoop QA commented on HBASE-17656:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 0s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 0s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 28s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 47s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 58s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 101m 15s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 48s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 56s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
56s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 170m 25s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.replication.regionserver.TestTableBasedReplicationSourceManagerImpl
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853122/HBASE-17656.master.001.patch
|
| JIRA Issue | HBASE-17656 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile rubocop ruby_lint |
| uname
[jira] [Commented] (HBASE-17172) Optimize mob compaction with _del files
[
https://issues.apache.org/jira/browse/HBASE-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870829#comment-15870829
]
Hadoop QA commented on HBASE-17172:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 3s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 16s
{color} | {color:red} hbase-server generated 2 new + 0 unchanged - 0 fixed = 2
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m 6s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 132m 6s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Invocation of hashCode on array in
org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactionRequest$CompactionDelPartitionId.hashCode()
At PartitionedMobCompactionRequest.java:in
org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactionRequest$CompactionDelPartitionId.hashCode()
At PartitionedMobCompactionRequest.java:[line 317] |
| | Invocation of hashCode on array in
org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactionRequest$CompactionDelPartitionId.hashCode()
At PartitionedMobCompactionRequest.java:in
org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactionRequest$CompactionDelPartitionId.hashCode()
At PartitionedMobCompactionRequest.java:[line 316] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853127/HBASE-17172.master.002.patch
|
| JIRA Issue | HBASE-17172 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 8910d2e18c5d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0446ed0 |
| Default Java | 1.8.0_121 |
| findbugs
[jira] [Commented] (HBASE-17312) [JDK8] Use default method for Observer Coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870784#comment-15870784 ] Enis Soztutar commented on HBASE-17312: --- bq. Phoenix has no support for master branch and no development for same currently planned. Also, no plans to move up to JDK8 even. I don't remember seeing that kind of discussion on the mailing list. Regardless, it does not mean that Phoenix will be stuck forever on the 1.x line. My understanding is that you have to fix your coprocessor anyway between 1.x and 2.0 (because of other API changes). Removing the base classes should be fine for now. > [JDK8] Use default method for Observer Coprocessors > --- > > Key: HBASE-17312 > URL: https://issues.apache.org/jira/browse/HBASE-17312 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17312.master.001.patch, > HBASE-17312.master.001.patch > > > Use default method in MasterObserver, RegionObserver, RegionServerObserver > and WALObserver. And mark the BaseRegionObserver, > BaseMasterAndRegionObserver, BaseRegionServerObserver and BaseWALObserver. > User can implement the interface directly and will not break compatibility > when add new default methods. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870851#comment-15870851
]
Vladimir Rodionov commented on HBASE-14123:
---
{quote}
You should also mention https://issues.apache.org/jira/browse/HBASE-16825 in
the doc.
{quote}
This JIRA is linked to Phase 3 and is not resolved yet - meaning this is what
we need to finish before 2.0 release.
> HBase Backup/Restore Phase 2
>
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
> Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: HBASE-7912
>
> Attachments: 14123-master.v14.txt, 14123-master.v15.txt,
> 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt,
> 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt,
> 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt,
> 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt,
> 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt,
> 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt,
> 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt,
> 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch,
> 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch,
> 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch,
> 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch,
> 14123.master.v52.patch, 14123.master.v54.patch, 14123.master.v56.patch,
> 14123.master.v57.patch, 14123-master.v5.txt, 14123-master.v6.txt,
> 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt,
> Backup-restoreinHBase2.0.pdf, HBASE-14123-for-7912-v1.patch,
> HBASE-14123-for-7912-v6.patch, HBASE-14123-v10.patch, HBASE-14123-v11.patch,
> HBASE-14123-v12.patch, HBASE-14123-v13.patch, HBASE-14123-v15.patch,
> HBASE-14123-v16.patch, HBASE-14123-v1.patch, HBASE-14123-v2.patch,
> HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch,
> HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17649) REST API for scan should return 410 when table is disabled
[ https://issues.apache.org/jira/browse/HBASE-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870768#comment-15870768 ] Hudson commented on HBASE-17649: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #102 (See [https://builds.apache.org/job/HBase-1.2-JDK7/102/]) HBASE-17649 REST API for scan should return 410 when table is disabled (tedyu: rev 81c0f24b9439d3393c7748b62e97e4117c4997d7) * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestScannerResource.java * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerResultGenerator.java > REST API for scan should return 410 when table is disabled > -- > > Key: HBASE-17649 > URL: https://issues.apache.org/jira/browse/HBASE-17649 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5 > > Attachments: 17649.branch-1.3.v1.txt, 17649.v1.txt > > > Background: > When scanning using REST API, if the table is disabled in the middle of the > scan, code 410 (Gone) should be returned. > Currently in 1.3 and lower releases, ScannerResultGenerator#next() doesn't > handle TableNotEnabledException, leading to the exception being ignored and > code 204 is returned to caller. > This was first spotted by [~Apache9] when investigating HBASE-17603. > Patch for HBASE-17603 contains fix for this problem. > This issue fixes the bug for 1.3 and lower branches and adds test for > branch-1 and master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870790#comment-15870790
]
Enis Soztutar commented on HBASE-17472:
---
please remove all mentions of "grant2" from the method names, etc.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0, 1.4.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17472.branch-1.3.v6.patch,
> HBASE-17472.branch-1.v6.patch, HBASE-17472.master.v6.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870831#comment-15870831
]
Ted Yu commented on HBASE-17623:
Got some compilation errors applying patch on branch-1:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile)
on project hbase-common: Compilation failure: Compilation failure:
[ERROR]
/Users/tyu/1-hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:[194,15]
error: constructor Bytes in class Bytes cannot be applied to given types;
[ERROR] int,int
[ERROR] reason: actual and formal argument lists differ in length
[ERROR]
/Users/tyu/1-hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:[198,15]
error: constructor Bytes in class Bytes cannot be applied to given types;
[ERROR] int,int
[ERROR] reason: actual and formal argument lists differ in length
[ERROR]
/Users/tyu/1-hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:[212,15]
error: constructor Bytes in class Bytes cannot be applied to given types;
{code}
Chiaping:
Can you attach patch for branch-1 ?
Thanks
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.v0.patch,
> HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17653) HBASE-17624 rsgroup synchronizations will (distributed) deadlock
[
https://issues.apache.org/jira/browse/HBASE-17653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870919#comment-15870919
]
stack commented on HBASE-17653:
---
[~toffer] Please review. Sorry for making you work.
> HBASE-17624 rsgroup synchronizations will (distributed) deadlock
>
>
> Key: HBASE-17653
> URL: https://issues.apache.org/jira/browse/HBASE-17653
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-17653.master.001.patch
>
>
> Follow-on from HBASE-17624. HBASE-17624 made it so one thread only has access
> to the rsgroup administrator. In tail of HBASE-17624 [~toffer] describes
> scenario under which we may end up in a deadlock (distributed). Let me
> repeat [~toffer] comment...
> {code}
> Both read/write access can't be single threaded. Consider the situation:
> 1. move_rsgroup_servers is called
> 2. while #1 is happening rsgroup region is in transition (rpc thread in #1
> holds monitor lock)
> 3. while #2 is happening meta is in transition.
> Balancer tries to figure out plan for meta region tries to get monitor lock
> but can't. rpc thread task won't release monitor lock since rsgroup region
> never gets assigned. rsgroup region never gets assigned because it can't
> update meta with new state.
> There's a good chance this can be reproduce just by moving both rsgroup and
> meta region onto the same RS and call move_rsgoup_servers on the same RS.
> A bunch different actors will query from group affiliation so we can't have
> writes block reads.
>
> In the code prior to this patch the getter methods that retrieve group
> information (getRSGroup, ofTable, OfServer, etc) don't require the monitor
> lock so the deadlock cycle is broken.
>
> The methods that does mutations and updates to zk and hbase:rsgroup are
> synchronized appropriately. Point me to where the incoherence is?
> {code}
> This issue is about testing/fixing/restoring rsgroup access. Will be back.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16981) Expand Mob Compaction Partition policy from daily to weekly, monthly
[
https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huaxiang sun updated HBASE-16981:
-
Release Note:
Mob compaction partition policy can be set by
hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'weekly'}
or
hbase> alter 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'monthly'}
Available MOB_COMPACT_PARTITION_POLICY options are "daily", "weekly" and
"monthly", the default is "daily".
When it is "weekly" policy, the mob compaction will try to compact files within
one calendar week into one for a specific partition, similar for "daily" and
"monthly".
With "weekly" policy, one mob file normally is compacted twice during its
lifetime, for one region, there normally are 52 files for one year. With
"Monthly" policy, one mob file normally is compacted 3 times during its
lifetime and normally there are 12 files for one year.
was:
Mob compaction partition policy can be set by
hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'weekly'}
or
hbase> alter 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'monthly'}
Available MOB_COMPACT_PARTITION_POLICY options are "daily", "weekly" and
"monthly", the default is "daily".
When it is "weekly" policy, the mob compaction will try to compact files within
one calendar week into one for a specific partition, similar for "daily" and
"monthly".
With "weekly" policy, one mob file normally is compacted twice during its
lifetime, for one region, there normally are 52 files for one year. With
"Monthly" policy, one mob file normally is compacted 3 times during its
lifetime and normally the are 12 files for one year.
> Expand Mob Compaction Partition policy from daily to weekly, monthly
>
>
> Key: HBASE-16981
> URL: https://issues.apache.org/jira/browse/HBASE-16981
> Project: HBase
> Issue Type: New Feature
> Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Fix For: 2.0.0
>
> Attachments: HBASE-16981.master.001.patch,
> HBASE-16981.master.002.patch, HBASE-16981.master.003.patch,
> HBASE-16981.master.004.patch, HBASE-16981.master.005.patch,
> HBASE-16981.master.006.patch, HBASE-16981.master.007.patch,
> HBASE-16981.master.008.patch,
> Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf
>
>
> Today the mob region holds all mob files for all regions. With daily
> partition mob compaction policy, after major mob compaction, there is still
> one file per region daily. Given there is 365 days in one year, at least 365
> files per region. Since HDFS has limitation for number of files under one
> folder, this is not going to scale if there are lots of regions. To reduce
> mob file number, we want to introduce other partition policies such as
> weekly, monthly to compact mob files within one week or month into one file.
> This jira is create to track this effort.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17312) [JDK8] Use default method for Observer Coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870811#comment-15870811 ] Zach York commented on HBASE-17312: --- bq. Phoenix has no support for master branch and no development for same currently planned. Also, no plans to move up to JDK8 even. I think when we get close to releasing HBase 2.0.0, Phoenix HBase branch-2.0.0 work/JDK 8 work should be discussed/start. I think it is okay to create JIRAs for that now in case people want to pick it up. In general, I think using default methods here is a good approach. (Much better to do it and remove the Base classes now than having them deprecated for all of the 2.x line). [~zghaobac] Can you post this on Review Board? I would like to review, but it is hard to review a patch. > [JDK8] Use default method for Observer Coprocessors > --- > > Key: HBASE-17312 > URL: https://issues.apache.org/jira/browse/HBASE-17312 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17312.master.001.patch, > HBASE-17312.master.001.patch > > > Use default method in MasterObserver, RegionObserver, RegionServerObserver > and WALObserver. And mark the BaseRegionObserver, > BaseMasterAndRegionObserver, BaseRegionServerObserver and BaseWALObserver. > User can implement the interface directly and will not break compatibility > when add new default methods. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huzheng updated HBASE-17472:
Attachment: HBASE-17472.master.v7.patch
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0, 1.4.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17472.branch-1.3.v6.patch,
> HBASE-17472.branch-1.v6.patch, HBASE-17472.branch-1.v7.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.master.v6.patch,
> HBASE-17472.master.v7.patch, HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17460) enable_table_replication can not perform cyclic replication of a table
[ https://issues.apache.org/jira/browse/HBASE-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871147#comment-15871147 ] Ted Yu commented on HBASE-17460: I reverted this from master branch as well : TestReplicationAdminWithClusters consistently failed in the past several trunk builds. > enable_table_replication can not perform cyclic replication of a table > -- > > Key: HBASE-17460 > URL: https://issues.apache.org/jira/browse/HBASE-17460 > Project: HBase > Issue Type: Bug > Components: Replication >Reporter: NITIN VERMA >Assignee: NITIN VERMA > Labels: replication > Fix For: 2.0.0, 1.4.0 > > Attachments: 17460.branch-1.v3.txt, HBASE-17460.patch, > HBASE-17460_v2.patch, HBASE-17460_v3.patch, HBASE-17460_v4.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > The enable_table_replication operation is broken for cyclic replication of > HBase table as we compare all the properties of column families (including > REPLICATION_SCOPE). > Below is exactly what happens: > 1. Running "enable_table_replication 'table1' " opeartion on first cluster > will set the REPLICATION_SCOPE of all column families to peer id '1'. This > will also create a table on second cluster where REPLICATION_SCOPE is still > set to peer id '0'. > 2. Now when we run "enable_table_replication 'table1'" on second cluster, we > compare all the properties of table (including REPLICATION_SCOPE_, which > obviously is different now. > I am proposing a fix for this issue where we should avoid comparing > REPLICATION_SCOPE inside HColumnDescriotor::compareTo() method, especially > when replication is not already enabled on the desired table. > I have made that change and it is working. I will submit the patch soon. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-17623:
--
Attachment: HBASE-17623.branch-1.v1.patch
# fix the findbugs warn.
# TestSimpleRpcScheduler passes locally.
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.branch-1.v1.patch, HBASE-17623.v0.patch, HBASE-17623.v1.patch,
> HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17608) Add suspend support for RawScanResultConsumer
[
https://issues.apache.org/jira/browse/HBASE-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871201#comment-15871201
]
Hadoop QA commented on HBASE-17608:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 25s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 1s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 11s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m 26s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 45s
{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
53s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 147m 42s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853194/HBASE-17608-v4.patch |
| JIRA Issue | HBASE-17608 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 3124e44db6bd 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f4202a1 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5746/testReport/ |
|
[jira] [Commented] (HBASE-16188) Add EventCounter information to log4j properties file
[
https://issues.apache.org/jira/browse/HBASE-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871258#comment-15871258
]
Hadoop QA commented on HBASE-16188:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 34s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 39s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 1s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 46s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 24s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s
{color} | {color:green} hbase-prefix-tree in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m 20s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 33s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 22s
{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 24s
{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 12s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 36s
{color} | {color:green} hbase-client-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 36s
{color} | {color:green} hbase-shaded-client-project in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 126m 9s {color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 3m
10s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 301m 14s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.thrift.TestThriftServerCmdLine |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853181/HBASE-16188.patch |
| JIRA Issue | HBASE-16188 |
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux 7e6405f57ca6 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ae840c0 |
|
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871302#comment-15871302
]
ChiaPing Tsai commented on HBASE-17623:
---
bq. Did u get a chance to run this for a longer time (May be use PE with 100GB+
data) and see the GC impacts (Positive or negative)?
copy that.
bq. You test with G1 or CMS?
CMS
Thanks for the feedback. [~anoop.hbase]
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.branch-1.v1.patch, HBASE-17623.v0.patch, HBASE-17623.v1.patch,
> HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871148#comment-15871148
]
huzheng commented on HBASE-17472:
-
Upload patch v7 for removing all grant2.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0, 1.4.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17472.branch-1.3.v6.patch,
> HBASE-17472.branch-1.v6.patch, HBASE-17472.branch-1.v7.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.master.v6.patch,
> HBASE-17472.master.v7.patch, HBASE-17472.v1.patch, HBASE-17472.v2.patch,
> HBASE-17472.v3.patch, HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-17623:
--
Status: Patch Available (was: Open)
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.branch-1.v1.patch, HBASE-17623.v0.patch, HBASE-17623.v1.patch,
> HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-17623:
--
Status: Open (was: Patch Available)
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.v0.patch, HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory
> allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17583) Add inclusive/exclusive support for startRow and endRow of scan for sync client
[ https://issues.apache.org/jira/browse/HBASE-17583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871128#comment-15871128 ] Duo Zhang commented on HBASE-17583: --- File a issue to track it? > Add inclusive/exclusive support for startRow and endRow of scan for sync > client > --- > > Key: HBASE-17583 > URL: https://issues.apache.org/jira/browse/HBASE-17583 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17583-branch-1.patch, > HBASE-17583-branch-1-v1.patch, HBASE-17583.patch, HBASE-17583-v1.patch, > HBASE-17583-v2.patch, HBASE-17583-v3.patch > > > Implement the same feature of HBASE-17320 for sync client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871146#comment-15871146
]
Hadoop QA commented on HBASE-17623:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 31s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} branch-1 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 48s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} branch-1 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 46s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 54s
{color} | {color:red} hbase-common generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 51s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 16s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 35s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-common |
| | org.apache.hadoop.hbase.util.Bytes.equals(Object) checks for operand
being a byte[] At Bytes.java:a byte[] At Bytes.java:[line 267] |
| Failed
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871203#comment-15871203
]
Anoop Sam John commented on HBASE-17623:
Did u get a chance to run this for a longer time (May be use PE with 100GB+
data) and see the GC impacts (Positive or negative)? Reuse the byte[] for
building the next blocks also, makes the life of this byte[] longer and make it
promote to old gen? Objects die at young gen is always better wrt GC. That
is why I was asking for a longer run and see the GC. You test with G1 or CMS?
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.branch-1.v1.patch, HBASE-17623.v0.patch, HBASE-17623.v1.patch,
> HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871315#comment-15871315
]
Hadoop QA commented on HBASE-17623:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 35s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
48s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} branch-1 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 4s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} branch-1 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 58s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 45s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 93m 34s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 132m 29s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:e01ee2f |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853213/HBASE-17623.branch-1.v1.patch
|
| JIRA Issue | HBASE-17623 |
| Optional Tests
[jira] [Commented] (HBASE-17656) Move new Address class from util to net package
[ https://issues.apache.org/jira/browse/HBASE-17656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871133#comment-15871133 ] Hudson commented on HBASE-17656: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2518 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2518/]) HBASE-17656 Move new Address class from util to net package (stack: rev ae840c0ccd123985d5cde186cc2d0a0e92cb2467) * (edit) hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/VerifyingRSGroupAdminClient.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminClient.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ServerName.java * (edit) hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsBase.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * (add) hbase-common/src/main/java/org/apache/hadoop/hbase/net/Address.java * (edit) hbase-shell/src/main/ruby/hbase/rsgroup_admin.rb * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdmin.java * (edit) hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRSGroupBasedLoadBalancer.java * (delete) hbase-common/src/main/java/org/apache/hadoop/hbase/util/Address.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterAndRegionObserver.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupSerDe.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/Utility.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManager.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java * (edit) hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroups.java > Move new Address class from util to net package > --- > > Key: HBASE-17656 > URL: https://issues.apache.org/jira/browse/HBASE-17656 > Project: HBase > Issue Type: Sub-task > Components: rsgroup >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17656.master.001.patch > > > I added the new Address type in the wrong package committing the parent JIRA > patch. It should be in net, not util. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16981) Expand Mob Compaction Partition policy from daily to weekly, monthly
[
https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-16981:
---
Release Note:
Mob compaction partition policy can be set by
hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'weekly'}
or
hbase> alter 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'monthly'}
Available MOB_COMPACT_PARTITION_POLICY options are "daily", "weekly" and
"monthly", the default is "daily".
When it is "weekly" policy, the mob compaction will try to compact files within
one calendar week into one for a specific partition, similar for "daily" and
"monthly".
With "weekly" policy, one mob file normally is compacted twice during its
lifetime (that is first on daily basis and then all such daily based compacted
files belonging to a week at the weekly interval), for one region, there
normally are 52 files for one year. With "Monthly" policy, one mob file
normally is compacted 3 times during its lifetime (First daily and then weekly
followed by monthly at end of every month) and normally there are 12 files for
one year.
was:
Mob compaction partition policy can be set by
hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'weekly'}
or
hbase> alter 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 100,
MOB_COMPACT_PARTITION_POLICY => 'monthly'}
Available MOB_COMPACT_PARTITION_POLICY options are "daily", "weekly" and
"monthly", the default is "daily".
When it is "weekly" policy, the mob compaction will try to compact files within
one calendar week into one for a specific partition, similar for "daily" and
"monthly".
With "weekly" policy, one mob file normally is compacted twice during its
lifetime, for one region, there normally are 52 files for one year. With
"Monthly" policy, one mob file normally is compacted 3 times during its
lifetime and normally there are 12 files for one year.
> Expand Mob Compaction Partition policy from daily to weekly, monthly
>
>
> Key: HBASE-16981
> URL: https://issues.apache.org/jira/browse/HBASE-16981
> Project: HBase
> Issue Type: New Feature
> Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Fix For: 2.0.0
>
> Attachments: HBASE-16981.master.001.patch,
> HBASE-16981.master.002.patch, HBASE-16981.master.003.patch,
> HBASE-16981.master.004.patch, HBASE-16981.master.005.patch,
> HBASE-16981.master.006.patch, HBASE-16981.master.007.patch,
> HBASE-16981.master.008.patch,
> Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf
>
>
> Today the mob region holds all mob files for all regions. With daily
> partition mob compaction policy, after major mob compaction, there is still
> one file per region daily. Given there is 365 days in one year, at least 365
> files per region. Since HDFS has limitation for number of files under one
> folder, this is not going to scale if there are lots of regions. To reduce
> mob file number, we want to introduce other partition policies such as
> weekly, monthly to compact mob files within one week or month into one file.
> This jira is create to track this effort.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16981) Expand Mob Compaction Partition policy from daily to weekly, monthly
[ https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871157#comment-15871157 ] Anoop Sam John commented on HBASE-16981: A bit of update is done so as to explain the weekly and monthly doing 2 or 3 times IO. Thanks.. Looks good. > Expand Mob Compaction Partition policy from daily to weekly, monthly > > > Key: HBASE-16981 > URL: https://issues.apache.org/jira/browse/HBASE-16981 > Project: HBase > Issue Type: New Feature > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0 > > Attachments: HBASE-16981.master.001.patch, > HBASE-16981.master.002.patch, HBASE-16981.master.003.patch, > HBASE-16981.master.004.patch, HBASE-16981.master.005.patch, > HBASE-16981.master.006.patch, HBASE-16981.master.007.patch, > HBASE-16981.master.008.patch, > Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf > > > Today the mob region holds all mob files for all regions. With daily > partition mob compaction policy, after major mob compaction, there is still > one file per region daily. Given there is 365 days in one year, at least 365 > files per region. Since HDFS has limitation for number of files under one > folder, this is not going to scale if there are lots of regions. To reduce > mob file number, we want to introduce other partition policies such as > weekly, monthly to compact mob files within one week or month into one file. > This jira is create to track this effort. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869910#comment-15869910
]
Hudson commented on HBASE-17069:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #94 (See
[https://builds.apache.org/job/HBase-1.2-JDK8/94/])
HBASE-17069 RegionServer writes invalid META entries for split daughters
(zhangduo: rev 3f9cb1ae2c8b316354d2ded7c41275cb27499ce0)
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMutateRowsRecovery.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960,
[jira] [Commented] (HBASE-17619) Add async admin Impl which connect to RegionServer and implement close region methods.
[ https://issues.apache.org/jira/browse/HBASE-17619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869934#comment-15869934 ] Duo Zhang commented on HBASE-17619: --- Will commit tomorrow if no objections. Thanks. > Add async admin Impl which connect to RegionServer and implement close region > methods. > -- > > Key: HBASE-17619 > URL: https://issues.apache.org/jira/browse/HBASE-17619 > Project: HBase > Issue Type: Sub-task > Components: asyncclient >Affects Versions: 2.0.0 >Reporter: huzheng >Assignee: huzheng > Labels: asynchronous > Fix For: 2.0.0 > > Attachments: HBASE-17619.v1.patch, HBASE-17619.v1.patch, > HBASE-17619.v2.patch > > > Currently , Async client has async master impl and async htable impl , but > not async admin impl , which connect to RS for managing regions in region > server . > This issue will add async admin which implement AdminService.Interface to > AsyncConnectionImpl, and implement close region method. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869882#comment-15869882
]
Hadoop QA commented on HBASE-17472:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m 44s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 9s
{color} | {color:green} branch-1.3 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} branch-1.3 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
7s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
50s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
49s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} branch-1.3 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} branch-1.3 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 9s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 21s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 39s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 81m 10s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 146m 28s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests
[jira] [Updated] (HBASE-17644) Always create ByteBufferCells after copying to MSLAB
[ https://issues.apache.org/jira/browse/HBASE-17644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-17644: --- Attachment: HBASE-17644_V3.patch > Always create ByteBufferCells after copying to MSLAB > > > Key: HBASE-17644 > URL: https://issues.apache.org/jira/browse/HBASE-17644 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-17644.patch, HBASE-17644_V2.patch, > HBASE-17644_V3.patch > > > We create a cell out of the Bytebuffer that is returned by MSLAB. This BB can > be offheap or onheap type. If MSLAB is full we return onheap byte buffer. So > the place where we create a cell by copying data to this buffer we either > create KeyValue or OffheapKV based on the buffer type. So what we saw in > tests is that since we have a combination of Cells the comparisons that > happens when adding to memstore happens millions of times and that has an > impact on the performance of write path. In read path this is not significant > enougth (though we have plans to just create one type of cells every where in > Server side). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869986#comment-15869986
]
Hudson commented on HBASE-17069:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #101 (See
[https://builds.apache.org/job/HBase-1.2-JDK7/101/])
HBASE-17069 RegionServer writes invalid META entries for split daughters
(zhangduo: rev 3f9cb1ae2c8b316354d2ded7c41275cb27499ce0)
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMutateRowsRecovery.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960,
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870005#comment-15870005
]
Ted Yu commented on HBASE-17623:
Planning to commit tomorrow morning, if there is no more review comment.
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.v0.patch,
> HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17472:
--
Affects Version/s: 1.4.0
Fix Version/s: 1.4.0
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0, 1.4.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17472.branch-1.3.v6.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.master.v6.patch,
> HBASE-17472.v1.patch, HBASE-17472.v2.patch, HBASE-17472.v3.patch,
> HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869895#comment-15869895
]
Duo Zhang commented on HBASE-17472:
---
I think master and branch-1 is enough. We do not need to commit this to 1.3
unless someone wants this feature.
If no other concerns let's commit the patch tomorrow.
And [~openinx], you need to take care of the shell command. If we also want to
introduce a flag for the grant command in shell, you need to find a proper to
do it as the default behavior on master and branch-1 are different.
Thanks.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0, 1.4.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17472.branch-1.3.v6.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.master.v6.patch,
> HBASE-17472.v1.patch, HBASE-17472.v2.patch, HBASE-17472.v3.patch,
> HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17627) Active workers metric for thrift
[ https://issues.apache.org/jira/browse/HBASE-17627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869633#comment-15869633 ] Hudson commented on HBASE-17627: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #116 (See [https://builds.apache.org/job/HBase-1.3-JDK8/116/]) HBASE-17627 Active workers metric for thrift (Ashu Pachauri) (garyh: rev f2c8b5c66e0b607c09cc5b8bf3fef33fa8e468df) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/TBoundedThreadPoolServer.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftMetrics.java * (add) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/THBaseThreadPoolExecutor.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSource.java > Active workers metric for thrift > > > Key: HBASE-17627 > URL: https://issues.apache.org/jira/browse/HBASE-17627 > Project: HBase > Issue Type: Improvement > Components: Thrift >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: HBASE-17627.branch-1.001.patch, > HBASE-17627.master.001.patch > > > It would be good to have a metric for number of active handlers on thrift > servers, which gives a good indication of business of a thrift server. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17644) Always create ByteBufferCells after copying to MSLAB
[ https://issues.apache.org/jira/browse/HBASE-17644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-17644: --- Attachment: HBASE-17644_V2.patch Fixed test failure. Those because of typecasting into KVs. Renamed TestOffheapKeyValue also as per the class name change > Always create ByteBufferCells after copying to MSLAB > > > Key: HBASE-17644 > URL: https://issues.apache.org/jira/browse/HBASE-17644 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-17644.patch, HBASE-17644_V2.patch > > > We create a cell out of the Bytebuffer that is returned by MSLAB. This BB can > be offheap or onheap type. If MSLAB is full we return onheap byte buffer. So > the place where we create a cell by copying data to this buffer we either > create KeyValue or OffheapKV based on the buffer type. So what we saw in > tests is that since we have a combination of Cells the comparisons that > happens when adding to memstore happens millions of times and that has an > impact on the performance of write path. In read path this is not significant > enougth (though we have plans to just create one type of cells every where in > Server side). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17654) RSGroup code refactoring
Appy created HBASE-17654: Summary: RSGroup code refactoring Key: HBASE-17654 URL: https://issues.apache.org/jira/browse/HBASE-17654 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy TBD -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-17623:
--
Attachment: before(snappy_hfilesize=5.04GB).png
after(snappy_hfilesize=5.04GB).png
The tests runs for 30 minutes.
|| ||before||after||
|memory allocation|241.18 GB|150.89 GB|
|pause time|107.59 s|60.732 s|
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.v0.patch,
> HBASE-17623.v1.patch, HBASE-17623.v1.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The uncompressedBlockBytesWithHeader and onDiskBlockBytesWithHeader should
> maintain a bytes array which can be reused when building the hfile.
> # The uncompressedBlockBytesWithHeader/onDiskBlockBytesWithHeader is copied
> to an new bytes array only when we need to cache the block.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huzheng updated HBASE-17472:
Attachment: HBASE-17472.branch-1.3.v6.patch
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.branch-1.3.v6.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.master.v6.patch,
> HBASE-17472.v1.patch, HBASE-17472.v2.patch, HBASE-17472.v3.patch,
> HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869714#comment-15869714
]
Hudson commented on HBASE-17069:
FAILURE: Integrated in Jenkins build HBase-1.3-IT #831 (See
[https://builds.apache.org/job/HBase-1.3-IT/831/])
HBASE-17069 RegionServer writes invalid META entries for split daughters
(zhangduo: rev 3f9cb1ae2c8b316354d2ded7c41275cb27499ce0)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMutateRowsRecovery.java
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960,
[jira] [Commented] (HBASE-17644) Always create ByteBufferCells after copying to MSLAB
[
https://issues.apache.org/jira/browse/HBASE-17644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869817#comment-15869817
]
Hadoop QA commented on HBASE-17644:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 14 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 27s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 3s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 10s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 1s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m 48s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 135m 0s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.security.visibility.TestVisibilityLabelReplicationWithExpAsString |
| | hadoop.hbase.client.TestZKAsyncRegistry |
| Timed out junit tests |
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithCustomVisLabService
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853018/HBASE-17644_V2.patch |
| JIRA Issue | HBASE-17644 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 0fb6f6892e17 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0446ed0 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
[jira] [Updated] (HBASE-17421) Update refguide w.r.t. MOB Sweeper
[ https://issues.apache.org/jira/browse/HBASE-17421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-17421: - Attachment: HBASE-17421.patch Upload the patch for review. Please take a look, I will commit it if it is okay. Thanks. > Update refguide w.r.t. MOB Sweeper > -- > > Key: HBASE-17421 > URL: https://issues.apache.org/jira/browse/HBASE-17421 > Project: HBase > Issue Type: Bug > Components: documentation, mob >Reporter: Ted Yu >Assignee: Jingcheng Du > Attachments: HBASE-17421.patch > > > The http://hbase.apache.org/book.html#_mob_sweeper section is out of date. > It mentions settings such as hbase.mob.sweep.tool.compaction.mergeable.size > which no longer apply in master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17421) Update refguide w.r.t. MOB Sweeper
[ https://issues.apache.org/jira/browse/HBASE-17421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-17421: - Status: Patch Available (was: Open) > Update refguide w.r.t. MOB Sweeper > -- > > Key: HBASE-17421 > URL: https://issues.apache.org/jira/browse/HBASE-17421 > Project: HBase > Issue Type: Bug > Components: documentation, mob >Reporter: Ted Yu >Assignee: Jingcheng Du > Attachments: HBASE-17421.patch > > > The http://hbase.apache.org/book.html#_mob_sweeper section is out of date. > It mentions settings such as hbase.mob.sweep.tool.compaction.mergeable.size > which no longer apply in master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869660#comment-15869660
]
Hadoop QA commented on HBASE-17472:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
2s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 50s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 9s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 10s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 150m 15s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852998/HBASE-17472.master.v6.patch
|
| JIRA Issue | HBASE-17472 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile cc hbaseprotoc |
| uname | Linux ac301409a16a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d2c083d |
| Default Java
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869779#comment-15869779
]
Hudson commented on HBASE-17069:
FAILURE: Integrated in Jenkins build HBase-1.2-IT #597 (See
[https://builds.apache.org/job/HBase-1.2-IT/597/])
HBASE-17069 RegionServer writes invalid META entries for split daughters
(zhangduo: rev 3f9cb1ae2c8b316354d2ded7c41275cb27499ce0)
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMutateRowsRecovery.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960,
[jira] [Commented] (HBASE-17069) RegionServer writes invalid META entries for split daughters in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869794#comment-15869794
]
Hudson commented on HBASE-17069:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2514 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2514/])
HBASE-17069 RegionServer writes invalid META entries for split daughters
(zhangduo: rev 0446ed0f0c662608a7816a9c3fc3becacc682fb4)
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMutateRowsRecovery.java
> RegionServer writes invalid META entries for split daughters in some
> circumstances
> --
>
> Key: HBASE-17069
> URL: https://issues.apache.org/jira/browse/HBASE-17069
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>
> Attachments: daughter_1_d55ef81c2f8299abbddfce0445067830.log,
> daughter_2_08629d59564726da2497f70451aafcdb.log,
> HBASE-17069.branch-1.3.001.patch, HBASE-17069.branch-1.3.002.patch,
> HBASE-17069.master.001.patch, logs.tar.gz,
> parent-393d2bfd8b1c52ce08540306659624f2.log
>
>
> I have been seeing frequent ITBLL failures testing various versions of 1.2.x.
> Over the lifetime of 1.2.x the following issues have been fixed:
> - HBASE-15315 (Remove always set super user call as high priority)
> - HBASE-16093 (Fix splits failed before creating daughter regions leave meta
> inconsistent)
> And this one is pending:
> - HBASE-17044 (Fix merge failed before creating merged region leaves meta
> inconsistent)
> I can apply all of the above to branch-1.2 and still see this failure:
> *The life of stillborn region d55ef81c2f8299abbddfce0445067830*
> *Master sees SPLITTING_NEW*
> {noformat}
> 2016-11-08 04:23:21,186 INFO [AM.ZK.Worker-pool2-t82] master.RegionStates:
> Transition null to {d55ef81c2f8299abbddfce0445067830 state=SPLITTING_NEW,
> ts=1478579001186, server=node-3.cluster,16020,1478578389506}
> {noformat}
> *The RegionServer creates it*
> {noformat}
> 2016-11-08 04:23:26,035 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for GomnU: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,038 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for big: blockCache=LruBlockCache{blockCount=34,
> currentSize=14996112, freeSize=12823716208, maxSize=12838712320,
> heapSize=14996112, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,442 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for meta: blockCache=LruBlockCache{blockCount=63,
> currentSize=17187656, freeSize=12821524664, maxSize=12838712320,
> heapSize=17187656, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,713 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for nwmrW: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5, singleSize=3049194240, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2016-11-08 04:23:26,715 INFO
> [StoreOpener-d55ef81c2f8299abbddfce0445067830-1] hfile.CacheConfig: Created
> cacheConfig for piwbr: blockCache=LruBlockCache{blockCount=96,
> currentSize=19178440, freeSize=12819533880, maxSize=12838712320,
> heapSize=19178440, minSize=12196776960, minFactor=0.95, multiSize=6098388480,
> multiFactor=0.5,
[jira] [Updated] (HBASE-17518) HBase Reference Guide has a syntax error
[ https://issues.apache.org/jira/browse/HBASE-17518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang Ningbo updated HBASE-17518: - Resolution: Fixed Status: Resolved (was: Patch Available) > HBase Reference Guide has a syntax error > > > Key: HBASE-17518 > URL: https://issues.apache.org/jira/browse/HBASE-17518 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Zhang Ningbo >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17518.patch > > > The image of "HFile Version 2 Structure" in Appendix F of HBase Reference > Guide (pdf) is missing because of a wrong asciidoc syntax: > image:hfilev2.png[HFile Version 2] > modified as: > image::hfilev2.png[HFile Version 2] > it should be a double colon instead of single one -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869707#comment-15869707
]
huzheng commented on HBASE-17472:
-
Upload patch for branch-1.3: Provide a extra grant(...,
mergeExistingPermissions) rpc api for user, and grant(...) behavior the
same as before.
After that, I'll add a flag in grant ruby shell command for branch-1.3 to
invoke grant(..., mergeExstingPermissions). patch will be uploaded under
HBASE-17651.
[~busbey], Thanks for your advise.
> Correct the semantic of permission grant
> -
>
> Key: HBASE-17472
> URL: https://issues.apache.org/jira/browse/HBASE-17472
> Project: HBase
> Issue Type: Improvement
> Components: Admin
>Affects Versions: 2.0.0
>Reporter: huzheng
>Assignee: huzheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17472.branch-1.3.v6.patch,
> HBASE-17472.master.v6.patch, HBASE-17472.master.v6.patch,
> HBASE-17472.v1.patch, HBASE-17472.v2.patch, HBASE-17472.v3.patch,
> HBASE-17472.v4.patch, HBASE-17472.v5.patch
>
>
> Currently, HBase grant operation has following semantic:
> {code}
> hbase(main):019:0> grant 'hbase_tst', 'RW', 'ycsb'
> 0 row(s) in 0.0960 seconds
> hbase(main):020:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
>
> hbase_tst default,ycsb,,:
> [Permission:actions=READ,WRITE]
>
>
> 1 row(s) in 0.0550 seconds
> hbase(main):021:0> grant 'hbase_tst', 'CA', 'ycsb'
> 0 row(s) in 0.0820 seconds
> hbase(main):022:0> user_permission 'ycsb'
> User
> Namespace,Table,Family,Qualifier:Permission
>
>
> hbase_tst default,ycsb,,:
> [Permission: actions=CREATE,ADMIN]
>
>
> 1 row(s) in 0.0490 seconds
> {code}
> Later permission will replace previous granted permissions, which confused
> most of HBase administrator.
> It's seems more reasonable that HBase merge multiple granted permission.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17651) Implement grant2 ruby shell command.
[ https://issues.apache.org/jira/browse/HBASE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huzheng updated HBASE-17651: Fix Version/s: (was: 2.0.0) 1.3.0 > Implement grant2 ruby shell command. > > > Key: HBASE-17651 > URL: https://issues.apache.org/jira/browse/HBASE-17651 > Project: HBase > Issue Type: Sub-task > Components: Admin >Reporter: huzheng >Assignee: huzheng > Labels: shell > Fix For: 1.3.0 > > > As HBASE-17472 said, we'll implement a new shell command grant2 to add one > or more action(s) for user, and deprecated previous grant shell command. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17654) RSGroup code refactoring
[
https://issues.apache.org/jira/browse/HBASE-17654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869770#comment-15869770
]
Hadoop QA commented on HBASE-17654:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 33s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 55s {color}
| {color:red} hbase-rsgroup in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 28s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.rsgroup.TestRSGroups |
| | hadoop.hbase.rsgroup.TestRSGroupsOfflineMode |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853027/HBASE-17654.master.001.patch
|
| JIRA Issue | HBASE-17654 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 8a92eb9afdad 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0446ed0 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5734/artifact/patchprocess/patch-unit-hbase-rsgroup.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5734/artifact/patchprocess/patch-unit-hbase-rsgroup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5734/testReport/ |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5734/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> RSGroup code refactoring
>
>
> Key: HBASE-17654
>
[jira] [Commented] (HBASE-17651) Add a flag for grant ruby shell command to merge existing permissions.
[ https://issues.apache.org/jira/browse/HBASE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869710#comment-15869710 ] huzheng commented on HBASE-17651: - Will upload a patch as [~busbey] said. > Add a flag for grant ruby shell command to merge existing permissions. > -- > > Key: HBASE-17651 > URL: https://issues.apache.org/jira/browse/HBASE-17651 > Project: HBase > Issue Type: Sub-task > Components: Admin >Reporter: huzheng >Assignee: huzheng > Labels: shell > Fix For: 1.3.0 > > > As HBASE-17472 said, we'll implement a new shell command grant2 to add one > or more action(s) for user, and deprecated previous grant shell command. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17651) Add a flag for grant ruby shell command to merge existing permissions.
[ https://issues.apache.org/jira/browse/HBASE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huzheng updated HBASE-17651: Summary: Add a flag for grant ruby shell command to merge existing permissions. (was: Implement grant2 ruby shell command.) > Add a flag for grant ruby shell command to merge existing permissions. > -- > > Key: HBASE-17651 > URL: https://issues.apache.org/jira/browse/HBASE-17651 > Project: HBase > Issue Type: Sub-task > Components: Admin >Reporter: huzheng >Assignee: huzheng > Labels: shell > Fix For: 1.3.0 > > > As HBASE-17472 said, we'll implement a new shell command grant2 to add one > or more action(s) for user, and deprecated previous grant shell command. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17654) RSGroup code refactoring
[ https://issues.apache.org/jira/browse/HBASE-17654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17654: - Attachment: HBASE-17654.master.001.patch > RSGroup code refactoring > > > Key: HBASE-17654 > URL: https://issues.apache.org/jira/browse/HBASE-17654 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-17654.master.001.patch > > > TBD -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17472) Correct the semantic of permission grant
[
https://issues.apache.org/jira/browse/HBASE-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869669#comment-15869669
]
Hadoop QA commented on HBASE-17472:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 3s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 17s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 12s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 85m 55s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 144m 5s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852999/HBASE-17472.master.v6.patch
|
| JIRA Issue | HBASE-17472 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile cc hbaseprotoc |
| uname | Linux 1ceaf6d40d93 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d2c083d |
|
[jira] [Updated] (HBASE-17654) RSGroup code refactoring
[ https://issues.apache.org/jira/browse/HBASE-17654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17654: - Status: Patch Available (was: Open) > RSGroup code refactoring > > > Key: HBASE-17654 > URL: https://issues.apache.org/jira/browse/HBASE-17654 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-17654.master.001.patch > > > TBD -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17421) Update refguide w.r.t. MOB Sweeper
[
https://issues.apache.org/jira/browse/HBASE-17421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869737#comment-15869737
]
Hadoop QA commented on HBASE-17421:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 1s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 7s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 2s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 56s {color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 139m 51s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853011/HBASE-17421.patch |
| JIRA Issue | HBASE-17421 |
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux ff6b2f898601 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0446ed0 |
| Default Java | 1.8.0_121 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5731/artifact/patchprocess/patch-unit-root.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5731/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5731/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5731/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Update refguide w.r.t. MOB Sweeper
> --
>
> Key: HBASE-17421
> URL: https://issues.apache.org/jira/browse/HBASE-17421
> Project: HBase
> Issue Type: Bug
> Components: documentation, mob
>Reporter: Ted Yu
>Assignee: Jingcheng Du
> Attachments: HBASE-17421.patch
>
>
> The http://hbase.apache.org/book.html#_mob_sweeper section is out of date.
> It mentions settings such as hbase.mob.sweep.tool.compaction.mergeable.size
> which no longer apply in master branch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
