[jira] [Commented] (HBASE-20698) Master don't record right server version until new started region server call regionServerReport method
[
https://issues.apache.org/jira/browse/HBASE-20698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505729#comment-16505729
]
Hadoop QA commented on HBASE-20698:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
52s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 57s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}107m
9s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}147m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20698 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12927011/HBASE-20698.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d3a739adec53 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13154/testReport/ |
| Max. process+thread count | 4127 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13154/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Master don't record right
[jira] [Commented] (HBASE-20691) Storage policy should allow deferring to HDFS
[
https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505727#comment-16505727
]
Hadoop QA commented on HBASE-20691:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
52s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 12s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
23s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}116m 5s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 2m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaHighReplication
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20691 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12927008/HBASE-20691.v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 9f6149b89f2f 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache
[jira] [Commented] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[
https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505704#comment-16505704
]
Hadoop QA commented on HBASE-20704:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
19s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
58s{color} | {color:red} hbase-server: The patch generated 6 new + 43 unchanged
- 0 fixed = 49 total (was 43) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
27s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 28s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}156m
2s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}194m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20704 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12927001/HBASE-20704.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7f6a75428161 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11
12:16:42 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/13151/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13151/testReport/ |
| Max. process+thread count | 4532 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-20699) QuotaCache should cancel the QuotaRefresherChore service inside its stop()
[
https://issues.apache.org/jira/browse/HBASE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505699#comment-16505699
]
Pankaj Kumar commented on HBASE-20699:
--
Lgtm.
> QuotaCache should cancel the QuotaRefresherChore service inside its stop()
> --
>
> Key: HBASE-20699
> URL: https://issues.apache.org/jira/browse/HBASE-20699
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20699.master.001.patch,
> HBASE-20699.master.002.patch
>
>
> *ANALYSIS*
> * Called from {{HRegionServer.run()}} in case a rs is aborted:
> {code:java}
> // Stop the quota manager
> if (rsQuotaManager != null) {
> rsQuotaManager.stop();
> }
> {code}
> * Inside {{RegionServerRpcQuotaManager.stop()}}:
> {code:java}
> public void stop() {
> if (isQuotaEnabled()) {
> quotaCache.stop("shutdown");
> }
> }
> {code}
> * {{QuotaCache starts QuotaRefresherChore in}}{{ QuotaCache.start()}}:
> {code:java}
> public void start() throws IOException {
> stopped = false;
> // TODO: This will be replaced once we have the notification bus ready.
> Configuration conf = rsServices.getConfiguration();
> int period = conf.getInt(REFRESH_CONF_KEY, REFRESH_DEFAULT_PERIOD);
> refreshChore = new QuotaRefresherChore(period, this);
> rsServices.getChoreService().scheduleChore(refreshChore);
> }
> {code}
> * {{QuotaCache does not cancel refreshChore inside }}{{QuotaCache.stop()}}:
> {code:java}
> @Override
> public void stop(final String why) {
> stopped = true;
> }
> {code}
> *IMPACT:*
> QuotaRefresherChore may cause some retrying operation to delay rs abort
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20697) Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations()
[
https://issues.apache.org/jira/browse/HBASE-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505675#comment-16505675
]
zhaoyuan commented on HBASE-20697:
--
[~zghaobac] Let me try to work on this.
> Can't cache All region locations of the specify table by calling
> table.getRegionLocator().getAllRegionLocations()
> -
>
> Key: HBASE-20697
> URL: https://issues.apache.org/jira/browse/HBASE-20697
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6
>Reporter: zhaoyuan
>Assignee: zhaoyuan
>Priority: Minor
>
> When we upgrade and restart a new version application which will read and
> write to HBase, we will get some operation timeout. The time out is expected
> because when the application restarts，It will not hold any region locations
> cache and do communication with zk and meta regionserver to get region
> locations.
> We want to avoid these timeouts so we do warmup work and as far as I am
> concerned,the method table.getRegionLocator().getAllRegionLocations() will
> fetch all region locations and cache them. However, it didn't work good.
> There are still a lot of time outs,so it confused me.
> I dig into the source code and find something below
> {code:java}
> // code placeholder
> public List getAllRegionLocations() throws IOException {
> TableName tableName = getName();
> NavigableMap locations =
> MetaScanner.allTableRegions(this.connection, tableName);
> ArrayList regions = new ArrayList<>(locations.size());
> for (Entry entry : locations.entrySet()) {
> regions.add(new HRegionLocation(entry.getKey(), entry.getValue()));
> }
> if (regions.size() > 0) {
> connection.cacheLocation(tableName, new RegionLocations(regions));
> }
> return regions;
> }
> In MetaCache
> public void cacheLocation(final TableName tableName, final RegionLocations
> locations) {
> byte [] startKey =
> locations.getRegionLocation().getRegionInfo().getStartKey();
> ConcurrentMap tableLocations =
> getTableLocations(tableName);
> RegionLocations oldLocation = tableLocations.putIfAbsent(startKey,
> locations);
> boolean isNewCacheEntry = (oldLocation == null);
> if (isNewCacheEntry) {
> if (LOG.isTraceEnabled()) {
> LOG.trace("Cached location: " + locations);
> }
> addToCachedServers(locations);
> return;
> }
> {code}
> It will collect all regions into one RegionLocations object and only cache
> the first not null region location and then when we put or get to hbase, we
> do getCacheLocation()
> {code:java}
> // code placeholder
> public RegionLocations getCachedLocation(final TableName tableName, final
> byte [] row) {
> ConcurrentNavigableMap tableLocations =
> getTableLocations(tableName);
> Entry e = tableLocations.floorEntry(row);
> if (e == null) {
> if (metrics!= null) metrics.incrMetaCacheMiss();
> return null;
> }
> RegionLocations possibleRegion = e.getValue();
> // make sure that the end key is greater than the row we're looking
> // for, otherwise the row actually belongs in the next region, not
> // this one. the exception case is when the endkey is
> // HConstants.EMPTY_END_ROW, signifying that the region we're
> // checking is actually the last region in the table.
> byte[] endKey =
> possibleRegion.getRegionLocation().getRegionInfo().getEndKey();
> if (Bytes.equals(endKey, HConstants.EMPTY_END_ROW) ||
> getRowComparator(tableName).compareRows(
> endKey, 0, endKey.length, row, 0, row.length) > 0) {
> if (metrics != null) metrics.incrMetaCacheHit();
> return possibleRegion;
> }
> // Passed all the way through, so we got nothing - complete cache miss
> if (metrics != null) metrics.incrMetaCacheMiss();
> return null;
> }
> {code}
> It will choose the first location to be possibleRegion and possibly it will
> miss match.
> So did I forget something or may be wrong somewhere? If this is indeed a bug
> I think it can be fixed not very hard.
> Hope commiters and PMC review this !
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20695) Implement table level RegionServer replication metrics
[
https://issues.apache.org/jira/browse/HBASE-20695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505676#comment-16505676
]
Guanghao Zhang commented on HBASE-20695:
{code:java}
if (!source.getSourceMetrics().getSingleSourceSourceByTable()
.containsKey(tableName.getNameAsString())) {
source.getSourceMetrics().getSingleSourceSourceByTable()
.put(tableName.getNameAsString(),
CompatibilitySingletonFactory.getInstance(MetricsReplicationSourceFactory.class)
.getSource(tableName.getNameAsString()));
}
{code}
Do we have concurrent problem here? And maybe better put this logic in
MetricSource and use computeIfAbsent instead.
> Implement table level RegionServer replication metrics
> ---
>
> Key: HBASE-20695
> URL: https://issues.apache.org/jira/browse/HBASE-20695
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Xu Cang
>Assignee: Xu Cang
>Priority: Minor
> Attachments: HBASE-20695.master.001.patch,
> HBASE-20695.master.002.patch
>
>
> Region server metrics now are mainly global metrics. It would be nice to have
> table level metrics such as table level source.AgeOfLastShippedOp to indicate
> operators which table's replication is lagging behind.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20695) Implement table level RegionServer replication metrics
[ https://issues.apache.org/jira/browse/HBASE-20695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505677#comment-16505677 ] Guanghao Zhang commented on HBASE-20695: And please add a ut for this. > Implement table level RegionServer replication metrics > --- > > Key: HBASE-20695 > URL: https://issues.apache.org/jira/browse/HBASE-20695 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Minor > Attachments: HBASE-20695.master.001.patch, > HBASE-20695.master.002.patch > > > Region server metrics now are mainly global metrics. It would be nice to have > table level metrics such as table level source.AgeOfLastShippedOp to indicate > operators which table's replication is lagging behind. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20700) Move meta region when server crash can cause the procedure to be stuck
[ https://issues.apache.org/jira/browse/HBASE-20700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505674#comment-16505674 ] stack commented on HBASE-20700: --- Agree with your reasoning even that RMP is strange. It has in it all needed to onilne the meta region including recovery (meta recovery is not like other recovery -- it has its own dedicated WALs so it can be onlined before any other region). We run RMP always, because it looks for WALs to split always... its hard to discern it a clean startup from a messy one, just so there just the one way only of onlining meta. SCP fires off an RMP when it notices the crashed server was carrying meta. bq. Oh for meta there is another problem... The RecoverMetaProcedure will hold the exclusive lock for the meta table, and since the MRP for meta will hold the shared lock on meta table so the RecoverMetaProcedure can not be executed... This is a problem though. I need to run the unit test to manufacture the condition? MRP and hbase:meta needs particular treatment? Its not like any other region. It has to be online for all other stuff to work... .so RMP should have precedence. > Move meta region when server crash can cause the procedure to be stuck > -- > > Key: HBASE-20700 > URL: https://issues.apache.org/jira/browse/HBASE-20700 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20700-UT.patch > > > As said in HBASE-20682. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20697) Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations()
[
https://issues.apache.org/jira/browse/HBASE-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505662#comment-16505662
]
Guanghao Zhang commented on HBASE-20697:
{code:java}
/**
* Container for holding a list of {@link HRegionLocation}'s that correspond to
the
* same range. The list is indexed by the replicaId. This is an immutable list,
* however mutation operations are provided which returns a new List via
copy-on-write
* (assuming small number of locations)
*/
{code}
The javadoc shows that RegionLocations should be a list of HRegionLocation of
one region. So here is a misuse for RegionLocations. Are you interested to
attach a patch to fix this?
> Can't cache All region locations of the specify table by calling
> table.getRegionLocator().getAllRegionLocations()
> -
>
> Key: HBASE-20697
> URL: https://issues.apache.org/jira/browse/HBASE-20697
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6
>Reporter: zhaoyuan
>Assignee: zhaoyuan
>Priority: Minor
>
> When we upgrade and restart a new version application which will read and
> write to HBase, we will get some operation timeout. The time out is expected
> because when the application restarts，It will not hold any region locations
> cache and do communication with zk and meta regionserver to get region
> locations.
> We want to avoid these timeouts so we do warmup work and as far as I am
> concerned,the method table.getRegionLocator().getAllRegionLocations() will
> fetch all region locations and cache them. However, it didn't work good.
> There are still a lot of time outs,so it confused me.
> I dig into the source code and find something below
> {code:java}
> // code placeholder
> public List getAllRegionLocations() throws IOException {
> TableName tableName = getName();
> NavigableMap locations =
> MetaScanner.allTableRegions(this.connection, tableName);
> ArrayList regions = new ArrayList<>(locations.size());
> for (Entry entry : locations.entrySet()) {
> regions.add(new HRegionLocation(entry.getKey(), entry.getValue()));
> }
> if (regions.size() > 0) {
> connection.cacheLocation(tableName, new RegionLocations(regions));
> }
> return regions;
> }
> In MetaCache
> public void cacheLocation(final TableName tableName, final RegionLocations
> locations) {
> byte [] startKey =
> locations.getRegionLocation().getRegionInfo().getStartKey();
> ConcurrentMap tableLocations =
> getTableLocations(tableName);
> RegionLocations oldLocation = tableLocations.putIfAbsent(startKey,
> locations);
> boolean isNewCacheEntry = (oldLocation == null);
> if (isNewCacheEntry) {
> if (LOG.isTraceEnabled()) {
> LOG.trace("Cached location: " + locations);
> }
> addToCachedServers(locations);
> return;
> }
> {code}
> It will collect all regions into one RegionLocations object and only cache
> the first not null region location and then when we put or get to hbase, we
> do getCacheLocation()
> {code:java}
> // code placeholder
> public RegionLocations getCachedLocation(final TableName tableName, final
> byte [] row) {
> ConcurrentNavigableMap tableLocations =
> getTableLocations(tableName);
> Entry e = tableLocations.floorEntry(row);
> if (e == null) {
> if (metrics!= null) metrics.incrMetaCacheMiss();
> return null;
> }
> RegionLocations possibleRegion = e.getValue();
> // make sure that the end key is greater than the row we're looking
> // for, otherwise the row actually belongs in the next region, not
> // this one. the exception case is when the endkey is
> // HConstants.EMPTY_END_ROW, signifying that the region we're
> // checking is actually the last region in the table.
> byte[] endKey =
> possibleRegion.getRegionLocation().getRegionInfo().getEndKey();
> if (Bytes.equals(endKey, HConstants.EMPTY_END_ROW) ||
> getRowComparator(tableName).compareRows(
> endKey, 0, endKey.length, row, 0, row.length) > 0) {
> if (metrics != null) metrics.incrMetaCacheHit();
> return possibleRegion;
> }
> // Passed all the way through, so we got nothing - complete cache miss
> if (metrics != null) metrics.incrMetaCacheMiss();
> return null;
> }
> {code}
> It will choose the first location to be possibleRegion and possibly it will
> miss match.
> So did I forget something or may be wrong somewhere? If this is indeed a bug
> I think it can be fixed not very hard.
> Hope commiters and PMC review this !
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20698) Master don't record right server version until new started region server call regionServerReport method
[
https://issues.apache.org/jira/browse/HBASE-20698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505659#comment-16505659
]
Guanghao Zhang commented on HBASE-20698:
Add a 002 patch. Record the server version when region server startup. But it
still has chance to have inconsistent result between ServerManager and
AssignmentManager.
> Master don't record right server version until new started region server call
> regionServerReport method
> ---
>
> Key: HBASE-20698
> URL: https://issues.apache.org/jira/browse/HBASE-20698
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-20698.master.001.patch,
> HBASE-20698.master.002.patch
>
>
> When a new region server started, it will call regionServerStartup first.
> Master will record this server as a new online server and may dispath
> RemoteProcedure to the new server. But master only record the server version
> when the new region server call regionServerReport method. Dispatch a new
> RemoteProcedure to this new regionserver will fail if version is not right.
> {code:java}
> @Override
> protected void remoteDispatch(final ServerName serverName,
> final Set remoteProcedures) {
> final int rsVersion =
> master.getAssignmentManager().getServerVersion(serverName);
> if (rsVersion >= RS_VERSION_WITH_EXEC_PROCS) {
> LOG.trace("Using procedure batch rpc execution for serverName={}
> version={}",
> serverName, rsVersion);
> submitTask(new ExecuteProceduresRemoteCall(serverName,
> remoteProcedures));
> } else {
> LOG.info(String.format(
> "Fallback to compat rpc execution for serverName=%s version=%s",
> serverName, rsVersion));
> submitTask(new CompatRemoteProcedureResolver(serverName,
> remoteProcedures));
> }
> }
> {code}
> The above code use version to resolve compatibility problem. So dispatch will
> work right for old version region server. But for RefreshPeerProcedure, it is
> new since hbase 2.0. So RefreshPeerProcedure don't need this. But the new
> region server version is not right, it will use CompatRemoteProcedureResolver
> for RefreshPeerProcedure, too. So the RefreshPeerProcedure can't be executed
> rightly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20698) Master don't record right server version until new started region server call regionServerReport method
[
https://issues.apache.org/jira/browse/HBASE-20698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-20698:
---
Attachment: HBASE-20698.master.002.patch
> Master don't record right server version until new started region server call
> regionServerReport method
> ---
>
> Key: HBASE-20698
> URL: https://issues.apache.org/jira/browse/HBASE-20698
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-20698.master.001.patch,
> HBASE-20698.master.002.patch
>
>
> When a new region server started, it will call regionServerStartup first.
> Master will record this server as a new online server and may dispath
> RemoteProcedure to the new server. But master only record the server version
> when the new region server call regionServerReport method. Dispatch a new
> RemoteProcedure to this new regionserver will fail if version is not right.
> {code:java}
> @Override
> protected void remoteDispatch(final ServerName serverName,
> final Set remoteProcedures) {
> final int rsVersion =
> master.getAssignmentManager().getServerVersion(serverName);
> if (rsVersion >= RS_VERSION_WITH_EXEC_PROCS) {
> LOG.trace("Using procedure batch rpc execution for serverName={}
> version={}",
> serverName, rsVersion);
> submitTask(new ExecuteProceduresRemoteCall(serverName,
> remoteProcedures));
> } else {
> LOG.info(String.format(
> "Fallback to compat rpc execution for serverName=%s version=%s",
> serverName, rsVersion));
> submitTask(new CompatRemoteProcedureResolver(serverName,
> remoteProcedures));
> }
> }
> {code}
> The above code use version to resolve compatibility problem. So dispatch will
> work right for old version region server. But for RefreshPeerProcedure, it is
> new since hbase 2.0. So RefreshPeerProcedure don't need this. But the new
> region server version is not right, it will use CompatRemoteProcedureResolver
> for RefreshPeerProcedure, too. So the RefreshPeerProcedure can't be executed
> rightly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20700) Move meta region when server crash can cause the procedure to be stuck
[ https://issues.apache.org/jira/browse/HBASE-20700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505654#comment-16505654 ] Duo Zhang commented on HBASE-20700: --- Oh for meta there is another problem... The RecoverMetaProcedure will hold the exclusive lock for the meta table, and since the MRP for meta will hold the shared lock on meta table so the RecoverMetaProcedure can not be executed... This is not correct I believe. In SCP we do not hold any table/region lock so that we are free to execute and then we can fail other RIT procedures to let our assign procedures go. For me, the RecoverMetaProcedure is a bit strange. In general, if an RS is crashed then we will have a SCP for it and if it carries meta then we will assign meta somewhere else finally. When master start up, we just need to wait until the RS is online and do not need to mess up the recovery processing... > Move meta region when server crash can cause the procedure to be stuck > -- > > Key: HBASE-20700 > URL: https://issues.apache.org/jira/browse/HBASE-20700 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20700-UT.patch > > > As said in HBASE-20682. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20691) Storage policy should allow deferring to HDFS
[
https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505641#comment-16505641
]
Yu Li commented on HBASE-20691:
---
Thanks all for the comments.
bq. one to control whether we overwrite the hdfs storage policy
I guess we could assume user wants to overwrite the hdfs storage policy if they
explicitly set {{hbase.wal.storage.policy}} (the action itself indicates the
purpose)? And backward (configuration) compatibility is also one consideration
as [~busbey] mentioned.
The v2 patch addresses some review comments, please check and let me know your
thoughts. Thanks!
> Storage policy should allow deferring to HDFS
> -
>
> Key: HBASE-20691
> URL: https://issues.apache.org/jira/browse/HBASE-20691
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration, wal
>Affects Versions: 1.5.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Yu Li
>Priority: Minor
> Fix For: 2.1.0, 1.5.0
>
> Attachments: HBASE-20691.patch, HBASE-20691.v2.patch
>
>
> In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using
> "NONE" as the storage policy in hbase configs.
> As described on this [dev@hbase thread "WAL storage policies and interactions
> with Hadoop admin
> tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb2237@%3Cdev.hbase.apache.org%3E]
> we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now).
> Additionally, we can't set the policy to HOT in the event that HDFS has
> changed the policy for a parent directory of our WALs.
> We should put back that ability. Presuming this is done by re-adopting the
> "NONE" placeholder variable, we need to ensure that value doesn't get passed
> to HDFS APIs. Since it isn't a valid storage policy attempting to use it will
> cause a bunch of logging churn (which will be a regression of the problem
> HBASE-18118 sought to fix).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20691) Storage policy should allow deferring to HDFS
[ https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-20691: -- Attachment: HBASE-20691.v2.patch > Storage policy should allow deferring to HDFS > - > > Key: HBASE-20691 > URL: https://issues.apache.org/jira/browse/HBASE-20691 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Affects Versions: 1.5.0, 2.0.0 >Reporter: Sean Busbey >Assignee: Yu Li >Priority: Minor > Fix For: 2.1.0, 1.5.0 > > Attachments: HBASE-20691.patch, HBASE-20691.v2.patch > > > In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using > "NONE" as the storage policy in hbase configs. > As described on this [dev@hbase thread "WAL storage policies and interactions > with Hadoop admin > tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb2237@%3Cdev.hbase.apache.org%3E] > we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now). > Additionally, we can't set the policy to HOT in the event that HDFS has > changed the policy for a parent directory of our WALs. > We should put back that ability. Presuming this is done by re-adopting the > "NONE" placeholder variable, we need to ensure that value doesn't get passed > to HDFS APIs. Since it isn't a valid storage policy attempting to use it will > cause a bunch of logging churn (which will be a regression of the problem > HBASE-18118 sought to fix). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19064) Synchronous replication for HBase
[
https://issues.apache.org/jira/browse/HBASE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505634#comment-16505634
]
Hudson commented on HBASE-19064:
Results for branch HBASE-19064
[build #155 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/155/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/155//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/155//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/155//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Synchronous replication for HBase
> -
>
> Key: HBASE-19064
> URL: https://issues.apache.org/jira/browse/HBASE-19064
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> The guys from Alibaba made a presentation on HBaseCon Asia about the
> synchronous replication for HBase. We(Xiaomi) think this is a very useful
> feature for HBase so we want to bring it into the community version.
> This is a big feature so we plan to do it in a feature branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-13583) AOT compile our JRuby
[
https://issues.apache.org/jira/browse/HBASE-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505618#comment-16505618
]
Hadoop QA commented on HBASE-13583:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 3s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
52s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-13583 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12927003/HBASE-13583.v2.patch |
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux df032c26a3c7 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13152/testReport/ |
| Max. process+thread count | 2112 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13152/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> AOT compile our JRuby
> -
>
> Key: HBASE-13583
> URL: https://issues.apache.org/jira/browse/HBASE-13583
> Project: HBase
> Issue Type: Improvement
> Components: build, scripts, shell
>Reporter: Nick Dimiduk
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-13583.patch, HBASE-13583.v2.patch
>
>
> Our Jruby code seems to not keep up well with Java changes. We should
>
[jira] [Commented] (HBASE-20696) Shell list_peers print useless string
[
https://issues.apache.org/jira/browse/HBASE-20696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505605#comment-16505605
]
Zheng Hu commented on HBASE-20696:
--
Got it , Thanks [~busbey]
> Shell list_peers print useless string
> --
>
> Key: HBASE-20696
> URL: https://issues.apache.org/jira/browse/HBASE-20696
> Project: HBase
> Issue Type: Bug
>Reporter: Zheng Hu
>Priority: Major
>
> {code}
> hbase(main):020:0> list_peers
> PEER_ID CLUSTER_KEY ENDPOINT_CLASSNAME REMOTE_ROOT_DIR
> SYNC_REPLICATION_STATE STATE REPLICATE_ALL NAMESPACES TABLE_CFS BANDWIDTH
> SERIAL
> 1
> lg-hadoop-tst-st01.bj:10010,lg-hadoop-tst-st02.bj:10010,lg-hadoop-tst-st03.bj:10010:/hbase/test-hbase-slave
> nil hdfs://lg-hadoop-tst-st01.bj:20100/hbase/test-hbase-slave/remoteWALs
> ACTIVE ENABLED false default.ycsb-test 0 false
> 1 row(s)
> Took 0.0446 seconds
>
> => #> It's useless ..
> {code}
> Interested contributors are welcome to fix this bug...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-13583) AOT compile our JRuby
[ https://issues.apache.org/jira/browse/HBASE-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-13583: -- Status: Patch Available (was: Open) > AOT compile our JRuby > - > > Key: HBASE-13583 > URL: https://issues.apache.org/jira/browse/HBASE-13583 > Project: HBase > Issue Type: Improvement > Components: build, scripts, shell >Reporter: Nick Dimiduk >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-13583.patch, HBASE-13583.v2.patch > > > Our Jruby code seems to not keep up well with Java changes. We should > investigate adding a compilation step for our shell and the rb scripts in bin > to ensure they're calling methods that exist on classes that exist. This > looks like as good a starting point as any: > https://github.com/jruby/jruby/wiki/GeneratingJavaClasses -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-13583) AOT compile our JRuby
[
https://issues.apache.org/jira/browse/HBASE-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505600#comment-16505600
]
Mike Drob commented on HBASE-13583:
---
v2: get something that actually works! manually verified that there are class
files corresponding to the ruby source in our hbase-shell jar. We end up
including both the .rb and .class files, which might be ok for now? Can circle
back to that later.
It's actually much worse than what you suggested, [~busbey].
Checked the classpath with {{mvn -X}} and saw that we have asm 3.1 in use. I
tried using a newer version of asm both asm:asm:3.3.1 (the latest version in
the asm:asm coordinate space) and org.ow2.asm:asm-all:5.0.4 (the version jruby
is using) and they fail with:
{noformat}
[INFO] TypeError: failed to coerce org.objectweb.asm.ClassWriter to
org.jruby.org.objectweb.asm.ClassVisitor
[INFO] block in compile_files_with_options at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:189
[INFO] block in compile_files_with_options at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:291
[INFO] each at org/jruby/RubyArray.java:1734
[INFO] block in compile_files_with_options at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:290
[INFO] each at org/jruby/RubyArray.java:1734
[INFO]compile_files_with_options at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:281
[INFO] compile_argv at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:94
[INFO] at -e:3
{noformat}
which led me to https://github.com/jruby/jruby/issues/2887 that suggest using
shaded and unshaded asm in the same classpath will lead to a world of trouble.
So... I looked at the dependency tree and noted that we're pulling asm 3.1 via
jersey-server via a bunch of hadoop libs, so we can exclude that in dependency
management. And in hadoop-3, we're pulling in a different asm, so out that goes
as well. Should file another JIRA to see what else we can prune from that
dependency list, because it looks really long and full of hadoop stuff that I'm
sure we don't need. Removing MR entries for now since those obviously don't
belong, but leaving the rest because I don't want too invasive of a change.
> AOT compile our JRuby
> -
>
> Key: HBASE-13583
> URL: https://issues.apache.org/jira/browse/HBASE-13583
> Project: HBase
> Issue Type: Improvement
> Components: build, scripts, shell
>Reporter: Nick Dimiduk
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-13583.patch, HBASE-13583.v2.patch
>
>
> Our Jruby code seems to not keep up well with Java changes. We should
> investigate adding a compilation step for our shell and the rb scripts in bin
> to ensure they're calling methods that exist on classes that exist. This
> looks like as good a starting point as any:
> https://github.com/jruby/jruby/wiki/GeneratingJavaClasses
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-13583) AOT compile our JRuby
[ https://issues.apache.org/jira/browse/HBASE-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-13583: -- Attachment: HBASE-13583.v2.patch > AOT compile our JRuby > - > > Key: HBASE-13583 > URL: https://issues.apache.org/jira/browse/HBASE-13583 > Project: HBase > Issue Type: Improvement > Components: build, scripts, shell >Reporter: Nick Dimiduk >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-13583.patch, HBASE-13583.v2.patch > > > Our Jruby code seems to not keep up well with Java changes. We should > investigate adding a compilation step for our shell and the rb scripts in bin > to ensure they're calling methods that exist on classes that exist. This > looks like as good a starting point as any: > https://github.com/jruby/jruby/wiki/GeneratingJavaClasses -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[ https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-20704: Status: Patch Available (was: Open) Updated patch to fix small typo in the unit test > Sometimes some compacted storefiles are not archived on region close > > > Key: HBASE-20704 > URL: https://issues.apache.org/jira/browse/HBASE-20704 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 2.0.0, 1.4.0, 1.3.0, 3.0.0, 1.5.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch > > > During region close compacted files which have not yet been archived by the > discharger are archived as part of the region closing process. It is > important that these files are wholly archived to insure data consistency. ie > a storefile containing delete tombstones can be archived while older > storefiles containing cells that were supposed to be deleted are left > unarchived thereby undeleting those cells. > On region close a compacted storefile is skipped from archiving if it has > read references (ie open scanners). This behavior is correct for when the > discharger chore runs but on region close consistency is of course more > important so we should add a special case to ignore any references on the > storefile and go ahead and archive it. > Attached patch contains a unit test that reproduces the problem and the > proposed fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[ https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-20704: Attachment: HBASE-20704.002.patch > Sometimes some compacted storefiles are not archived on region close > > > Key: HBASE-20704 > URL: https://issues.apache.org/jira/browse/HBASE-20704 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch > > > During region close compacted files which have not yet been archived by the > discharger are archived as part of the region closing process. It is > important that these files are wholly archived to insure data consistency. ie > a storefile containing delete tombstones can be archived while older > storefiles containing cells that were supposed to be deleted are left > unarchived thereby undeleting those cells. > On region close a compacted storefile is skipped from archiving if it has > read references (ie open scanners). This behavior is correct for when the > discharger chore runs but on region close consistency is of course more > important so we should add a special case to ignore any references on the > storefile and go ahead and archive it. > Attached patch contains a unit test that reproduces the problem and the > proposed fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[
https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505588#comment-16505588
]
Hadoop QA commented on HBASE-20672:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
21s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
59s{color} | {color:red} hbase-server: The patch generated 2 new + 3 unchanged
- 0 fixed = 5 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
17s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 9s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
31s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 34m 58s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 78m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.ipc.TestSimpleRpcScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20672 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926987/HBASE-20672.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 5ee9ff0808ba 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
|
[jira] [Commented] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[
https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505585#comment-16505585
]
Allan Yang commented on HBASE-20672:
{code}
+ readRequestsRatePerSecond = readRequestsRatePerMilliSecond / 1000.0;
+ writeRequestsRatePerSecond = writeRequestsRatePerMilliSecond/ 1000.0;
{code}
Shouldn't be RatePerSecond = RatePerMilliSecond *1000 ?
> Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at
> every monitoring interval
> -
>
> Key: HBASE-20672
> URL: https://issues.apache.org/jira/browse/HBASE-20672
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Ankit Jain
>Assignee: Ankit Jain
>Priority: Minor
> Attachments: HBASE-20672.branch-1.001.patch,
> HBASE-20672.master.001.patch, HBASE-20672.master.002.patch
>
>
> Hbase currently provides counter read/write requests (ReadRequestCount,
> WriteRequestCount). That said it is not easy to use counter that reset only
> after a restart of the service, we would like to expose 2 new metrics in
> HBase to provide ReadRequestRate and WriteRequestRate at region server level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-13583) AOT compile our JRuby
[ https://issues.apache.org/jira/browse/HBASE-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505570#comment-16505570 ] Sean Busbey commented on HBASE-13583: - that's a problem with the version of [ASM|http://asm.ow2.io/] that must be on the classpath during the maven invocation. It looks like the [Opcodes class|http://asm.ow2.io/javadoc/org/objectweb/asm/Opcodes.html] in your classpath is limited to JDK6 and earlier, but JRuby is expecting 1.7+. Try adding it as a dependency to the jruby-maven-plugin plugin declaration. > AOT compile our JRuby > - > > Key: HBASE-13583 > URL: https://issues.apache.org/jira/browse/HBASE-13583 > Project: HBase > Issue Type: Improvement > Components: build, scripts, shell >Reporter: Nick Dimiduk >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-13583.patch > > > Our Jruby code seems to not keep up well with Java changes. We should > investigate adding a compilation step for our shell and the rb scripts in bin > to ensure they're calling methods that exist on classes that exist. This > looks like as good a starting point as any: > https://github.com/jruby/jruby/wiki/GeneratingJavaClasses -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[
https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505566#comment-16505566
]
Hadoop QA commented on HBASE-20672:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
55s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hbase-hadoop2-compat: The patch generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
9s{color} | {color:red} hbase-server: The patch generated 1 new + 3 unchanged -
0 fixed = 4 total (was 3) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 50s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
0s{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed =
1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
25s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}110m 54s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
57s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Dead store to intervalWriteRequestsCount in
org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl$RegionServerMetricsWrapperRunnable.run()
At
MetricsRegionServerWrapperImpl.java:org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl$RegionServerMetricsWrapperRunnable.run()
At
[jira] [Commented] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505556#comment-16505556
]
Hadoop QA commented on HBASE-19852:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} hbase-thrift: The patch generated 0 new + 0
unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
50s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 53s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
0s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-19852 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926986/HBASE-19852.master.012.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile findbugs hbaseanti checkstyle |
| uname | Linux 62925ab86893 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13149/testReport/ |
| Max. process+thread count | 1414 (vs. ulimit of 1) |
| modules | C: hbase-thrift U: hbase-thrift |
| Console
[jira] [Commented] (HBASE-20700) Move meta region when server crash can cause the procedure to be stuck
[ https://issues.apache.org/jira/browse/HBASE-20700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505537#comment-16505537 ] Duo Zhang commented on HBASE-20700: --- I could do it. And I plan to add more comments in MRP and SCP to say why the current approach works, for example which operation is the fencing point. Thanks. > Move meta region when server crash can cause the procedure to be stuck > -- > > Key: HBASE-20700 > URL: https://issues.apache.org/jira/browse/HBASE-20700 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20700-UT.patch > > > As said in HBASE-20682. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20682) MoveProcedure can be subtask of ModifyTableProcedure/ReopenTableRegionsProcedure; ensure all kosher.
[ https://issues.apache.org/jira/browse/HBASE-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505536#comment-16505536 ] Duo Zhang commented on HBASE-20682: --- Yeah, the checks in MRP may not be suitable for reopening a region. We could implement another procedure for reopening a region, but what I want to say is that, changing the RS side code maybe too much for at least 2.0. The new procedure could still be constructed with an unassign followed by an assign. What do you folks think? > MoveProcedure can be subtask of > ModifyTableProcedure/ReopenTableRegionsProcedure; ensure all kosher. > > > Key: HBASE-20682 > URL: https://issues.apache.org/jira/browse/HBASE-20682 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.1 > > > MoveProcedure is used in at least two places as a means of 'reopening' a > region after a config has changed. This issue is about review of MP so this > usage is first-class (and that MP is good procedure citizen running as a > subprocedure. In question is what should happen if the source server or the > target servers are offline when we go to run. As of the parent issue, we'll > skip to the end. Should we instead go ahead with the move (internally, if we > are asked to assign to an offline server, we'll pick another -- do same for > unassign? Would help in this reopen use case). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[ https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505463#comment-16505463 ] Ted Yu commented on HBASE-20672: lgtm > Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at > every monitoring interval > - > > Key: HBASE-20672 > URL: https://issues.apache.org/jira/browse/HBASE-20672 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > Attachments: HBASE-20672.branch-1.001.patch, > HBASE-20672.master.001.patch, HBASE-20672.master.002.patch > > > Hbase currently provides counter read/write requests (ReadRequestCount, > WriteRequestCount). That said it is not easy to use counter that reset only > after a restart of the service, we would like to expose 2 new metrics in > HBase to provide ReadRequestRate and WriteRequestRate at region server level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[ https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505451#comment-16505451 ] Ankit Jain edited comment on HBASE-20672 at 6/7/18 11:13 PM: - Updated the patch as per the comments. HBASE-20672.master.002.patch was (Author: jain.ankit): Update the patch as per the comments. HBASE-20672.master.002.patch > Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at > every monitoring interval > - > > Key: HBASE-20672 > URL: https://issues.apache.org/jira/browse/HBASE-20672 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > Attachments: HBASE-20672.branch-1.001.patch, > HBASE-20672.master.001.patch, HBASE-20672.master.002.patch > > > Hbase currently provides counter read/write requests (ReadRequestCount, > WriteRequestCount). That said it is not easy to use counter that reset only > after a restart of the service, we would like to expose 2 new metrics in > HBase to provide ReadRequestRate and WriteRequestRate at region server level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[ https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505451#comment-16505451 ] Ankit Jain commented on HBASE-20672: Update the patch as per the comments. HBASE-20672.master.002.patch > Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at > every monitoring interval > - > > Key: HBASE-20672 > URL: https://issues.apache.org/jira/browse/HBASE-20672 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > Attachments: HBASE-20672.branch-1.001.patch, > HBASE-20672.master.001.patch, HBASE-20672.master.002.patch > > > Hbase currently provides counter read/write requests (ReadRequestCount, > WriteRequestCount). That said it is not easy to use counter that reset only > after a restart of the service, we would like to expose 2 new metrics in > HBase to provide ReadRequestRate and WriteRequestRate at region server level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[ https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Jain updated HBASE-20672: --- Attachment: HBASE-20672.master.002.patch > Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at > every monitoring interval > - > > Key: HBASE-20672 > URL: https://issues.apache.org/jira/browse/HBASE-20672 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > Attachments: HBASE-20672.branch-1.001.patch, > HBASE-20672.master.001.patch, HBASE-20672.master.002.patch > > > Hbase currently provides counter read/write requests (ReadRequestCount, > WriteRequestCount). That said it is not easy to use counter that reset only > after a restart of the service, we would like to expose 2 new metrics in > HBase to provide ReadRequestRate and WriteRequestRate at region server level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505449#comment-16505449
]
Kevin Risden commented on HBASE-19852:
--
Uploaded a patch that randomizes the infoport option for the Thrift HTTP tests.
> HBase Thrift 1 server SPNEGO Improvements
> -
>
> Key: HBASE-19852
> URL: https://issues.apache.org/jira/browse/HBASE-19852
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Major
> Attachments: HBASE-19852.master.001.patch,
> HBASE-19852.master.002.patch, HBASE-19852.master.003.patch,
> HBASE-19852.master.004.patch, HBASE-19852.master.006.patch,
> HBASE-19852.master.007.patch.txt, HBASE-19852.master.008.patch,
> HBASE-19852.master.009.patch, HBASE-19852.master.010.patch,
> HBASE-19852.master.011.patch, HBASE-19852.master.012.patch
>
>
> HBase Thrift1 server has some issues when trying to use SPNEGO.
> From mailing list:
> http://mail-archives.apache.org/mod_mbox/hbase-user/201801.mbox/%3CCAJU9nmh5YtZ%2BmAQSLo91yKm8pRVzAPNLBU9vdVMCcxHRtRqgoA%40mail.gmail.com%3E
> {quote}While setting up the HBase Thrift server with HTTP, there were a
> significant amount of 401 errors where the HBase Thrift wasn't able to
> handle the incoming Kerberos request. Documentation online is sparse when
> it comes to setting up the principal/keytab for HTTP Kerberos.
> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
> principal/keytab like other Thrift based servers (HiveServer2). It looks
> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
> were very close to the same at one point. I made the following changes to
> HBase Thrift v1 server implementation to make it work:
> * add SPNEGO principal/keytab if in HTTP mode
> * return 401 immediately if no authorization header instead of waiting for
> try/catch down in program flow{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Risden updated HBASE-19852:
-
Attachment: HBASE-19852.master.012.patch
> HBase Thrift 1 server SPNEGO Improvements
> -
>
> Key: HBASE-19852
> URL: https://issues.apache.org/jira/browse/HBASE-19852
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Major
> Attachments: HBASE-19852.master.001.patch,
> HBASE-19852.master.002.patch, HBASE-19852.master.003.patch,
> HBASE-19852.master.004.patch, HBASE-19852.master.006.patch,
> HBASE-19852.master.007.patch.txt, HBASE-19852.master.008.patch,
> HBASE-19852.master.009.patch, HBASE-19852.master.010.patch,
> HBASE-19852.master.011.patch, HBASE-19852.master.012.patch
>
>
> HBase Thrift1 server has some issues when trying to use SPNEGO.
> From mailing list:
> http://mail-archives.apache.org/mod_mbox/hbase-user/201801.mbox/%3CCAJU9nmh5YtZ%2BmAQSLo91yKm8pRVzAPNLBU9vdVMCcxHRtRqgoA%40mail.gmail.com%3E
> {quote}While setting up the HBase Thrift server with HTTP, there were a
> significant amount of 401 errors where the HBase Thrift wasn't able to
> handle the incoming Kerberos request. Documentation online is sparse when
> it comes to setting up the principal/keytab for HTTP Kerberos.
> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
> principal/keytab like other Thrift based servers (HiveServer2). It looks
> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
> were very close to the same at one point. I made the following changes to
> HBase Thrift v1 server implementation to make it work:
> * add SPNEGO principal/keytab if in HTTP mode
> * return 401 immediately if no authorization header instead of waiting for
> try/catch down in program flow{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505438#comment-16505438
]
Kevin Risden commented on HBASE-19852:
--
Hmmm the error seems to be real where two tests ran at the same time and both
tried to grab the same info port for Thrift. I can't find anything that either
a) disabled the info server or b) randomizes the info port.
> HBase Thrift 1 server SPNEGO Improvements
> -
>
> Key: HBASE-19852
> URL: https://issues.apache.org/jira/browse/HBASE-19852
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Major
> Attachments: HBASE-19852.master.001.patch,
> HBASE-19852.master.002.patch, HBASE-19852.master.003.patch,
> HBASE-19852.master.004.patch, HBASE-19852.master.006.patch,
> HBASE-19852.master.007.patch.txt, HBASE-19852.master.008.patch,
> HBASE-19852.master.009.patch, HBASE-19852.master.010.patch,
> HBASE-19852.master.011.patch
>
>
> HBase Thrift1 server has some issues when trying to use SPNEGO.
> From mailing list:
> http://mail-archives.apache.org/mod_mbox/hbase-user/201801.mbox/%3CCAJU9nmh5YtZ%2BmAQSLo91yKm8pRVzAPNLBU9vdVMCcxHRtRqgoA%40mail.gmail.com%3E
> {quote}While setting up the HBase Thrift server with HTTP, there were a
> significant amount of 401 errors where the HBase Thrift wasn't able to
> handle the incoming Kerberos request. Documentation online is sparse when
> it comes to setting up the principal/keytab for HTTP Kerberos.
> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
> principal/keytab like other Thrift based servers (HiveServer2). It looks
> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
> were very close to the same at one point. I made the following changes to
> HBase Thrift v1 server implementation to make it work:
> * add SPNEGO principal/keytab if in HTTP mode
> * return 401 immediately if no authorization header instead of waiting for
> try/catch down in program flow{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20702) Processing crash, skip ONLINE'ing empty rows
[
https://issues.apache.org/jira/browse/HBASE-20702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505435#comment-16505435
]
Hudson commented on HBASE-20702:
Results for branch branch-2
[build #835 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Processing crash, skip ONLINE'ing empty rows
>
>
> Key: HBASE-20702
> URL: https://issues.apache.org/jira/browse/HBASE-20702
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: HBASE-20702.branch-2.0.001.patch
>
>
> This patch comes from the parent issue. Parent issue identifies us ONLINE'ing
> a region though it has nothing in the row (in parent issue scenario, region
> info family was deleted in a merge region parent). We shouldn't do this.
> Committing patch from parent here in this subtask since the parent issue is
> still under investigation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20665) "Already cached block XXX" message should be DEBUG
[
https://issues.apache.org/jira/browse/HBASE-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505433#comment-16505433
]
Hudson commented on HBASE-20665:
Results for branch branch-2
[build #835 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> "Already cached block XXX" message should be DEBUG
> --
>
> Key: HBASE-20665
> URL: https://issues.apache.org/jira/browse/HBASE-20665
> Project: HBase
> Issue Type: Task
> Components: BlockCache
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Eric Maynard
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.1.0, 1.2.7, 1.3.3, 2.0.1
>
> Attachments: HBASE-20665.master.001.patch,
> HBASE-20665.master.001.patch
>
>
> Testing a local cluster that relies on the LruBlockCache for a scan-heavy
> workload and I'm getting a bunch of log entries at WARN
> {code}
> 2018-05-30 12:28:47,192 WARN org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Cached an already cached block: df01f5bf6a244f6bb1a626b927377fff_54780812
> cb:df01f5bf6a244f6bb1a626b927377fff_54780812. This is harmless and can happen
> in rare cases (see HBASE-8547)
> {code}
> As the log message notes (and the code confirms) this is a harmless result of
> contention for getting a block into the CHM. the message should be at DEBUG.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-8547) Fix java.lang.RuntimeException: Cached an already cached block
[
https://issues.apache.org/jira/browse/HBASE-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505434#comment-16505434
]
Hudson commented on HBASE-8547:
---
Results for branch branch-2
[build #835 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/835//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix java.lang.RuntimeException: Cached an already cached block
> --
>
> Key: HBASE-8547
> URL: https://issues.apache.org/jira/browse/HBASE-8547
> Project: HBase
> Issue Type: Bug
> Components: io, regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8547_v1-0.94.patch, hbase-8547_v1-0.94.patch,
> hbase-8547_v1.patch, hbase-8547_v2-0.94-reduced.patch,
> hbase-8547_v2-addendum2+3-0.94.patch, hbase-8547_v2-addendum2.patch,
> hbase-8547_v2-addendum2.patch, hbase-8547_v2-addendum3.patch,
> hbase-8547_v2-trunk.patch
>
>
> In one test, one of the region servers received the following on 0.94.
> Note HalfStoreFileReader in the stack trace. I think the root cause is that
> after the region is split, the mid point can be in the middle of the block
> (for store files that the mid point is not chosen from). Each half store file
> tries to load the half block and put it in the block cache. Since IdLock is
> instantiated per store file reader, they do not share the same IdLock
> instance, thus does not lock against each other effectively.
> {code}
> 2013-05-12 01:30:37,733 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:·
> java.lang.RuntimeException: Cached an already cached block
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:279)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:353)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:480)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:501)
> at
> org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:237)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:351)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:354)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:312)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:277)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:543)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:411)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3829)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3778)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3770)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2643)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> {code}
> I can see two possible fixes:
> #
[jira] [Commented] (HBASE-13583) AOT compile our JRuby
[
https://issues.apache.org/jira/browse/HBASE-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505430#comment-16505430
]
Mike Drob commented on HBASE-13583:
---
Running {{java -jar
~/.m2/repository/org/jruby/jruby-complete/9.1.13.0/jruby-complete-9.1.13.0.jar
-S jrubyc hbase-shell/src/main/ruby -t /tmp/}} worked great, so I have some
hope here.
Patch in current state fails due to arcane error:
{noformat}
[INFO] NameError: uninitialized constant Java::OrgObjectwebAsm::Opcodes::V1_7
[INFO] Did you mean? Java::OrgObjectwebAsm::Opcodes::V1_4
[INFO]Java::OrgObjectwebAsm::Opcodes::V1_5
[INFO]Java::OrgObjectwebAsm::Opcodes::V1_6
[INFO]Java::OrgObjectwebAsm::Opcodes::V1_1
[INFO]Java::OrgObjectwebAsm::Opcodes::V1_2
[INFO]Java::OrgObjectwebAsm::Opcodes::V1_3
[INFO] const_missing at org/jruby/RubyModule.java:3343
[INFO] block in compile_files_with_options at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:171
[INFO] block in compile_files_with_options at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:291
[INFO] each at org/jruby/RubyArray.java:1734
[INFO] block in compile_files_with_options at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:290
[INFO] each at org/jruby/RubyArray.java:1734
[INFO]compile_files_with_options at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:281
[INFO] compile_argv at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby/compiler.rb:94
[INFO] at -e:3
{noformat}
Not sure what it expects from us, but in accordance with Yonik's Law of
Patches, here it is.
> AOT compile our JRuby
> -
>
> Key: HBASE-13583
> URL: https://issues.apache.org/jira/browse/HBASE-13583
> Project: HBase
> Issue Type: Improvement
> Components: build, scripts, shell
>Reporter: Nick Dimiduk
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-13583.patch
>
>
> Our Jruby code seems to not keep up well with Java changes. We should
> investigate adding a compilation step for our shell and the rb scripts in bin
> to ensure they're calling methods that exist on classes that exist. This
> looks like as good a starting point as any:
> https://github.com/jruby/jruby/wiki/GeneratingJavaClasses
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-8547) Fix java.lang.RuntimeException: Cached an already cached block
[
https://issues.apache.org/jira/browse/HBASE-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505427#comment-16505427
]
Hudson commented on HBASE-8547:
---
Results for branch branch-2.0
[build #400 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix java.lang.RuntimeException: Cached an already cached block
> --
>
> Key: HBASE-8547
> URL: https://issues.apache.org/jira/browse/HBASE-8547
> Project: HBase
> Issue Type: Bug
> Components: io, regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8547_v1-0.94.patch, hbase-8547_v1-0.94.patch,
> hbase-8547_v1.patch, hbase-8547_v2-0.94-reduced.patch,
> hbase-8547_v2-addendum2+3-0.94.patch, hbase-8547_v2-addendum2.patch,
> hbase-8547_v2-addendum2.patch, hbase-8547_v2-addendum3.patch,
> hbase-8547_v2-trunk.patch
>
>
> In one test, one of the region servers received the following on 0.94.
> Note HalfStoreFileReader in the stack trace. I think the root cause is that
> after the region is split, the mid point can be in the middle of the block
> (for store files that the mid point is not chosen from). Each half store file
> tries to load the half block and put it in the block cache. Since IdLock is
> instantiated per store file reader, they do not share the same IdLock
> instance, thus does not lock against each other effectively.
> {code}
> 2013-05-12 01:30:37,733 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:·
> java.lang.RuntimeException: Cached an already cached block
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:279)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:353)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:480)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:501)
> at
> org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:237)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:351)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:354)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:312)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:277)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:543)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:411)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3829)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3778)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3770)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2643)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> {code}
> I can see two possible fixes:
[jira] [Commented] (HBASE-20665) "Already cached block XXX" message should be DEBUG
[
https://issues.apache.org/jira/browse/HBASE-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505426#comment-16505426
]
Hudson commented on HBASE-20665:
Results for branch branch-2.0
[build #400 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> "Already cached block XXX" message should be DEBUG
> --
>
> Key: HBASE-20665
> URL: https://issues.apache.org/jira/browse/HBASE-20665
> Project: HBase
> Issue Type: Task
> Components: BlockCache
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Eric Maynard
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.1.0, 1.2.7, 1.3.3, 2.0.1
>
> Attachments: HBASE-20665.master.001.patch,
> HBASE-20665.master.001.patch
>
>
> Testing a local cluster that relies on the LruBlockCache for a scan-heavy
> workload and I'm getting a bunch of log entries at WARN
> {code}
> 2018-05-30 12:28:47,192 WARN org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Cached an already cached block: df01f5bf6a244f6bb1a626b927377fff_54780812
> cb:df01f5bf6a244f6bb1a626b927377fff_54780812. This is harmless and can happen
> in rare cases (see HBASE-8547)
> {code}
> As the log message notes (and the code confirms) this is a harmless result of
> contention for getting a block into the CHM. the message should be at DEBUG.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20702) Processing crash, skip ONLINE'ing empty rows
[
https://issues.apache.org/jira/browse/HBASE-20702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505428#comment-16505428
]
Hudson commented on HBASE-20702:
Results for branch branch-2.0
[build #400 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/400//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Processing crash, skip ONLINE'ing empty rows
>
>
> Key: HBASE-20702
> URL: https://issues.apache.org/jira/browse/HBASE-20702
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: HBASE-20702.branch-2.0.001.patch
>
>
> This patch comes from the parent issue. Parent issue identifies us ONLINE'ing
> a region though it has nothing in the row (in parent issue scenario, region
> info family was deleted in a merge region parent). We shouldn't do this.
> Committing patch from parent here in this subtask since the parent issue is
> still under investigation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-13583) AOT compile our JRuby
[ https://issues.apache.org/jira/browse/HBASE-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob reassigned HBASE-13583: - Assignee: Mike Drob > AOT compile our JRuby > - > > Key: HBASE-13583 > URL: https://issues.apache.org/jira/browse/HBASE-13583 > Project: HBase > Issue Type: Improvement > Components: build, scripts, shell >Reporter: Nick Dimiduk >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-13583.patch > > > Our Jruby code seems to not keep up well with Java changes. We should > investigate adding a compilation step for our shell and the rb scripts in bin > to ensure they're calling methods that exist on classes that exist. This > looks like as good a starting point as any: > https://github.com/jruby/jruby/wiki/GeneratingJavaClasses -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13583) AOT compile our JRuby
[ https://issues.apache.org/jira/browse/HBASE-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-13583: -- Attachment: HBASE-13583.patch > AOT compile our JRuby > - > > Key: HBASE-13583 > URL: https://issues.apache.org/jira/browse/HBASE-13583 > Project: HBase > Issue Type: Improvement > Components: build, scripts, shell >Reporter: Nick Dimiduk >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-13583.patch > > > Our Jruby code seems to not keep up well with Java changes. We should > investigate adding a compilation step for our shell and the rb scripts in bin > to ensure they're calling methods that exist on classes that exist. This > looks like as good a starting point as any: > https://github.com/jruby/jruby/wiki/GeneratingJavaClasses -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505412#comment-16505412
]
Hadoop QA commented on HBASE-20703:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 28 new + 14 unchanged - 12 fixed =
42 total (was 26) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 4s{color} | {color:orange} The patch generated 19 new + 24 unchanged - 19
fixed = 43 total (was 43) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
50s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20703 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926979/HBASE-20703.master.005.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux ea1b8fb40925 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| rubocop | v0.57.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/13148/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/13148/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13148/testReport/ |
| Max. process+thread count | 2126 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13148/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.004.patch, HBASE-20703.master.005.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
[jira] [Commented] (HBASE-20635) Support to convert the shaded user permission proto to client user permission object
[
https://issues.apache.org/jira/browse/HBASE-20635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505410#comment-16505410
]
Josh Elser commented on HBASE-20635:
v2 looks fine to me too
> Support to convert the shaded user permission proto to client user permission
> object
>
>
> Key: HBASE-20635
> URL: https://issues.apache.org/jira/browse/HBASE-20635
> Project: HBase
> Issue Type: Bug
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Attachments: HBASE-20635.patch, HBASE-20635_v2.patch,
> PHOENIX-4528_5.x-HBase-2.0_v2.patch
>
>
> Currently we have API to build the protobuf UserPermission to client user
> permission in AccessControlUtil but we cannot do the same when we use shaded
> protobufs.
> {noformat}
> /**
>* Converts a user permission proto to a client user permission object.
>*
>* @param proto the protobuf UserPermission
>* @return the converted UserPermission
>*/
> public static UserPermission
> toUserPermission(AccessControlProtos.UserPermission proto) {
> return new UserPermission(proto.getUser().toByteArray(),
> toTablePermission(proto.getPermission()));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20682) MoveProcedure can be subtask of ModifyTableProcedure/ReopenTableRegionsProcedure; ensure all kosher.
[
https://issues.apache.org/jira/browse/HBASE-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505405#comment-16505405
]
Josh Elser commented on HBASE-20682:
Have been talking with Stack in private-chat today, we are running into a nasty
manifestation of this core problem (at least Stack things I am ;))
We have a client who submits a MTP right after HBase is rolling-restarted.
We're not positive how we got here, but we have a few RITs (some in CLOSED,
CLOSING, and OPENING). However, the active HBase Master is stuck in a fast-loop
on MTP trying to execute the final state {{MODIFY_TABLE_REOPEN_ALL_REGIONS}}.
The MTP bails out because it submits the MoveProcedure for each region,
regardless of them being {{OPEN}} or not. This causes the entire list of
subProcs this submits to fail:
{noformat}
2018-06-07 03:21:29,448 WARN [PEWorker-1] procedure.ModifyTableProcedure:
Retriable error trying to modify table=METRIC_RECORD_HOURLY_UUID (in
state=MODIFY_TABLE_REOPEN_ALL_REGIONS)
org.apache.hadoop.hbase.client.DoNotRetryRegionException:
a3dc333606d38aeb6e2ab4b94233cfbc is not OPEN
at
org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure.checkOnline(AbstractStateMachineTableProcedure.java:193)
at
org.apache.hadoop.hbase.master.assignment.MoveRegionProcedure.(MoveRegionProcedure.java:67)
at
org.apache.hadoop.hbase.master.assignment.AssignmentManager.createMoveRegionProcedure(AssignmentManager.java:767)
at
org.apache.hadoop.hbase.master.assignment.AssignmentManager.createReopenProcedures(AssignmentManager.java:705)
at
org.apache.hadoop.hbase.master.procedure.ModifyTableProcedure.executeFromState(ModifyTableProcedure.java:128)
at
org.apache.hadoop.hbase.master.procedure.ModifyTableProcedure.executeFromState(ModifyTableProcedure.java:50)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1472)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1240)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1760)
{noformat}
This essentially bubbles up to PrcoedureExecutor which flags this as
{{reExecute}}, and then re-runs the same thing. So this has the fun side-effect
of just dumping loads of data into pv2 WALs, while this spins hot+fast. While
the lock may be relinquished and re-obtained, I can't seem to get the RITs
assigned via the HBase shell to unstick this.
> MoveProcedure can be subtask of
> ModifyTableProcedure/ReopenTableRegionsProcedure; ensure all kosher.
>
>
> Key: HBASE-20682
> URL: https://issues.apache.org/jira/browse/HBASE-20682
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
>
> MoveProcedure is used in at least two places as a means of 'reopening' a
> region after a config has changed. This issue is about review of MP so this
> usage is first-class (and that MP is good procedure citizen running as a
> subprocedure. In question is what should happen if the source server or the
> target servers are offline when we go to run. As of the parent issue, we'll
> skip to the end. Should we instead go ahead with the move (internally, if we
> are asked to assign to an offline server, we'll pick another -- do same for
> unassign? Would help in this reopen use case).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20682) MoveProcedure can be subtask of ModifyTableProcedure/ReopenTableRegionsProcedure; ensure all kosher.
[ https://issues.apache.org/jira/browse/HBASE-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505401#comment-16505401 ] stack commented on HBASE-20682: --- bq. With the new remote procedure framework we can just introduce a new remote procedure callable and use executeProcedure to run it, so we do not need to add a new method. And yeah we do not have this framework in 2.0... That sounds sweet. Another thought was adding a flag to CloseRegionRequest. The flag would be reopen. If present, the RS would close and then reopen the Region and return in CloseRegionRequest that it had done the reopen. If the RS did not support this functionality, the Procedure running the 'reopen' would follow the close by an open request. bq. But if there is a MRP which is moving the meta region, then the region is offline check will return false and we will stuck there forever... Is this HBASE-20700? Good find. > MoveProcedure can be subtask of > ModifyTableProcedure/ReopenTableRegionsProcedure; ensure all kosher. > > > Key: HBASE-20682 > URL: https://issues.apache.org/jira/browse/HBASE-20682 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.1 > > > MoveProcedure is used in at least two places as a means of 'reopening' a > region after a config has changed. This issue is about review of MP so this > usage is first-class (and that MP is good procedure citizen running as a > subprocedure. In question is what should happen if the source server or the > target servers are offline when we go to run. As of the parent issue, we'll > skip to the end. Should we instead go ahead with the move (internally, if we > are asked to assign to an offline server, we'll pick another -- do same for > unassign? Would help in this reopen use case). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[ https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505397#comment-16505397 ] Xu Cang commented on HBASE-20672: - [~jain.ankit] you can import this format xml to eclipse to help you format code. [https://hbase.apache.org/book.html#_ides] > Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at > every monitoring interval > - > > Key: HBASE-20672 > URL: https://issues.apache.org/jira/browse/HBASE-20672 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > Attachments: HBASE-20672.branch-1.001.patch, > HBASE-20672.master.001.patch > > > Hbase currently provides counter read/write requests (ReadRequestCount, > WriteRequestCount). That said it is not easy to use counter that reset only > after a restart of the service, we would like to expose 2 new metrics in > HBase to provide ReadRequestRate and WriteRequestRate at region server level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20691) Storage policy should allow deferring to HDFS
[ https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505393#comment-16505393 ] Sean Busbey commented on HBASE-20691: - two properties would make it easier for us to avoid messing up the implementation, but isn't very ops friendly given that configuring it only required one property before. I guess I can see either way being fine. > Storage policy should allow deferring to HDFS > - > > Key: HBASE-20691 > URL: https://issues.apache.org/jira/browse/HBASE-20691 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Affects Versions: 1.5.0, 2.0.0 >Reporter: Sean Busbey >Assignee: Yu Li >Priority: Minor > Fix For: 2.1.0, 1.5.0 > > Attachments: HBASE-20691.patch > > > In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using > "NONE" as the storage policy in hbase configs. > As described on this [dev@hbase thread "WAL storage policies and interactions > with Hadoop admin > tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb2237@%3Cdev.hbase.apache.org%3E] > we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now). > Additionally, we can't set the policy to HOT in the event that HDFS has > changed the policy for a parent directory of our WALs. > We should put back that ability. Presuming this is done by re-adopting the > "NONE" placeholder variable, we need to ensure that value doesn't get passed > to HDFS APIs. Since it isn't a valid storage policy attempting to use it will > cause a bunch of logging churn (which will be a regression of the problem > HBASE-18118 sought to fix). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[
https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505391#comment-16505391
]
Ted Yu commented on HBASE-20672:
{code}
+ .addGauge(Interns.info(READ_REQUEST_RATE_PER_SECOND,
READ_REQUEST_RATE_DESC),
+ rsWrap.getReadRequestsRatePerSecond())
+ .addGauge(Interns.info(WRITE_REQUEST_RATE_PER_SECOND,
WRITE_REQUEST_RATE_DESC),
+ rsWrap.getWriteRequestsRatePerSecond());
{code}
Indentation for both metrics are off. Please align with existing code.
{code}
+ writeRequestsRatePerSecond = readRequestsRatePerMilliSecond/ 1000.0;
{code}
I think there is typo in the above assignment. Correspondingly, the computation
for writeRequestsRatePerMilliSecond should be added.
> Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at
> every monitoring interval
> -
>
> Key: HBASE-20672
> URL: https://issues.apache.org/jira/browse/HBASE-20672
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Ankit Jain
>Assignee: Ankit Jain
>Priority: Minor
> Attachments: HBASE-20672.branch-1.001.patch,
> HBASE-20672.master.001.patch
>
>
> Hbase currently provides counter read/write requests (ReadRequestCount,
> WriteRequestCount). That said it is not easy to use counter that reset only
> after a restart of the service, we would like to expose 2 new metrics in
> HBase to provide ReadRequestRate and WriteRequestRate at region server level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20703:
Attachment: HBASE-20703.master.005.patch
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.004.patch, HBASE-20703.master.005.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_snapshot_sizes"'
> Took 1.6285 seconds
>
>
> 2.2.1 :002 > list_quota_snapshots
> TABLE USAGE LIMIT IN_VIOLATION POLICY
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_quota_snapshots"'
> Took 0.0371 seconds
> {code}
> Or it just doesn't mention that quotas can't exist:
> {code}
> 2.2.1 :003 > list_quotas
> OWNERQUOTAS
>
>
> 0 row(s)
> Took 0.0475 seconds
>
>
> 2.2.1 :004 > list_quota_table_sizes
> TABLESIZE
>
>
> 0 row(s)
> Took 0.1221 seconds
> {code}
> set quota gives a better pointer that the problem is the feature is off:
> {code}
> 2.2.1 :005 > set_quota USER => 'busbey', GLOBAL_BYPASS => true
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsupportedOperationException: quota support disabled
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkQuotaSupport(MasterQuotaManager.java:442)
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:124)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1555)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.UnsupportedOperationException: quota support disabled
> ... 8 more
> For usage try 'help "set_quota"'
> {code}
> Instead we should give a nice message, like you get if visibility labels is
> off:
> {code}
> 2.2.1 :06 > list_labels
> ERROR: DISABLED: Visibility labels feature is not available
> For usage try 'help "list_labels"'
> Took 0.0426 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20705) Having RPC Quota on a table prevents Space quota to be recreated/removed
Biju Nair created HBASE-20705:
-
Summary: Having RPC Quota on a table prevents Space quota to be
recreated/removed
Key: HBASE-20705
URL: https://issues.apache.org/jira/browse/HBASE-20705
Project: HBase
Issue Type: Bug
Reporter: Biju Nair
* Property {{hbase.quota.remove.on.table.delete}} is set to {{true}} by default
* Create a table and set RPC and Space quota
{noformat}
hbase(main):022:0> create 't2','cf1'
Created table t2
Took 0.7420 seconds
=> Hbase::Table - t2
hbase(main):023:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
POLICY => NO_WRITES
Took 0.0105 seconds
hbase(main):024:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT => '10M/sec'
Took 0.0186 seconds
hbase(main):025:0> list_quotas
TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT => 10M/sec,
SCOPE => MACHINE
TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824, VIOLATION_POLICY
=> NO_WRITES{noformat}
* Drop the table and the Space quota is set to {{REMOVE => true}}
{noformat}
hbase(main):026:0> disable 't2'
Took 0.4363 seconds
hbase(main):027:0> drop 't2'
Took 0.2344 seconds
hbase(main):028:0> list_quotas
TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE => true
USER => u1 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT => 10M/sec,
SCOPE => MACHINE{noformat}
* Recreate the table and set Space quota back. The Space quota on the table is
still set to {{REMOVE => true}}
{noformat}
hbase(main):029:0> create 't2','cf1'
Created table t2
Took 0.7348 seconds
=> Hbase::Table - t2
hbase(main):031:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
POLICY => NO_WRITES
Took 0.0088 seconds
hbase(main):032:0> list_quotas
OWNER QUOTAS
TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT => 10M/sec,
SCOPE => MACHINE
TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE => true{noformat}
* Remove RPC quota and drop the table, the Space Quota is not removed
{noformat}
hbase(main):033:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT => NONE
Took 0.0193 seconds
hbase(main):036:0> disable 't2'
Took 0.4305 seconds
hbase(main):037:0> drop 't2'
Took 0.2353 seconds
hbase(main):038:0> list_quotas
OWNER QUOTAS
TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE =>
true{noformat}
* Deleting the quota entry from {{hbase:quota}} seems to be the option to
reset it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[ https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Jain updated HBASE-20672: --- Attachment: HBASE-20672.master.001.patch > Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at > every monitoring interval > - > > Key: HBASE-20672 > URL: https://issues.apache.org/jira/browse/HBASE-20672 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > Attachments: HBASE-20672.branch-1.001.patch, > HBASE-20672.master.001.patch > > > Hbase currently provides counter read/write requests (ReadRequestCount, > WriteRequestCount). That said it is not easy to use counter that reset only > after a restart of the service, we would like to expose 2 new metrics in > HBase to provide ReadRequestRate and WriteRequestRate at region server level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505370#comment-16505370
]
Hadoop QA commented on HBASE-19852:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} hbase-thrift: The patch generated 0 new + 0
unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
23s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 55s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 54s{color}
| {color:red} hbase-thrift in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.thrift.TestThriftSpnegoHttpServer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-19852 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926972/HBASE-19852.master.011.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile findbugs hbaseanti checkstyle |
| uname | Linux 561d4330288e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/13146/artifact/patchprocess/patch-unit-hbase-thrift.txt
[jira] [Commented] (HBASE-20556) Backport HBASE-16490 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505373#comment-16505373 ] Tak Lon (Stephen) Wu commented on HBASE-20556: -- ping ? do I need to provide any changes ? > Backport HBASE-16490 to branch-1 > > > Key: HBASE-20556 > URL: https://issues.apache.org/jira/browse/HBASE-20556 > Project: HBase > Issue Type: Sub-task > Components: HFile, snapshots >Affects Versions: 1.4.4, 1.4.5 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Major > Attachments: HBASE-20556.branch-1.001.patch, > HBASE-20556.branch-1.002.patch, HBASE-20556.branch-1.003.patch, > HBASE-20556.branch-1.004.patch > > > As part of HBASE-20555, HBASE-16490 is the first patch that is needed for > backporting HBASE-18083 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505352#comment-16505352
]
Hadoop QA commented on HBASE-20703:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 43 new + 14 unchanged - 12 fixed =
57 total (was 26) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 5s{color} | {color:orange} The patch generated 19 new + 24 unchanged - 19
fixed = 43 total (was 43) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
32s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20703 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926973/HBASE-20703.master.004.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux bf3e85707d30 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| rubocop | v0.57.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/13145/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/13145/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13145/testReport/ |
| Max. process+thread count | 2157 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13145/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.004.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table
[jira] [Commented] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505344#comment-16505344
]
Hadoop QA commented on HBASE-19852:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} hbase-thrift: The patch generated 0 new + 0
unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 48s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
58s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-19852 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926970/HBASE-19852.master.010.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile findbugs hbaseanti checkstyle |
| uname | Linux 8fecae531086 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13144/testReport/ |
| Max. process+thread count | 1830 (vs. ulimit of 1) |
| modules | C: hbase-thrift U: hbase-thrift |
| Console
[jira] [Commented] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505332#comment-16505332
]
Xu Cang commented on HBASE-20611:
-
@[~chia7712]
Yes, snapshot is a potentially good way to tackle this issue. Will do. Thanks
for your review.
> UnsupportedOperationException may thrown when calling getCallQueueInfo()
>
>
> Key: HBASE-20611
> URL: https://issues.apache.org/jira/browse/HBASE-20611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20611.master.002.patch,
> HBASE-20611.master.003.patch, HBASE-20611.master.004.patch,
> HBASE-20611.master.005.patch, HBASE-20611.master.006.patch,
> HBASE-20611.master.007.patch
>
>
> HBASE-16290 added a new feature to dump queue info, the method
> getCallQueueInfo() need to iterate the queue to get the elements in the
> queue. But, except the Java's LinkedBlockingQueue, the other queue
> implementations like BoundedPriorityBlockingQueue and
> AdaptiveLifoCoDelCallQueue don't implement the method iterator(). If those
> queues are used, a UnsupportedOperationException will be thrown.
> This can be easily be reproduced by the UT testCallQueueInfo while adding a
> conf: conf.set("hbase.ipc.server.callqueue.type", "deadline")
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.iterator(BoundedPriorityBlockingQueue.java:285)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.getCallQueueCountsSummary(RpcExecutor.java:166)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.getCallQueueInfo(SimpleRpcScheduler.java:241)
> at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCallQueueInfo(TestSimpleRpcScheduler.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20703:
Attachment: HBASE-20703.master.004.patch
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.004.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_snapshot_sizes"'
> Took 1.6285 seconds
>
>
> 2.2.1 :002 > list_quota_snapshots
> TABLE USAGE LIMIT IN_VIOLATION POLICY
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_quota_snapshots"'
> Took 0.0371 seconds
> {code}
> Or it just doesn't mention that quotas can't exist:
> {code}
> 2.2.1 :003 > list_quotas
> OWNERQUOTAS
>
>
> 0 row(s)
> Took 0.0475 seconds
>
>
> 2.2.1 :004 > list_quota_table_sizes
> TABLESIZE
>
>
> 0 row(s)
> Took 0.1221 seconds
> {code}
> set quota gives a better pointer that the problem is the feature is off:
> {code}
> 2.2.1 :005 > set_quota USER => 'busbey', GLOBAL_BYPASS => true
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsupportedOperationException: quota support disabled
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkQuotaSupport(MasterQuotaManager.java:442)
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:124)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1555)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.UnsupportedOperationException: quota support disabled
> ... 8 more
> For usage try 'help "set_quota"'
> {code}
> Instead we should give a nice message, like you get if visibility labels is
> off:
> {code}
> 2.2.1 :06 > list_labels
> ERROR: DISABLED: Visibility labels feature is not available
> For usage try 'help "list_labels"'
> Took 0.0426 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20703:
Attachment: (was: HBASE-20703.master.003.patch)
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.004.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_snapshot_sizes"'
> Took 1.6285 seconds
>
>
> 2.2.1 :002 > list_quota_snapshots
> TABLE USAGE LIMIT IN_VIOLATION POLICY
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_quota_snapshots"'
> Took 0.0371 seconds
> {code}
> Or it just doesn't mention that quotas can't exist:
> {code}
> 2.2.1 :003 > list_quotas
> OWNERQUOTAS
>
>
> 0 row(s)
> Took 0.0475 seconds
>
>
> 2.2.1 :004 > list_quota_table_sizes
> TABLESIZE
>
>
> 0 row(s)
> Took 0.1221 seconds
> {code}
> set quota gives a better pointer that the problem is the feature is off:
> {code}
> 2.2.1 :005 > set_quota USER => 'busbey', GLOBAL_BYPASS => true
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsupportedOperationException: quota support disabled
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkQuotaSupport(MasterQuotaManager.java:442)
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:124)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1555)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.UnsupportedOperationException: quota support disabled
> ... 8 more
> For usage try 'help "set_quota"'
> {code}
> Instead we should give a nice message, like you get if visibility labels is
> off:
> {code}
> 2.2.1 :06 > list_labels
> ERROR: DISABLED: Visibility labels feature is not available
> For usage try 'help "list_labels"'
> Took 0.0426 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505311#comment-16505311
]
Hadoop QA commented on HBASE-20703:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 41 new + 14 unchanged - 12 fixed =
55 total (was 26) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 4s{color} | {color:orange} The patch generated 19 new + 24 unchanged - 19
fixed = 43 total (was 43) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
32s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
6s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20703 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926969/HBASE-20703.master.003.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux e25926faaafd 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| rubocop | v0.57.1 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/13143/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/13143/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13143/testReport/ |
| Max. process+thread count | 2592 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13143/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.003.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table
[jira] [Updated] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Risden updated HBASE-19852:
-
Attachment: HBASE-19852.master.011.patch
> HBase Thrift 1 server SPNEGO Improvements
> -
>
> Key: HBASE-19852
> URL: https://issues.apache.org/jira/browse/HBASE-19852
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Major
> Attachments: HBASE-19852.master.001.patch,
> HBASE-19852.master.002.patch, HBASE-19852.master.003.patch,
> HBASE-19852.master.004.patch, HBASE-19852.master.006.patch,
> HBASE-19852.master.007.patch.txt, HBASE-19852.master.008.patch,
> HBASE-19852.master.009.patch, HBASE-19852.master.010.patch,
> HBASE-19852.master.011.patch
>
>
> HBase Thrift1 server has some issues when trying to use SPNEGO.
> From mailing list:
> http://mail-archives.apache.org/mod_mbox/hbase-user/201801.mbox/%3CCAJU9nmh5YtZ%2BmAQSLo91yKm8pRVzAPNLBU9vdVMCcxHRtRqgoA%40mail.gmail.com%3E
> {quote}While setting up the HBase Thrift server with HTTP, there were a
> significant amount of 401 errors where the HBase Thrift wasn't able to
> handle the incoming Kerberos request. Documentation online is sparse when
> it comes to setting up the principal/keytab for HTTP Kerberos.
> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
> principal/keytab like other Thrift based servers (HiveServer2). It looks
> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
> were very close to the same at one point. I made the following changes to
> HBase Thrift v1 server implementation to make it work:
> * add SPNEGO principal/keytab if in HTTP mode
> * return 401 immediately if no authorization header instead of waiting for
> try/catch down in program flow{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505298#comment-16505298
]
Hadoop QA commented on HBASE-20703:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
16s{color} | {color:red} The patch generated 42 new + 14 unchanged - 12 fixed =
56 total (was 26) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 4s{color} | {color:orange} The patch generated 19 new + 24 unchanged - 19
fixed = 43 total (was 43) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
53s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20703 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926968/HBASE-20703.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux ef5e099b9772 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| rubocop | v0.57.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/13142/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/13142/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13142/testReport/ |
| Max. process+thread count | 2096 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13142/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.003.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table
[jira] [Commented] (HBASE-20691) Storage policy should allow deferring to HDFS
[ https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505296#comment-16505296 ] Mike Drob commented on HBASE-20691: --- Would this be better broken out into two properties, one to control whether we overwrite the hdfs storage policy, and one to choose what we set it to? Then we don't have to worry about the special meaning of NONE and work special logic around it? > Storage policy should allow deferring to HDFS > - > > Key: HBASE-20691 > URL: https://issues.apache.org/jira/browse/HBASE-20691 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Affects Versions: 1.5.0, 2.0.0 >Reporter: Sean Busbey >Assignee: Yu Li >Priority: Minor > Fix For: 2.1.0, 1.5.0 > > Attachments: HBASE-20691.patch > > > In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using > "NONE" as the storage policy in hbase configs. > As described on this [dev@hbase thread "WAL storage policies and interactions > with Hadoop admin > tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb2237@%3Cdev.hbase.apache.org%3E] > we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now). > Additionally, we can't set the policy to HOT in the event that HDFS has > changed the policy for a parent directory of our WALs. > We should put back that ability. Presuming this is done by re-adopting the > "NONE" placeholder variable, we need to ensure that value doesn't get passed > to HDFS APIs. Since it isn't a valid storage policy attempting to use it will > cause a bunch of logging churn (which will be a regression of the problem > HBASE-18118 sought to fix). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Risden updated HBASE-19852:
-
Attachment: HBASE-19852.master.010.patch
> HBase Thrift 1 server SPNEGO Improvements
> -
>
> Key: HBASE-19852
> URL: https://issues.apache.org/jira/browse/HBASE-19852
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Major
> Attachments: HBASE-19852.master.001.patch,
> HBASE-19852.master.002.patch, HBASE-19852.master.003.patch,
> HBASE-19852.master.004.patch, HBASE-19852.master.006.patch,
> HBASE-19852.master.007.patch.txt, HBASE-19852.master.008.patch,
> HBASE-19852.master.009.patch, HBASE-19852.master.010.patch
>
>
> HBase Thrift1 server has some issues when trying to use SPNEGO.
> From mailing list:
> http://mail-archives.apache.org/mod_mbox/hbase-user/201801.mbox/%3CCAJU9nmh5YtZ%2BmAQSLo91yKm8pRVzAPNLBU9vdVMCcxHRtRqgoA%40mail.gmail.com%3E
> {quote}While setting up the HBase Thrift server with HTTP, there were a
> significant amount of 401 errors where the HBase Thrift wasn't able to
> handle the incoming Kerberos request. Documentation online is sparse when
> it comes to setting up the principal/keytab for HTTP Kerberos.
> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
> principal/keytab like other Thrift based servers (HiveServer2). It looks
> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
> were very close to the same at one point. I made the following changes to
> HBase Thrift v1 server implementation to make it work:
> * add SPNEGO principal/keytab if in HTTP mode
> * return 401 immediately if no authorization header instead of waiting for
> try/catch down in program flow{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20703:
Attachment: (was: HBASE-20703.master.002.patch)
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.003.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_snapshot_sizes"'
> Took 1.6285 seconds
>
>
> 2.2.1 :002 > list_quota_snapshots
> TABLE USAGE LIMIT IN_VIOLATION POLICY
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_quota_snapshots"'
> Took 0.0371 seconds
> {code}
> Or it just doesn't mention that quotas can't exist:
> {code}
> 2.2.1 :003 > list_quotas
> OWNERQUOTAS
>
>
> 0 row(s)
> Took 0.0475 seconds
>
>
> 2.2.1 :004 > list_quota_table_sizes
> TABLESIZE
>
>
> 0 row(s)
> Took 0.1221 seconds
> {code}
> set quota gives a better pointer that the problem is the feature is off:
> {code}
> 2.2.1 :005 > set_quota USER => 'busbey', GLOBAL_BYPASS => true
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsupportedOperationException: quota support disabled
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkQuotaSupport(MasterQuotaManager.java:442)
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:124)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1555)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.UnsupportedOperationException: quota support disabled
> ... 8 more
> For usage try 'help "set_quota"'
> {code}
> Instead we should give a nice message, like you get if visibility labels is
> off:
> {code}
> 2.2.1 :06 > list_labels
> ERROR: DISABLED: Visibility labels feature is not available
> For usage try 'help "list_labels"'
> Took 0.0426 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20703:
Attachment: HBASE-20703.master.003.patch
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.003.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_snapshot_sizes"'
> Took 1.6285 seconds
>
>
> 2.2.1 :002 > list_quota_snapshots
> TABLE USAGE LIMIT IN_VIOLATION POLICY
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_quota_snapshots"'
> Took 0.0371 seconds
> {code}
> Or it just doesn't mention that quotas can't exist:
> {code}
> 2.2.1 :003 > list_quotas
> OWNERQUOTAS
>
>
> 0 row(s)
> Took 0.0475 seconds
>
>
> 2.2.1 :004 > list_quota_table_sizes
> TABLESIZE
>
>
> 0 row(s)
> Took 0.1221 seconds
> {code}
> set quota gives a better pointer that the problem is the feature is off:
> {code}
> 2.2.1 :005 > set_quota USER => 'busbey', GLOBAL_BYPASS => true
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsupportedOperationException: quota support disabled
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkQuotaSupport(MasterQuotaManager.java:442)
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:124)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1555)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.UnsupportedOperationException: quota support disabled
> ... 8 more
> For usage try 'help "set_quota"'
> {code}
> Instead we should give a nice message, like you get if visibility labels is
> off:
> {code}
> 2.2.1 :06 > list_labels
> ERROR: DISABLED: Visibility labels feature is not available
> For usage try 'help "list_labels"'
> Took 0.0426 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505262#comment-16505262
]
Xu Cang commented on HBASE-20703:
-
New commands' ouput:
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.002.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_snapshot_sizes"'
> Took 1.6285 seconds
>
>
> 2.2.1 :002 > list_quota_snapshots
> TABLE USAGE LIMIT IN_VIOLATION POLICY
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_quota_snapshots"'
> Took 0.0371 seconds
> {code}
> Or it just doesn't mention that quotas can't exist:
> {code}
> 2.2.1 :003 > list_quotas
> OWNERQUOTAS
>
>
> 0 row(s)
> Took 0.0475 seconds
>
>
> 2.2.1 :004 > list_quota_table_sizes
> TABLESIZE
>
>
> 0 row(s)
> Took 0.1221 seconds
> {code}
> set quota gives a better pointer that the problem is the feature is off:
> {code}
> 2.2.1 :005 > set_quota USER => 'busbey', GLOBAL_BYPASS => true
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsupportedOperationException: quota support disabled
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkQuotaSupport(MasterQuotaManager.java:442)
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:124)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1555)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.UnsupportedOperationException: quota support disabled
> ... 8 more
> For usage try 'help "set_quota"'
> {code}
> Instead we should give a nice message, like you get if visibility labels is
> off:
> {code}
> 2.2.1 :06 > list_labels
> ERROR: DISABLED: Visibility labels feature is not available
> For usage try 'help "list_labels"'
> Took 0.0426 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505262#comment-16505262
]
Xu Cang edited comment on HBASE-20703 at 6/7/18 8:57 PM:
-
New commands' ouput:
hbase(main):035:0> list_quota_snapshots
TABLE USAGE LIMIT IN_VIOLATION POLICY
ERROR: Quota feature is disabled.
For usage try 'help "list_quota_snapshots"'
Took 0.0130 seconds
hbase(main):037:0> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT => '50T',
POLICY => NO_WRITES
ERROR: Quota feature is disabled.
For usage try 'help "set_quota"'
Took 0.0125 seconds
hbase(main):038:0> set_quota TYPE => THROTTLE, USER => 'u1', LIMIT =>
'10req/sec'
ERROR: Quota feature is disabled.
For usage try 'help "set_quota"'
Took 0.0079 seconds
hbase(main):039:0>
was (Author: xucang):
New commands' ouput:
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.002.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_snapshot_sizes"'
> Took 1.6285 seconds
>
>
> 2.2.1 :002 > list_quota_snapshots
> TABLE USAGE LIMIT IN_VIOLATION POLICY
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_quota_snapshots"'
> Took 0.0371 seconds
> {code}
> Or it just doesn't mention that quotas can't exist:
> {code}
> 2.2.1 :003 > list_quotas
> OWNERQUOTAS
>
>
> 0 row(s)
> Took 0.0475 seconds
>
>
> 2.2.1 :004 > list_quota_table_sizes
> TABLESIZE
>
>
> 0 row(s)
> Took 0.1221 seconds
> {code}
> set quota gives a better pointer that the problem is the feature is off:
> {code}
> 2.2.1 :005 > set_quota USER => 'busbey', GLOBAL_BYPASS => true
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsupportedOperationException: quota support disabled
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkQuotaSupport(MasterQuotaManager.java:442)
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:124)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1555)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.UnsupportedOperationException: quota support disabled
> ... 8 more
> For usage try 'help "set_quota"'
> {code}
> Instead we should give a nice message, like you get if visibility labels is
> off:
> {code}
> 2.2.1 :06 > list_labels
> ERROR: DISABLED: Visibility labels feature is not available
> For usage try 'help "list_labels"'
> Took 0.0426 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20703:
Attachment: HBASE-20703.master.002.patch
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch,
> HBASE-20703.master.002.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_snapshot_sizes"'
> Took 1.6285 seconds
>
>
> 2.2.1 :002 > list_quota_snapshots
> TABLE USAGE LIMIT IN_VIOLATION POLICY
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_quota_snapshots"'
> Took 0.0371 seconds
> {code}
> Or it just doesn't mention that quotas can't exist:
> {code}
> 2.2.1 :003 > list_quotas
> OWNERQUOTAS
>
>
> 0 row(s)
> Took 0.0475 seconds
>
>
> 2.2.1 :004 > list_quota_table_sizes
> TABLESIZE
>
>
> 0 row(s)
> Took 0.1221 seconds
> {code}
> set quota gives a better pointer that the problem is the feature is off:
> {code}
> 2.2.1 :005 > set_quota USER => 'busbey', GLOBAL_BYPASS => true
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsupportedOperationException: quota support disabled
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkQuotaSupport(MasterQuotaManager.java:442)
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:124)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1555)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.UnsupportedOperationException: quota support disabled
> ... 8 more
> For usage try 'help "set_quota"'
> {code}
> Instead we should give a nice message, like you get if visibility labels is
> off:
> {code}
> 2.2.1 :06 > list_labels
> ERROR: DISABLED: Visibility labels feature is not available
> For usage try 'help "list_labels"'
> Took 0.0426 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20635) Support to convert the shaded user permission proto to client user permission object
[
https://issues.apache.org/jira/browse/HBASE-20635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505255#comment-16505255
]
Ted Yu commented on HBASE-20635:
lgtm
> Support to convert the shaded user permission proto to client user permission
> object
>
>
> Key: HBASE-20635
> URL: https://issues.apache.org/jira/browse/HBASE-20635
> Project: HBase
> Issue Type: Bug
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Attachments: HBASE-20635.patch, HBASE-20635_v2.patch,
> PHOENIX-4528_5.x-HBase-2.0_v2.patch
>
>
> Currently we have API to build the protobuf UserPermission to client user
> permission in AccessControlUtil but we cannot do the same when we use shaded
> protobufs.
> {noformat}
> /**
>* Converts a user permission proto to a client user permission object.
>*
>* @param proto the protobuf UserPermission
>* @return the converted UserPermission
>*/
> public static UserPermission
> toUserPermission(AccessControlProtos.UserPermission proto) {
> return new UserPermission(proto.getUser().toByteArray(),
> toTablePermission(proto.getPermission()));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-8547) Fix java.lang.RuntimeException: Cached an already cached block
[
https://issues.apache.org/jira/browse/HBASE-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505253#comment-16505253
]
Hudson commented on HBASE-8547:
---
Results for branch branch-1.3
[build #354 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix java.lang.RuntimeException: Cached an already cached block
> --
>
> Key: HBASE-8547
> URL: https://issues.apache.org/jira/browse/HBASE-8547
> Project: HBase
> Issue Type: Bug
> Components: io, regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8547_v1-0.94.patch, hbase-8547_v1-0.94.patch,
> hbase-8547_v1.patch, hbase-8547_v2-0.94-reduced.patch,
> hbase-8547_v2-addendum2+3-0.94.patch, hbase-8547_v2-addendum2.patch,
> hbase-8547_v2-addendum2.patch, hbase-8547_v2-addendum3.patch,
> hbase-8547_v2-trunk.patch
>
>
> In one test, one of the region servers received the following on 0.94.
> Note HalfStoreFileReader in the stack trace. I think the root cause is that
> after the region is split, the mid point can be in the middle of the block
> (for store files that the mid point is not chosen from). Each half store file
> tries to load the half block and put it in the block cache. Since IdLock is
> instantiated per store file reader, they do not share the same IdLock
> instance, thus does not lock against each other effectively.
> {code}
> 2013-05-12 01:30:37,733 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:·
> java.lang.RuntimeException: Cached an already cached block
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:279)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:353)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:480)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:501)
> at
> org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:237)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:351)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:354)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:312)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:277)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:543)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:411)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3829)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3778)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3770)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2643)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> {code}
> I can see two possible fixes:
> # Allow this kind of rare
[jira] [Commented] (HBASE-20605) Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
[
https://issues.apache.org/jira/browse/HBASE-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505251#comment-16505251
]
Hudson commented on HBASE-20605:
Results for branch branch-1.3
[build #354 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission
> check
> -
>
> Key: HBASE-20605
> URL: https://issues.apache.org/jira/browse/HBASE-20605
> Project: HBase
> Issue Type: Improvement
> Components: security
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.5
>
> Attachments: HBASE-20605.001.branch-1.patch,
> HBASE-20605.002.branch-1.patch
>
>
> Some folks in Hadoop are working on landing a new FileSystem from the Azure
> team: HADOOP-15407
> At present, this FileSystem doesn't support permissions which causes the
> SecureBulkLoadEndpoint to balk because it the staging directory doesn't have
> the proper 711 permissions.
> We have a static list of FileSystem schemes which we ignore this check on. I
> have a patch on an HBase 1.1ish which:
> # Adds the new FileSystem scheme
> # Makes this list configurable for the future
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20665) "Already cached block XXX" message should be DEBUG
[
https://issues.apache.org/jira/browse/HBASE-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505252#comment-16505252
]
Hudson commented on HBASE-20665:
Results for branch branch-1.3
[build #354 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/354//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> "Already cached block XXX" message should be DEBUG
> --
>
> Key: HBASE-20665
> URL: https://issues.apache.org/jira/browse/HBASE-20665
> Project: HBase
> Issue Type: Task
> Components: BlockCache
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Eric Maynard
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.1.0, 1.2.7, 1.3.3, 2.0.1
>
> Attachments: HBASE-20665.master.001.patch,
> HBASE-20665.master.001.patch
>
>
> Testing a local cluster that relies on the LruBlockCache for a scan-heavy
> workload and I'm getting a bunch of log entries at WARN
> {code}
> 2018-05-30 12:28:47,192 WARN org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Cached an already cached block: df01f5bf6a244f6bb1a626b927377fff_54780812
> cb:df01f5bf6a244f6bb1a626b927377fff_54780812. This is harmless and can happen
> in rare cases (see HBASE-8547)
> {code}
> As the log message notes (and the code confirms) this is a harmless result of
> contention for getting a block into the CHM. the message should be at DEBUG.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20674) clean up short circuit read logic and docs
[
https://issues.apache.org/jira/browse/HBASE-20674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505238#comment-16505238
]
Hadoop QA commented on HBASE-20674:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 4m
16s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
8s{color} | {color:red} root: The patch generated 8 new + 161 unchanged - 13
fixed = 169 total (was 174) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
8s{color} | {color:blue} patch has no errors when building the reference guide.
See footer for rendered docs, which you should manually inspect. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
47s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}143m 41s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}210m 18s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.hfile.TestChecksumWithHdfs |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (HBASE-20656) Validate pre-2.0 coprocessors against HBase 2.0+
[ https://issues.apache.org/jira/browse/HBASE-20656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505222#comment-16505222 ] Mike Drob commented on HBASE-20656: --- Looks like there is some issue with applying the patch in PreUpgradeValidator.java - [~balazs.meszaros] can you take another look? > Validate pre-2.0 coprocessors against HBase 2.0+ > > > Key: HBASE-20656 > URL: https://issues.apache.org/jira/browse/HBASE-20656 > Project: HBase > Issue Type: New Feature > Components: tooling >Affects Versions: 3.0.0 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: HBASE-20656.master.001.patch, > HBASE-20656.master.002.patch, HBASE-20656.master.003.patch, > HBASE-20656.master.004.patch, HBASE-20656.master.005.patch, > HBASE-20656.master.006.patch > > > We have co-processors for a while, but the API has been changed recently. We > should give some tooling for our users to determine if they can use the > previous co-processors safely or not. > The tool should: > - try to load the co-processors on our current classpath for ensuring class > references are on our classpath, > - should check for previously removed co-processor methods. > In this version we check only method signatures. Code references should be > checked in further versions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20656) Validate pre-2.0 coprocessors against HBase 2.0+
[ https://issues.apache.org/jira/browse/HBASE-20656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505219#comment-16505219 ] Mike Drob commented on HBASE-20656: --- Retriggered QA manually. Patch looks good to me, will commit later if QA comes back happy. > Validate pre-2.0 coprocessors against HBase 2.0+ > > > Key: HBASE-20656 > URL: https://issues.apache.org/jira/browse/HBASE-20656 > Project: HBase > Issue Type: New Feature > Components: tooling >Affects Versions: 3.0.0 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: HBASE-20656.master.001.patch, > HBASE-20656.master.002.patch, HBASE-20656.master.003.patch, > HBASE-20656.master.004.patch, HBASE-20656.master.005.patch, > HBASE-20656.master.006.patch > > > We have co-processors for a while, but the API has been changed recently. We > should give some tooling for our users to determine if they can use the > previous co-processors safely or not. > The tool should: > - try to load the co-processors on our current classpath for ensuring class > references are on our classpath, > - should check for previously removed co-processor methods. > In this version we check only method signatures. Code references should be > checked in further versions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505220#comment-16505220
]
Hadoop QA commented on HBASE-19852:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 4m
33s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
36s{color} | {color:red} hbase-thrift in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 36s{color}
| {color:red} hbase-thrift in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} hbase-thrift: The patch generated 0 new + 0
unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
16s{color} | {color:red} The patch causes 18 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
39s{color} | {color:red} The patch causes 18 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} hbase-thrift in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 25s{color}
| {color:red} hbase-thrift in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-19852 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926960/HBASE-19852.master.009.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile findbugs hbaseanti checkstyle |
| uname | Linux 3d12b4f2a57a 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| mvninstall |
[jira] [Commented] (HBASE-20656) Validate pre-2.0 coprocessors against HBase 2.0+
[
https://issues.apache.org/jira/browse/HBASE-20656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505217#comment-16505217
]
Hadoop QA commented on HBASE-20656:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-20656 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20656 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926747/HBASE-20656.master.006.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13141/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Validate pre-2.0 coprocessors against HBase 2.0+
>
>
> Key: HBASE-20656
> URL: https://issues.apache.org/jira/browse/HBASE-20656
> Project: HBase
> Issue Type: New Feature
> Components: tooling
>Affects Versions: 3.0.0
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
> Attachments: HBASE-20656.master.001.patch,
> HBASE-20656.master.002.patch, HBASE-20656.master.003.patch,
> HBASE-20656.master.004.patch, HBASE-20656.master.005.patch,
> HBASE-20656.master.006.patch
>
>
> We have co-processors for a while, but the API has been changed recently. We
> should give some tooling for our users to determine if they can use the
> previous co-processors safely or not.
> The tool should:
> - try to load the co-processors on our current classpath for ensuring class
> references are on our classpath,
> - should check for previously removed co-processor methods.
> In this version we check only method signatures. Code references should be
> checked in further versions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20605) Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
[
https://issues.apache.org/jira/browse/HBASE-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505195#comment-16505195
]
Hudson commented on HBASE-20605:
Results for branch branch-1.4
[build #347 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/347/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/347//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/347//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/347//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission
> check
> -
>
> Key: HBASE-20605
> URL: https://issues.apache.org/jira/browse/HBASE-20605
> Project: HBase
> Issue Type: Improvement
> Components: security
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.5
>
> Attachments: HBASE-20605.001.branch-1.patch,
> HBASE-20605.002.branch-1.patch
>
>
> Some folks in Hadoop are working on landing a new FileSystem from the Azure
> team: HADOOP-15407
> At present, this FileSystem doesn't support permissions which causes the
> SecureBulkLoadEndpoint to balk because it the staging directory doesn't have
> the proper 711 permissions.
> We have a static list of FileSystem schemes which we ignore this check on. I
> have a patch on an HBase 1.1ish which:
> # Adds the new FileSystem scheme
> # Makes this list configurable for the future
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20699) QuotaCache should cancel the QuotaRefresherChore service inside its stop()
[
https://issues.apache.org/jira/browse/HBASE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505178#comment-16505178
]
Hadoop QA commented on HBASE-20699:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
50s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 54s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}113m 8s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}156m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20699 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926932/HBASE-20699.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f6ceff264af1 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/13134/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13134/testReport/ |
| Max. process+thread count | 4217 (vs. ulimit of 1) |
| modules | C: hbase-server U:
[jira] [Updated] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Risden updated HBASE-19852:
-
Attachment: HBASE-19852.master.009.patch
> HBase Thrift 1 server SPNEGO Improvements
> -
>
> Key: HBASE-19852
> URL: https://issues.apache.org/jira/browse/HBASE-19852
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Major
> Attachments: HBASE-19852.master.001.patch,
> HBASE-19852.master.002.patch, HBASE-19852.master.003.patch,
> HBASE-19852.master.004.patch, HBASE-19852.master.006.patch,
> HBASE-19852.master.007.patch.txt, HBASE-19852.master.008.patch,
> HBASE-19852.master.009.patch
>
>
> HBase Thrift1 server has some issues when trying to use SPNEGO.
> From mailing list:
> http://mail-archives.apache.org/mod_mbox/hbase-user/201801.mbox/%3CCAJU9nmh5YtZ%2BmAQSLo91yKm8pRVzAPNLBU9vdVMCcxHRtRqgoA%40mail.gmail.com%3E
> {quote}While setting up the HBase Thrift server with HTTP, there were a
> significant amount of 401 errors where the HBase Thrift wasn't able to
> handle the incoming Kerberos request. Documentation online is sparse when
> it comes to setting up the principal/keytab for HTTP Kerberos.
> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
> principal/keytab like other Thrift based servers (HiveServer2). It looks
> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
> were very close to the same at one point. I made the following changes to
> HBase Thrift v1 server implementation to make it work:
> * add SPNEGO principal/keytab if in HTTP mode
> * return 401 immediately if no authorization header instead of waiting for
> try/catch down in program flow{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505171#comment-16505171
]
Hadoop QA commented on HBASE-19852:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-19852 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19852 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12917369/HBASE-19852.master.008.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13139/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> HBase Thrift 1 server SPNEGO Improvements
> -
>
> Key: HBASE-19852
> URL: https://issues.apache.org/jira/browse/HBASE-19852
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Major
> Attachments: HBASE-19852.master.001.patch,
> HBASE-19852.master.002.patch, HBASE-19852.master.003.patch,
> HBASE-19852.master.004.patch, HBASE-19852.master.006.patch,
> HBASE-19852.master.007.patch.txt, HBASE-19852.master.008.patch
>
>
> HBase Thrift1 server has some issues when trying to use SPNEGO.
> From mailing list:
> http://mail-archives.apache.org/mod_mbox/hbase-user/201801.mbox/%3CCAJU9nmh5YtZ%2BmAQSLo91yKm8pRVzAPNLBU9vdVMCcxHRtRqgoA%40mail.gmail.com%3E
> {quote}While setting up the HBase Thrift server with HTTP, there were a
> significant amount of 401 errors where the HBase Thrift wasn't able to
> handle the incoming Kerberos request. Documentation online is sparse when
> it comes to setting up the principal/keytab for HTTP Kerberos.
> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
> principal/keytab like other Thrift based servers (HiveServer2). It looks
> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
> were very close to the same at one point. I made the following changes to
> HBase Thrift v1 server implementation to make it work:
> * add SPNEGO principal/keytab if in HTTP mode
> * return 401 immediately if no authorization header instead of waiting for
> try/catch down in program flow{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19852) HBase Thrift 1 server SPNEGO Improvements
[
https://issues.apache.org/jira/browse/HBASE-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505164#comment-16505164
]
Kevin Risden commented on HBASE-19852:
--
Checking the latest patch noticed it doesn't apply cleanly to master. Will look
into this and reupload patch and address the KDC_SERVER_HOST comment.
> HBase Thrift 1 server SPNEGO Improvements
> -
>
> Key: HBASE-19852
> URL: https://issues.apache.org/jira/browse/HBASE-19852
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Major
> Attachments: HBASE-19852.master.001.patch,
> HBASE-19852.master.002.patch, HBASE-19852.master.003.patch,
> HBASE-19852.master.004.patch, HBASE-19852.master.006.patch,
> HBASE-19852.master.007.patch.txt, HBASE-19852.master.008.patch
>
>
> HBase Thrift1 server has some issues when trying to use SPNEGO.
> From mailing list:
> http://mail-archives.apache.org/mod_mbox/hbase-user/201801.mbox/%3CCAJU9nmh5YtZ%2BmAQSLo91yKm8pRVzAPNLBU9vdVMCcxHRtRqgoA%40mail.gmail.com%3E
> {quote}While setting up the HBase Thrift server with HTTP, there were a
> significant amount of 401 errors where the HBase Thrift wasn't able to
> handle the incoming Kerberos request. Documentation online is sparse when
> it comes to setting up the principal/keytab for HTTP Kerberos.
> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
> principal/keytab like other Thrift based servers (HiveServer2). It looks
> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
> were very close to the same at one point. I made the following changes to
> HBase Thrift v1 server implementation to make it work:
> * add SPNEGO principal/keytab if in HTTP mode
> * return 401 immediately if no authorization header instead of waiting for
> try/catch down in program flow{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20605) Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
[
https://issues.apache.org/jira/browse/HBASE-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505163#comment-16505163
]
Hudson commented on HBASE-20605:
Results for branch branch-1.2
[build #358 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission
> check
> -
>
> Key: HBASE-20605
> URL: https://issues.apache.org/jira/browse/HBASE-20605
> Project: HBase
> Issue Type: Improvement
> Components: security
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.5
>
> Attachments: HBASE-20605.001.branch-1.patch,
> HBASE-20605.002.branch-1.patch
>
>
> Some folks in Hadoop are working on landing a new FileSystem from the Azure
> team: HADOOP-15407
> At present, this FileSystem doesn't support permissions which causes the
> SecureBulkLoadEndpoint to balk because it the staging directory doesn't have
> the proper 711 permissions.
> We have a static list of FileSystem schemes which we ignore this check on. I
> have a patch on an HBase 1.1ish which:
> # Adds the new FileSystem scheme
> # Makes this list configurable for the future
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20665) "Already cached block XXX" message should be DEBUG
[
https://issues.apache.org/jira/browse/HBASE-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505165#comment-16505165
]
Hudson commented on HBASE-20665:
Results for branch branch-1.2
[build #358 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> "Already cached block XXX" message should be DEBUG
> --
>
> Key: HBASE-20665
> URL: https://issues.apache.org/jira/browse/HBASE-20665
> Project: HBase
> Issue Type: Task
> Components: BlockCache
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Eric Maynard
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.1.0, 1.2.7, 1.3.3, 2.0.1
>
> Attachments: HBASE-20665.master.001.patch,
> HBASE-20665.master.001.patch
>
>
> Testing a local cluster that relies on the LruBlockCache for a scan-heavy
> workload and I'm getting a bunch of log entries at WARN
> {code}
> 2018-05-30 12:28:47,192 WARN org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Cached an already cached block: df01f5bf6a244f6bb1a626b927377fff_54780812
> cb:df01f5bf6a244f6bb1a626b927377fff_54780812. This is harmless and can happen
> in rare cases (see HBASE-8547)
> {code}
> As the log message notes (and the code confirms) this is a harmless result of
> contention for getting a block into the CHM. the message should be at DEBUG.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-8547) Fix java.lang.RuntimeException: Cached an already cached block
[
https://issues.apache.org/jira/browse/HBASE-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505166#comment-16505166
]
Hudson commented on HBASE-8547:
---
Results for branch branch-1.2
[build #358 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/358//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix java.lang.RuntimeException: Cached an already cached block
> --
>
> Key: HBASE-8547
> URL: https://issues.apache.org/jira/browse/HBASE-8547
> Project: HBase
> Issue Type: Bug
> Components: io, regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8547_v1-0.94.patch, hbase-8547_v1-0.94.patch,
> hbase-8547_v1.patch, hbase-8547_v2-0.94-reduced.patch,
> hbase-8547_v2-addendum2+3-0.94.patch, hbase-8547_v2-addendum2.patch,
> hbase-8547_v2-addendum2.patch, hbase-8547_v2-addendum3.patch,
> hbase-8547_v2-trunk.patch
>
>
> In one test, one of the region servers received the following on 0.94.
> Note HalfStoreFileReader in the stack trace. I think the root cause is that
> after the region is split, the mid point can be in the middle of the block
> (for store files that the mid point is not chosen from). Each half store file
> tries to load the half block and put it in the block cache. Since IdLock is
> instantiated per store file reader, they do not share the same IdLock
> instance, thus does not lock against each other effectively.
> {code}
> 2013-05-12 01:30:37,733 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:·
> java.lang.RuntimeException: Cached an already cached block
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:279)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:353)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:480)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:501)
> at
> org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:237)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:351)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:354)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:312)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:277)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:543)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:411)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3829)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3778)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3770)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2643)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> {code}
> I can see two possible fixes:
> # Allow this kind of rare
[jira] [Commented] (HBASE-20681) IntegrationTestDriver fails after HADOOP-15406 due to missing hamcrest-core
[
https://issues.apache.org/jira/browse/HBASE-20681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505148#comment-16505148
]
Josh Elser commented on HBASE-20681:
Folks happy with this?
> IntegrationTestDriver fails after HADOOP-15406 due to missing hamcrest-core
> ---
>
> Key: HBASE-20681
> URL: https://issues.apache.org/jira/browse/HBASE-20681
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20681.001.patch, HBASE-20681.002.patch,
> HBASE-20681.003.patch, HBASE-20681.004.patch, HBASE-20681.004.patch
>
>
> HADOOP-15406 marked mockito and junit as test-only dependencies which, I
> believe, has stopped them from being included in a stock Hadoop classpath.
> Prior, you'd get hamcrest at {{share/hadoop/common/lib/hamcrest-core-1.3.jar}}
> However, we depend on it being there for our junit in hbase-it:
> {noformat}
> [INFO] --- maven-dependency-plugin:3.0.1:tree (default-cli) @ hbase-it ---
> [INFO] org.apache.hbase:hbase-it:jar:2.0.1-SNAPSHOT
> [INFO] +- junit:junit:jar:4.12:test
> [INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test
> {noformat}
> We need to make sure we include it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20699) QuotaCache should cancel the QuotaRefresherChore service inside its stop()
[
https://issues.apache.org/jira/browse/HBASE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505132#comment-16505132
]
Hadoop QA commented on HBASE-20699:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
5s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 47s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}112m
3s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}148m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20699 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926926/HBASE-20699.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b5955aa3986a 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 9a80907760 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13132/testReport/ |
| Max. process+thread count | 5174 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13132/console |
| Powered by |
[jira] [Commented] (HBASE-20681) IntegrationTestDriver fails after HADOOP-15406 due to missing hamcrest-core
[
https://issues.apache.org/jira/browse/HBASE-20681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505129#comment-16505129
]
Hadoop QA commented on HBASE-20681:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 6s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
9s{color} | {color:green} hbase-resource-bundle in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hbase-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20681 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926950/HBASE-20681.004.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux a4393783b274 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13137/testReport/ |
| Max. process+thread count | 83 (vs. ulimit of 1) |
| modules | C: hbase-resource-bundle hbase-assembly hbase-shaded U: . |
| Console output |
[jira] [Commented] (HBASE-20546) Improve perf of RegionLocationFinder.mapHostNameToServerName
[ https://issues.apache.org/jira/browse/HBASE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505127#comment-16505127 ] Thiruvel Thirumoolan commented on HBASE-20546: -- I can move this map to ServerManager, so its always updated. Would that be ok? Since its called from picker, its significant. It came up when I profiled balancer on our cluster setup. > Improve perf of RegionLocationFinder.mapHostNameToServerName > > > Key: HBASE-20546 > URL: https://issues.apache.org/jira/browse/HBASE-20546 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 1.5.0, 2.0.1, 1.4.6 > > Attachments: HBASE-20546.branch-1.4.001.patch > > > RegionLocationFinder.getTopBlockLocations() is called multiple times during > balancer. While profiling on a large table balance, mapHostNameToServerName() > seem to take a lot of time. One of the maps is repeatedly created for each > iteration, while we can just initialize it once. > Goes into both branch-1 and branch-2, although patches differ slightly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505120#comment-16505120
]
Hadoop QA commented on HBASE-20703:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
6s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
4s{color} | {color:red} The patch generated 10 new + 2 unchanged - 0 fixed = 12
total (was 2) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 1s{color} | {color:orange} The patch generated 2 new + 3 unchanged - 2
fixed = 5 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
49s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20703 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926949/HBASE-20703.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux 7136ee9da5dd 4.4.0-98-generic #121-Ubuntu SMP Tue Oct 10
14:24:03 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cfeb26d27a |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| rubocop | v0.57.1 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/13136/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/13136/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13136/testReport/ |
| Max. process+thread count | 2523 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13136/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help
[jira] [Commented] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[
https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505098#comment-16505098
]
Hadoop QA commented on HBASE-20672:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m
2s{color} | {color:red} Docker failed to build yetus/hbase:36a7029. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20672 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926953/HBASE-20672.branch-1.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13138/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at
> every monitoring interval
> -
>
> Key: HBASE-20672
> URL: https://issues.apache.org/jira/browse/HBASE-20672
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Ankit Jain
>Assignee: Ankit Jain
>Priority: Minor
> Attachments: HBASE-20672.branch-1.001.patch
>
>
> Hbase currently provides counter read/write requests (ReadRequestCount,
> WriteRequestCount). That said it is not easy to use counter that reset only
> after a restart of the service, we would like to expose 2 new metrics in
> HBase to provide ReadRequestRate and WriteRequestRate at region server level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20649) Validate HFiles do not have PREFIX_TREE DataBlockEncoding
[ https://issues.apache.org/jira/browse/HBASE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505097#comment-16505097 ] Mike Drob commented on HBASE-20649: --- bq. HBCK uses the same utility when -checkCorruptHFiles is used with 50 threads. Can we use hbck for this then? Maybe we don't have to write a new feature at all? Could be confusing from a messaging standpoint I guess > Validate HFiles do not have PREFIX_TREE DataBlockEncoding > - > > Key: HBASE-20649 > URL: https://issues.apache.org/jira/browse/HBASE-20649 > Project: HBase > Issue Type: New Feature >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Minor > Attachments: HBASE-20649.master.001.patch, > HBASE-20649.master.002.patch > > > HBASE-20592 adds a tool to check column families on the cluster do not have > PREFIX_TREE encoding. > Since it is possible that DataBlockEncoding was already changed but HFiles > are not rewritten yet we would need a tool that can verify the content of > hfiles in the cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[ https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Jain updated HBASE-20672: --- Attachment: (was: HBASE-20672.branch-1.v1.patch) > Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at > every monitoring interval > - > > Key: HBASE-20672 > URL: https://issues.apache.org/jira/browse/HBASE-20672 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > Attachments: HBASE-20672.branch-1.001.patch > > > Hbase currently provides counter read/write requests (ReadRequestCount, > WriteRequestCount). That said it is not easy to use counter that reset only > after a restart of the service, we would like to expose 2 new metrics in > HBase to provide ReadRequestRate and WriteRequestRate at region server level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20672) Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at every monitoring interval
[ https://issues.apache.org/jira/browse/HBASE-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Jain updated HBASE-20672: --- Attachment: HBASE-20672.branch-1.001.patch > Create new HBase metrics ReadRequestRate and WriteRequestRate that reset at > every monitoring interval > - > > Key: HBASE-20672 > URL: https://issues.apache.org/jira/browse/HBASE-20672 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > Attachments: HBASE-20672.branch-1.001.patch > > > Hbase currently provides counter read/write requests (ReadRequestCount, > WriteRequestCount). That said it is not easy to use counter that reset only > after a restart of the service, we would like to expose 2 new metrics in > HBase to provide ReadRequestRate and WriteRequestRate at region server level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20649) Validate HFiles do not have PREFIX_TREE DataBlockEncoding
[ https://issues.apache.org/jira/browse/HBASE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505077#comment-16505077 ] Peter Somogyi commented on HBASE-20649: --- HFileCorruptionChecker reads the HFile Trailers so it does not depend on the size of the files. Unfortunately I don't have numbers about runtime. I can try to collect some. HBCK uses the same utility when -checkCorruptHFiles is used with 50 threads. > Validate HFiles do not have PREFIX_TREE DataBlockEncoding > - > > Key: HBASE-20649 > URL: https://issues.apache.org/jira/browse/HBASE-20649 > Project: HBase > Issue Type: New Feature >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Minor > Attachments: HBASE-20649.master.001.patch, > HBASE-20649.master.002.patch > > > HBASE-20592 adds a tool to check column families on the cluster do not have > PREFIX_TREE encoding. > Since it is possible that DataBlockEncoding was already changed but HFiles > are not rewritten yet we would need a tool that can verify the content of > hfiles in the cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20703) When quota feature is off shell should give a nice message
[
https://issues.apache.org/jira/browse/HBASE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505078#comment-16505078
]
Xu Cang commented on HBASE-20703:
-
Will do. thanks [~busbey]
> When quota feature is off shell should give a nice message
> --
>
> Key: HBASE-20703
> URL: https://issues.apache.org/jira/browse/HBASE-20703
> Project: HBase
> Issue Type: Improvement
> Components: shell, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20703.master.001.patch
>
>
> When quota is off the shell gives a error that requires knowledge of our
> implementation details to understand:
> {code}
> 2.2.1 :001 > list_snapshot_sizes
> SNAPSHOT SIZE
>
>
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_snapshot_sizes"'
> Took 1.6285 seconds
>
>
> 2.2.1 :002 > list_quota_snapshots
> TABLE USAGE LIMIT IN_VIOLATION POLICY
> ERROR: Unknown table hbase:quota!
> For usage try 'help "list_quota_snapshots"'
> Took 0.0371 seconds
> {code}
> Or it just doesn't mention that quotas can't exist:
> {code}
> 2.2.1 :003 > list_quotas
> OWNERQUOTAS
>
>
> 0 row(s)
> Took 0.0475 seconds
>
>
> 2.2.1 :004 > list_quota_table_sizes
> TABLESIZE
>
>
> 0 row(s)
> Took 0.1221 seconds
> {code}
> set quota gives a better pointer that the problem is the feature is off:
> {code}
> 2.2.1 :005 > set_quota USER => 'busbey', GLOBAL_BYPASS => true
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsupportedOperationException: quota support disabled
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkQuotaSupport(MasterQuotaManager.java:442)
> at
> org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:124)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1555)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.UnsupportedOperationException: quota support disabled
> ... 8 more
> For usage try 'help "set_quota"'
> {code}
> Instead we should give a nice message, like you get if visibility labels is
> off:
> {code}
> 2.2.1 :06 > list_labels
> ERROR: DISABLED: Visibility labels feature is not available
> For usage try 'help "list_labels"'
> Took 0.0426 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
