[jira] [Commented] (HBASE-18759) Fix hbase-shaded-check-invariants failure
[
https://issues.apache.org/jira/browse/HBASE-18759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159768#comment-16159768
]
Hadoop QA commented on HBASE-18759:
---
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at
https://builds.apache.org/job/PreCommit-HBASE-Build/8535/console in case of
problems.
> Fix hbase-shaded-check-invariants failure
> -
>
> Key: HBASE-18759
> URL: https://issues.apache.org/jira/browse/HBASE-18759
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18759.0.patch
>
>
> Here's the error.
> {noformat}
> [ERROR] Found artifact with unexpected contents:
> '/Users/appy/apache/hbase/hbase-shaded/hbase-shaded-server/target/hbase-shaded-server-3.0.0-SNAPSHOT.jar'
> Please check the following and either correct the build or update
> the allowed list with reasoning.
> about.html
> org/eclipse/
> org/eclipse/jetty/
> org/eclipse/jetty/http/
> org/eclipse/jetty/http/encoding.properties
> org/eclipse/jetty/http/GZIPContentDecoder$State.class
> org/eclipse/jetty/http/HttpFields$Itr.class
> org/eclipse/jetty/http/HttpParser$HttpHandler.class
> org/eclipse/jetty/http/HttpScheme.class
> org/eclipse/jetty/http/HttpURI.class
> org/eclipse/jetty/http/pathmap/
> org/eclipse/jetty/http/pathmap/PathMappings$1.class
> org/eclipse/jetty/http/pathmap/PathSpecGroup.class
> org/eclipse/jetty/http/pathmap/PathSpecSet$1.class
> org/eclipse/jetty/http/pathmap/RegexPathSpec.class
> org/eclipse/jetty/http/PathMap$MappedEntry.class
> org/eclipse/jetty/http/QuotedCSV$State.class
> org/eclipse/jetty/http/QuotedQualityCSV$1.class
> org/eclipse/jetty/http/DateParser.class
> org/eclipse/jetty/http/Http1FieldPreEncoder.class
> org/eclipse/jetty/http/HttpContent$ContentFactory.class
> org/eclipse/jetty/http/HttpFields$1.class
> org/eclipse/jetty/http/HttpFields.class
> org/eclipse/jetty/http/HttpStatus$Code.class
> org/eclipse/jetty/http/pathmap/UriTemplatePathSpec.class
> org/eclipse/jetty/http/ResourceHttpContent.class
> org/eclipse/jetty/http/BadMessageException.class
> org/eclipse/jetty/http/GZIPContentDecoder$1.class
> org/eclipse/jetty/http/HttpGenerator$Result.class
> org/eclipse/jetty/http/HttpParser$State.class
> org/eclipse/jetty/http/HttpURI$State.class
> org/eclipse/jetty/http/QuotedCSV.class
> org/eclipse/jetty/http/CompressedContentFormat.class
> org/eclipse/jetty/http/HttpField$LongValueHttpField.class
> org/eclipse/jetty/http/HttpGenerator$2.class
> org/eclipse/jetty/http/HttpMethod.class
> org/eclipse/jetty/http/HttpParser$ComplianceHandler.class
> org/eclipse/jetty/http/HttpParser$RequestHandler.class
> org/eclipse/jetty/http/HttpStatus.class
> org/eclipse/jetty/http/MetaData$Response.class
> org/eclipse/jetty/http/MimeTypes$Type.class
> org/eclipse/jetty/http/pathmap/PathMappings.class
> org/eclipse/jetty/http/PathMap$PathSet.class
> org/eclipse/jetty/http/QuotedQualityCSV.class
> org/eclipse/jetty/http/CookieCompliance.class
> org/eclipse/jetty/http/GZIPContentDecoder.class
> org/eclipse/jetty/http/HttpFields$2.class
> org/eclipse/jetty/http/HttpGenerator$State.class
> org/eclipse/jetty/http/HttpHeader.class
> org/eclipse/jetty/http/HttpParser$IllegalCharacterException.class
> org/eclipse/jetty/http/HttpTokens$EndOfContent.class
> org/eclipse/jetty/http/HttpURI$1.class
> org/eclipse/jetty/http/mime.properties
> org/eclipse/jetty/http/PreEncodedHttpField.class
> org/eclipse/jetty/http/DateGenerator$1.class
> org/eclipse/jetty/http/HostPortHttpField.class
> org/eclipse/jetty/http/HttpCompliance.class
> org/eclipse/jetty/http/HttpFieldPreEncoder.class
> org/eclipse/jetty/http/HttpGenerator$PreparedResponse.class
> org/eclipse/jetty/http/HttpParser$1.class
> org/eclipse/jetty/http/HttpParser$ResponseHandler.class
> org/eclipse/jetty/http/HttpTokens.class
> org/eclipse/jetty/http/HttpVersion.class
> org/eclipse/jetty/http/MetaData.class
> org/eclipse/jetty/http/pathmap/MappedResource.class
> org/eclipse/jetty/http/pathmap/PathSpec.class
> org/eclipse/jetty/http/pathmap/PathSpecSet.class
> org/eclipse/jetty/http/PreEncodedHttpField$1.class
> org/eclipse/jetty/http/QuotedCSV$1.class
> org/eclipse/jetty/http/Syntax.class
> org/eclipse/jetty/http/DateGenerator.class
> org/eclipse/jetty/http/HttpContent.class
> org/eclipse/jetty/http/HttpField$IntValueHttpField.class
> org/eclipse/jetty/http/HttpField.class
>
[jira] [Commented] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[
https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159767#comment-16159767
]
Hadoop QA commented on HBASE-18760:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
4s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
50s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 4m
5s{color} | {color:red} branch has 13 errors when building our shaded client
artifacts. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
5s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 3m
57s{color} | {color:red} patch has 13 errors when building our shaded client
artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 29s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18760 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886226/HBASE-18760.2.patch |
| Optional Tests | asflicense shadedclient shellcheck shelldocs |
| uname | Linux 5143e3dc571d 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 77ca743 |
| shellcheck | v0.4.6 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8534/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Make hbase-shaded-check-invariants part of precommit
>
>
> Key: HBASE-18760
> URL: https://issues.apache.org/jira/browse/HBASE-18760
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Appy
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-18760.0.patch, HBASE-18760.1.patch,
> HBASE-18760.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18759) Fix hbase-shaded-check-invariants failure
[
https://issues.apache.org/jira/browse/HBASE-18759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18759:
Attachment: HBASE-18759.0.patch
-0
- relocate commons-lang 2.y from hadoop
- exclude more jetty / glassfish / etc
also includes a patch for HBASE-18760 to test it.
> Fix hbase-shaded-check-invariants failure
> -
>
> Key: HBASE-18759
> URL: https://issues.apache.org/jira/browse/HBASE-18759
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18759.0.patch
>
>
> Here's the error.
> {noformat}
> [ERROR] Found artifact with unexpected contents:
> '/Users/appy/apache/hbase/hbase-shaded/hbase-shaded-server/target/hbase-shaded-server-3.0.0-SNAPSHOT.jar'
> Please check the following and either correct the build or update
> the allowed list with reasoning.
> about.html
> org/eclipse/
> org/eclipse/jetty/
> org/eclipse/jetty/http/
> org/eclipse/jetty/http/encoding.properties
> org/eclipse/jetty/http/GZIPContentDecoder$State.class
> org/eclipse/jetty/http/HttpFields$Itr.class
> org/eclipse/jetty/http/HttpParser$HttpHandler.class
> org/eclipse/jetty/http/HttpScheme.class
> org/eclipse/jetty/http/HttpURI.class
> org/eclipse/jetty/http/pathmap/
> org/eclipse/jetty/http/pathmap/PathMappings$1.class
> org/eclipse/jetty/http/pathmap/PathSpecGroup.class
> org/eclipse/jetty/http/pathmap/PathSpecSet$1.class
> org/eclipse/jetty/http/pathmap/RegexPathSpec.class
> org/eclipse/jetty/http/PathMap$MappedEntry.class
> org/eclipse/jetty/http/QuotedCSV$State.class
> org/eclipse/jetty/http/QuotedQualityCSV$1.class
> org/eclipse/jetty/http/DateParser.class
> org/eclipse/jetty/http/Http1FieldPreEncoder.class
> org/eclipse/jetty/http/HttpContent$ContentFactory.class
> org/eclipse/jetty/http/HttpFields$1.class
> org/eclipse/jetty/http/HttpFields.class
> org/eclipse/jetty/http/HttpStatus$Code.class
> org/eclipse/jetty/http/pathmap/UriTemplatePathSpec.class
> org/eclipse/jetty/http/ResourceHttpContent.class
> org/eclipse/jetty/http/BadMessageException.class
> org/eclipse/jetty/http/GZIPContentDecoder$1.class
> org/eclipse/jetty/http/HttpGenerator$Result.class
> org/eclipse/jetty/http/HttpParser$State.class
> org/eclipse/jetty/http/HttpURI$State.class
> org/eclipse/jetty/http/QuotedCSV.class
> org/eclipse/jetty/http/CompressedContentFormat.class
> org/eclipse/jetty/http/HttpField$LongValueHttpField.class
> org/eclipse/jetty/http/HttpGenerator$2.class
> org/eclipse/jetty/http/HttpMethod.class
> org/eclipse/jetty/http/HttpParser$ComplianceHandler.class
> org/eclipse/jetty/http/HttpParser$RequestHandler.class
> org/eclipse/jetty/http/HttpStatus.class
> org/eclipse/jetty/http/MetaData$Response.class
> org/eclipse/jetty/http/MimeTypes$Type.class
> org/eclipse/jetty/http/pathmap/PathMappings.class
> org/eclipse/jetty/http/PathMap$PathSet.class
> org/eclipse/jetty/http/QuotedQualityCSV.class
> org/eclipse/jetty/http/CookieCompliance.class
> org/eclipse/jetty/http/GZIPContentDecoder.class
> org/eclipse/jetty/http/HttpFields$2.class
> org/eclipse/jetty/http/HttpGenerator$State.class
> org/eclipse/jetty/http/HttpHeader.class
> org/eclipse/jetty/http/HttpParser$IllegalCharacterException.class
> org/eclipse/jetty/http/HttpTokens$EndOfContent.class
> org/eclipse/jetty/http/HttpURI$1.class
> org/eclipse/jetty/http/mime.properties
> org/eclipse/jetty/http/PreEncodedHttpField.class
> org/eclipse/jetty/http/DateGenerator$1.class
> org/eclipse/jetty/http/HostPortHttpField.class
> org/eclipse/jetty/http/HttpCompliance.class
> org/eclipse/jetty/http/HttpFieldPreEncoder.class
> org/eclipse/jetty/http/HttpGenerator$PreparedResponse.class
> org/eclipse/jetty/http/HttpParser$1.class
> org/eclipse/jetty/http/HttpParser$ResponseHandler.class
> org/eclipse/jetty/http/HttpTokens.class
> org/eclipse/jetty/http/HttpVersion.class
> org/eclipse/jetty/http/MetaData.class
> org/eclipse/jetty/http/pathmap/MappedResource.class
> org/eclipse/jetty/http/pathmap/PathSpec.class
> org/eclipse/jetty/http/pathmap/PathSpecSet.class
> org/eclipse/jetty/http/PreEncodedHttpField$1.class
> org/eclipse/jetty/http/QuotedCSV$1.class
> org/eclipse/jetty/http/Syntax.class
> org/eclipse/jetty/http/DateGenerator.class
> org/eclipse/jetty/http/HttpContent.class
> org/eclipse/jetty/http/HttpField$IntValueHttpField.class
> org/eclipse/jetty/http/HttpField.class
> org/eclipse/jetty/http/HttpGenerator.class
> org/eclipse/jetty/http/MetaData$Request.class
>
[jira] [Updated] (HBASE-18759) Fix hbase-shaded-check-invariants failure
[
https://issues.apache.org/jira/browse/HBASE-18759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18759:
Status: Patch Available (was: In Progress)
> Fix hbase-shaded-check-invariants failure
> -
>
> Key: HBASE-18759
> URL: https://issues.apache.org/jira/browse/HBASE-18759
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18759.0.patch
>
>
> Here's the error.
> {noformat}
> [ERROR] Found artifact with unexpected contents:
> '/Users/appy/apache/hbase/hbase-shaded/hbase-shaded-server/target/hbase-shaded-server-3.0.0-SNAPSHOT.jar'
> Please check the following and either correct the build or update
> the allowed list with reasoning.
> about.html
> org/eclipse/
> org/eclipse/jetty/
> org/eclipse/jetty/http/
> org/eclipse/jetty/http/encoding.properties
> org/eclipse/jetty/http/GZIPContentDecoder$State.class
> org/eclipse/jetty/http/HttpFields$Itr.class
> org/eclipse/jetty/http/HttpParser$HttpHandler.class
> org/eclipse/jetty/http/HttpScheme.class
> org/eclipse/jetty/http/HttpURI.class
> org/eclipse/jetty/http/pathmap/
> org/eclipse/jetty/http/pathmap/PathMappings$1.class
> org/eclipse/jetty/http/pathmap/PathSpecGroup.class
> org/eclipse/jetty/http/pathmap/PathSpecSet$1.class
> org/eclipse/jetty/http/pathmap/RegexPathSpec.class
> org/eclipse/jetty/http/PathMap$MappedEntry.class
> org/eclipse/jetty/http/QuotedCSV$State.class
> org/eclipse/jetty/http/QuotedQualityCSV$1.class
> org/eclipse/jetty/http/DateParser.class
> org/eclipse/jetty/http/Http1FieldPreEncoder.class
> org/eclipse/jetty/http/HttpContent$ContentFactory.class
> org/eclipse/jetty/http/HttpFields$1.class
> org/eclipse/jetty/http/HttpFields.class
> org/eclipse/jetty/http/HttpStatus$Code.class
> org/eclipse/jetty/http/pathmap/UriTemplatePathSpec.class
> org/eclipse/jetty/http/ResourceHttpContent.class
> org/eclipse/jetty/http/BadMessageException.class
> org/eclipse/jetty/http/GZIPContentDecoder$1.class
> org/eclipse/jetty/http/HttpGenerator$Result.class
> org/eclipse/jetty/http/HttpParser$State.class
> org/eclipse/jetty/http/HttpURI$State.class
> org/eclipse/jetty/http/QuotedCSV.class
> org/eclipse/jetty/http/CompressedContentFormat.class
> org/eclipse/jetty/http/HttpField$LongValueHttpField.class
> org/eclipse/jetty/http/HttpGenerator$2.class
> org/eclipse/jetty/http/HttpMethod.class
> org/eclipse/jetty/http/HttpParser$ComplianceHandler.class
> org/eclipse/jetty/http/HttpParser$RequestHandler.class
> org/eclipse/jetty/http/HttpStatus.class
> org/eclipse/jetty/http/MetaData$Response.class
> org/eclipse/jetty/http/MimeTypes$Type.class
> org/eclipse/jetty/http/pathmap/PathMappings.class
> org/eclipse/jetty/http/PathMap$PathSet.class
> org/eclipse/jetty/http/QuotedQualityCSV.class
> org/eclipse/jetty/http/CookieCompliance.class
> org/eclipse/jetty/http/GZIPContentDecoder.class
> org/eclipse/jetty/http/HttpFields$2.class
> org/eclipse/jetty/http/HttpGenerator$State.class
> org/eclipse/jetty/http/HttpHeader.class
> org/eclipse/jetty/http/HttpParser$IllegalCharacterException.class
> org/eclipse/jetty/http/HttpTokens$EndOfContent.class
> org/eclipse/jetty/http/HttpURI$1.class
> org/eclipse/jetty/http/mime.properties
> org/eclipse/jetty/http/PreEncodedHttpField.class
> org/eclipse/jetty/http/DateGenerator$1.class
> org/eclipse/jetty/http/HostPortHttpField.class
> org/eclipse/jetty/http/HttpCompliance.class
> org/eclipse/jetty/http/HttpFieldPreEncoder.class
> org/eclipse/jetty/http/HttpGenerator$PreparedResponse.class
> org/eclipse/jetty/http/HttpParser$1.class
> org/eclipse/jetty/http/HttpParser$ResponseHandler.class
> org/eclipse/jetty/http/HttpTokens.class
> org/eclipse/jetty/http/HttpVersion.class
> org/eclipse/jetty/http/MetaData.class
> org/eclipse/jetty/http/pathmap/MappedResource.class
> org/eclipse/jetty/http/pathmap/PathSpec.class
> org/eclipse/jetty/http/pathmap/PathSpecSet.class
> org/eclipse/jetty/http/PreEncodedHttpField$1.class
> org/eclipse/jetty/http/QuotedCSV$1.class
> org/eclipse/jetty/http/Syntax.class
> org/eclipse/jetty/http/DateGenerator.class
> org/eclipse/jetty/http/HttpContent.class
> org/eclipse/jetty/http/HttpField$IntValueHttpField.class
> org/eclipse/jetty/http/HttpField.class
> org/eclipse/jetty/http/HttpGenerator.class
> org/eclipse/jetty/http/MetaData$Request.class
> org/eclipse/jetty/http/MimeTypes.class
> org/eclipse/jetty/http/pathmap/ServletPathSpec$1.class
>
[jira] [Commented] (HBASE-18782) Module untangling work
[
https://issues.apache.org/jira/browse/HBASE-18782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159762#comment-16159762
]
Hadoop QA commented on HBASE-18782:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-spark-it {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
10s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-spark-it {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 3s{color}
| {color:red} hbase-shell in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
56s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
3s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
13s{color} | {color:green} hbase-external-blockcache in the patch passed.
{color} |
|
[jira] [Work started] (HBASE-18759) Fix hbase-shaded-check-invariants failure
[
https://issues.apache.org/jira/browse/HBASE-18759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-18759 started by Sean Busbey.
---
> Fix hbase-shaded-check-invariants failure
> -
>
> Key: HBASE-18759
> URL: https://issues.apache.org/jira/browse/HBASE-18759
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0-alpha-3
>
>
> Here's the error.
> {noformat}
> [ERROR] Found artifact with unexpected contents:
> '/Users/appy/apache/hbase/hbase-shaded/hbase-shaded-server/target/hbase-shaded-server-3.0.0-SNAPSHOT.jar'
> Please check the following and either correct the build or update
> the allowed list with reasoning.
> about.html
> org/eclipse/
> org/eclipse/jetty/
> org/eclipse/jetty/http/
> org/eclipse/jetty/http/encoding.properties
> org/eclipse/jetty/http/GZIPContentDecoder$State.class
> org/eclipse/jetty/http/HttpFields$Itr.class
> org/eclipse/jetty/http/HttpParser$HttpHandler.class
> org/eclipse/jetty/http/HttpScheme.class
> org/eclipse/jetty/http/HttpURI.class
> org/eclipse/jetty/http/pathmap/
> org/eclipse/jetty/http/pathmap/PathMappings$1.class
> org/eclipse/jetty/http/pathmap/PathSpecGroup.class
> org/eclipse/jetty/http/pathmap/PathSpecSet$1.class
> org/eclipse/jetty/http/pathmap/RegexPathSpec.class
> org/eclipse/jetty/http/PathMap$MappedEntry.class
> org/eclipse/jetty/http/QuotedCSV$State.class
> org/eclipse/jetty/http/QuotedQualityCSV$1.class
> org/eclipse/jetty/http/DateParser.class
> org/eclipse/jetty/http/Http1FieldPreEncoder.class
> org/eclipse/jetty/http/HttpContent$ContentFactory.class
> org/eclipse/jetty/http/HttpFields$1.class
> org/eclipse/jetty/http/HttpFields.class
> org/eclipse/jetty/http/HttpStatus$Code.class
> org/eclipse/jetty/http/pathmap/UriTemplatePathSpec.class
> org/eclipse/jetty/http/ResourceHttpContent.class
> org/eclipse/jetty/http/BadMessageException.class
> org/eclipse/jetty/http/GZIPContentDecoder$1.class
> org/eclipse/jetty/http/HttpGenerator$Result.class
> org/eclipse/jetty/http/HttpParser$State.class
> org/eclipse/jetty/http/HttpURI$State.class
> org/eclipse/jetty/http/QuotedCSV.class
> org/eclipse/jetty/http/CompressedContentFormat.class
> org/eclipse/jetty/http/HttpField$LongValueHttpField.class
> org/eclipse/jetty/http/HttpGenerator$2.class
> org/eclipse/jetty/http/HttpMethod.class
> org/eclipse/jetty/http/HttpParser$ComplianceHandler.class
> org/eclipse/jetty/http/HttpParser$RequestHandler.class
> org/eclipse/jetty/http/HttpStatus.class
> org/eclipse/jetty/http/MetaData$Response.class
> org/eclipse/jetty/http/MimeTypes$Type.class
> org/eclipse/jetty/http/pathmap/PathMappings.class
> org/eclipse/jetty/http/PathMap$PathSet.class
> org/eclipse/jetty/http/QuotedQualityCSV.class
> org/eclipse/jetty/http/CookieCompliance.class
> org/eclipse/jetty/http/GZIPContentDecoder.class
> org/eclipse/jetty/http/HttpFields$2.class
> org/eclipse/jetty/http/HttpGenerator$State.class
> org/eclipse/jetty/http/HttpHeader.class
> org/eclipse/jetty/http/HttpParser$IllegalCharacterException.class
> org/eclipse/jetty/http/HttpTokens$EndOfContent.class
> org/eclipse/jetty/http/HttpURI$1.class
> org/eclipse/jetty/http/mime.properties
> org/eclipse/jetty/http/PreEncodedHttpField.class
> org/eclipse/jetty/http/DateGenerator$1.class
> org/eclipse/jetty/http/HostPortHttpField.class
> org/eclipse/jetty/http/HttpCompliance.class
> org/eclipse/jetty/http/HttpFieldPreEncoder.class
> org/eclipse/jetty/http/HttpGenerator$PreparedResponse.class
> org/eclipse/jetty/http/HttpParser$1.class
> org/eclipse/jetty/http/HttpParser$ResponseHandler.class
> org/eclipse/jetty/http/HttpTokens.class
> org/eclipse/jetty/http/HttpVersion.class
> org/eclipse/jetty/http/MetaData.class
> org/eclipse/jetty/http/pathmap/MappedResource.class
> org/eclipse/jetty/http/pathmap/PathSpec.class
> org/eclipse/jetty/http/pathmap/PathSpecSet.class
> org/eclipse/jetty/http/PreEncodedHttpField$1.class
> org/eclipse/jetty/http/QuotedCSV$1.class
> org/eclipse/jetty/http/Syntax.class
> org/eclipse/jetty/http/DateGenerator.class
> org/eclipse/jetty/http/HttpContent.class
> org/eclipse/jetty/http/HttpField$IntValueHttpField.class
> org/eclipse/jetty/http/HttpField.class
> org/eclipse/jetty/http/HttpGenerator.class
> org/eclipse/jetty/http/MetaData$Request.class
> org/eclipse/jetty/http/MimeTypes.class
> org/eclipse/jetty/http/pathmap/ServletPathSpec$1.class
> org/eclipse/jetty/http/PathMap.class
> org/eclipse/jetty/http/DateParser$1.class
>
[jira] [Commented] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[ https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159747#comment-16159747 ] Hadoop QA commented on HBASE-18760: --- (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HBASE-Build/8534/console in case of problems. > Make hbase-shaded-check-invariants part of precommit > > > Key: HBASE-18760 > URL: https://issues.apache.org/jira/browse/HBASE-18760 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Appy >Assignee: Sean Busbey >Priority: Blocker > Attachments: HBASE-18760.0.patch, HBASE-18760.1.patch, > HBASE-18760.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[ https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18760: Status: Patch Available (was: In Progress) > Make hbase-shaded-check-invariants part of precommit > > > Key: HBASE-18760 > URL: https://issues.apache.org/jira/browse/HBASE-18760 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Appy >Assignee: Sean Busbey >Priority: Blocker > Attachments: HBASE-18760.0.patch, HBASE-18760.1.patch, > HBASE-18760.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[ https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18760: Attachment: HBASE-18760.2.patch -2 - also fix the branch / patch message. > Make hbase-shaded-check-invariants part of precommit > > > Key: HBASE-18760 > URL: https://issues.apache.org/jira/browse/HBASE-18760 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Appy >Assignee: Sean Busbey >Priority: Blocker > Attachments: HBASE-18760.0.patch, HBASE-18760.1.patch, > HBASE-18760.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[ https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18760: Attachment: HBASE-18760.1.patch -1 - forgot the -Prelease flag :p > Make hbase-shaded-check-invariants part of precommit > > > Key: HBASE-18760 > URL: https://issues.apache.org/jira/browse/HBASE-18760 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Appy >Assignee: Sean Busbey >Priority: Blocker > Attachments: HBASE-18760.0.patch, HBASE-18760.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[ https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18760: Status: In Progress (was: Patch Available) > Make hbase-shaded-check-invariants part of precommit > > > Key: HBASE-18760 > URL: https://issues.apache.org/jira/browse/HBASE-18760 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Appy >Assignee: Sean Busbey >Priority: Blocker > Attachments: HBASE-18760.0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[
https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159722#comment-16159722
]
Hadoop QA commented on HBASE-18760:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
4s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
2m 17s{color} | {color:green} The patch has no errors when building our shaded
client artifacts. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
6s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
2m 8s{color} | {color:green} The patch has no errors when building our shaded
client artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18760 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886218/HBASE-18760.0.patch |
| Optional Tests | asflicense shadedclient shellcheck shelldocs |
| uname | Linux af0cd3c3917c 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 77ca743 |
| shellcheck | v0.4.6 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8532/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Make hbase-shaded-check-invariants part of precommit
>
>
> Key: HBASE-18760
> URL: https://issues.apache.org/jira/browse/HBASE-18760
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Appy
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-18760.0.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159721#comment-16159721 ] stack commented on HBASE-16478: --- Have this depend on HBASE-18733 > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159720#comment-16159720 ] stack commented on HBASE-16769: --- Sorry. Took me a while. I know you lads have been in here a while but this one is awful. What we going to do here? All CPs are broke because we relocated PB (as you say above [~anoop.hbase] -- already, a year ago). Let me study what you fellows have written above. POJOs or remove the method altogether it seems is way to go. > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16478) Rename WALKey in PB to WALEdit
[
https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159717#comment-16159717
]
Hadoop QA commented on HBASE-16478:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 14s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
0m 58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 93m
35s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}152m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-16478 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886214/HBASE-16478.master.001.patch
|
| Optional Tests | asflicense cc unit hbaseprotoc javac javadoc findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f1b22fa01a8d 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 77ca743 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
[jira] [Commented] (HBASE-18782) Module untangling work
[ https://issues.apache.org/jira/browse/HBASE-18782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159716#comment-16159716 ] stack commented on HBASE-18782: --- .002 is less ambitious. > Module untangling work > -- > > Key: HBASE-18782 > URL: https://issues.apache.org/jira/browse/HBASE-18782 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Attachments: HBASE-18782.master.001.patch, > HBASE-18782.master.002.patch > > > Here is a bit of module untangling work done using structure 101. In a few > places we depend on modules but are not making use of the dependency. In > other places we depended on a module just because we had one references only, > one we could undo. > After this little exercise, we have nicer layering and stuff like > hbase-protocol and hbase-protocol-shaded no longer depend on other hbase > components. Not everything depends on annotations anymore. > There is way more work to do here but this will do for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18782) Module untangling work
[ https://issues.apache.org/jira/browse/HBASE-18782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18782: -- Attachment: HBASE-18782.master.002.patch > Module untangling work > -- > > Key: HBASE-18782 > URL: https://issues.apache.org/jira/browse/HBASE-18782 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Attachments: HBASE-18782.master.001.patch, > HBASE-18782.master.002.patch > > > Here is a bit of module untangling work done using structure 101. In a few > places we depend on modules but are not making use of the dependency. In > other places we depended on a module just because we had one references only, > one we could undo. > After this little exercise, we have nicer layering and stuff like > hbase-protocol and hbase-protocol-shaded no longer depend on other hbase > components. Not everything depends on annotations anymore. > There is way more work to do here but this will do for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18641) Include block content verification logic used in lruCache in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159714#comment-16159714 ] stack commented on HBASE-18641: --- [~biju74] I read over original issue. Yeah, clashing keys should be rare. Can worry about it for bucket cache and lrucache is we start to see high incidence. On patch, LGTM. Thanks for working on the neglected block cache. > Include block content verification logic used in lruCache in bucketCache > > > Key: HBASE-18641 > URL: https://issues.apache.org/jira/browse/HBASE-18641 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair >Priority: Minor > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18641-branch-1.PATCH, HBASE-18641.PATCH, > HBASE-18641-V1.0.PATCH, HBASE-18641-WIP.PATCH > > > With off-heap/bucketCache being used to cache data blocks without going > through on-heap cache, the logic used in lruCache to check the content of > already cached block need to be included in bucketCache. Please see this > [discussion|https://mail-archives.apache.org/mod_mbox/hbase-dev/201708.mbox/%3cCAO40JLCnXLw3=0bbUaXdDx=w2fklljefvgj6-uvj_2jhfvo...@mail.gmail.com%3e] > for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[ https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159706#comment-16159706 ] Hadoop QA commented on HBASE-18760: --- (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HBASE-Build/8532/console in case of problems. > Make hbase-shaded-check-invariants part of precommit > > > Key: HBASE-18760 > URL: https://issues.apache.org/jira/browse/HBASE-18760 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Appy >Assignee: Sean Busbey >Priority: Blocker > Attachments: HBASE-18760.0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[ https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18760: Attachment: HBASE-18760.0.patch -0 - adds test to our personality that runs verify on the shaded test module always > Make hbase-shaded-check-invariants part of precommit > > > Key: HBASE-18760 > URL: https://issues.apache.org/jira/browse/HBASE-18760 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Appy >Assignee: Sean Busbey >Priority: Blocker > Attachments: HBASE-18760.0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
[ https://issues.apache.org/jira/browse/HBASE-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18760: Status: Patch Available (was: In Progress) > Make hbase-shaded-check-invariants part of precommit > > > Key: HBASE-18760 > URL: https://issues.apache.org/jira/browse/HBASE-18760 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Appy >Assignee: Sean Busbey >Priority: Blocker > Attachments: HBASE-18760.0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[
https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159702#comment-16159702
]
stack commented on HBASE-18298:
---
So what did you did [~anoop.hbase]?
Just call CoprocessorRSServices CoprocessorServices?
We have Stoppable... Could add isStopping to this interface. That'd take care
of isStopping.
Is the getFileSystem in this Interface ever different than that you'd get with
a FileSystem.getFileSytem(Conf)?
The region methods we could add to OnlineRegions?
getServerName seems a bit odd in a CP.
What you think of this CPServices Interface boss? You reckon it makes sense?
This is not right? public CoprocessorRSServices getRegionServerServices() {
> RegionServerServices Interface cleanup for CP expose
>
>
> Key: HBASE-18298
> URL: https://issues.apache.org/jira/browse/HBASE-18298
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18298.patch, HBASE-18298_V2.patch,
> HBASE-18298_V3.patch, HBASE-18298_V4.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17823) Migrate to Apache Yetus Audience Annotations
[ https://issues.apache.org/jira/browse/HBASE-17823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159699#comment-16159699 ] Sean Busbey commented on HBASE-17823: - okay, let me get [~misty]'s docs feedback from the PR in place. > Migrate to Apache Yetus Audience Annotations > > > Key: HBASE-17823 > URL: https://issues.apache.org/jira/browse/HBASE-17823 > Project: HBase > Issue Type: Improvement > Components: API >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-17823.0.patch, HBASE-17823.1.patch > > > Migrate from our own audience annotation handling to apache yetus' > implementation. > [discussion thread on > dev@hbase|https://lists.apache.org/thread.html/5a83d37c9c763b3fc4114231489a073167ac69dbade9774af5ca4fb4@%3Cdev.hbase.apache.org%3E] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17823) Migrate to Apache Yetus Audience Annotations
[ https://issues.apache.org/jira/browse/HBASE-17823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17823: -- Fix Version/s: (was: 2.0.0) 2.0.0-alpha-3 > Migrate to Apache Yetus Audience Annotations > > > Key: HBASE-17823 > URL: https://issues.apache.org/jira/browse/HBASE-17823 > Project: HBase > Issue Type: Improvement > Components: API >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-17823.0.patch, HBASE-17823.1.patch > > > Migrate from our own audience annotation handling to apache yetus' > implementation. > [discussion thread on > dev@hbase|https://lists.apache.org/thread.html/5a83d37c9c763b3fc4114231489a073167ac69dbade9774af5ca4fb4@%3Cdev.hbase.apache.org%3E] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16339) update version of tomcat jasper
[ https://issues.apache.org/jira/browse/HBASE-16339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16339: -- Fix Version/s: (was: 2.0.0) > update version of tomcat jasper > --- > > Key: HBASE-16339 > URL: https://issues.apache.org/jira/browse/HBASE-16339 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey > > our dependency on the jasper-compiler v5.5.23 from the Tomcat project is from > ~8 years ago. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16339) update version of tomcat jasper
[ https://issues.apache.org/jira/browse/HBASE-16339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159694#comment-16159694 ] stack commented on HBASE-16339: --- jspc seems to be an old class where every you go. Our trick w/ anttask precompiling jsps needs updating but still same old jspc. Moving out of 2.0.0 unless alternative. > update version of tomcat jasper > --- > > Key: HBASE-16339 > URL: https://issues.apache.org/jira/browse/HBASE-16339 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey > Fix For: 2.0.0 > > > our dependency on the jasper-compiler v5.5.23 from the Tomcat project is from > ~8 years ago. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
[ https://issues.apache.org/jira/browse/HBASE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159690#comment-16159690 ] stack commented on HBASE-18767: --- Do this HBASE-18648 > Release hbase-2.0.0-alpha-3; Theme "Scrubbed API" > - > > Key: HBASE-18767 > URL: https://issues.apache.org/jira/browse/HBASE-18767 > Project: HBase > Issue Type: Task >Reporter: stack > Fix For: 2.0.0-alpha-3 > > > From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API. > I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/ > the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17730: -- Fix Version/s: (was: 2.0.0) 2.0.0-alpha-4 > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17823) Migrate to Apache Yetus Audience Annotations
[ https://issues.apache.org/jira/browse/HBASE-17823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159689#comment-16159689 ] stack commented on HBASE-17823: --- +1 for hbase2 at least. > Migrate to Apache Yetus Audience Annotations > > > Key: HBASE-17823 > URL: https://issues.apache.org/jira/browse/HBASE-17823 > Project: HBase > Issue Type: Improvement > Components: API >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0 > > Attachments: HBASE-17823.0.patch, HBASE-17823.1.patch > > > Migrate from our own audience annotation handling to apache yetus' > implementation. > [discussion thread on > dev@hbase|https://lists.apache.org/thread.html/5a83d37c9c763b3fc4114231489a073167ac69dbade9774af5ca4fb4@%3Cdev.hbase.apache.org%3E] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18183) Region interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18183: -- Fix Version/s: (was: 2.0.0) 2.0.0-alpha-4 > Region interface cleanup for CP expose > -- > > Key: HBASE-18183 > URL: https://issues.apache.org/jira/browse/HBASE-18183 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18649) Deprecate KV Usage in MR to move to Cells in 3.0
[
https://issues.apache.org/jira/browse/HBASE-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159685#comment-16159685
]
stack commented on HBASE-18649:
---
[~ram_krish] You thinking we commit this to 3.0. What we do for 2.0? Can we do
any deprecations?
Is it correct putting this in method
ExtendedCellWrapper.class
How come we do KVSortReducer in below?
604 } else if
(ExtendedCellWrapper.class.equals(job.getMapOutputValueClass())) {
605 job.setReducerClass(KeyValueSortReducer.class);
Should it be a CellSortReducer?
This stuff internal or external?
job.setOutputValueClass(ExtendedCellWrapper.class);
Can we change this... KeyValueWritableComparable to be CellWC?
We have to do this stuff?
530 } else {
531 kv = new KeyValue(kv.getRowArray(), // row buffer
532 kv.getRowOffset(),// row offset
533 kv.getRowLength(),// row length
534 newCfName,// CF buffer
535 0,// CF offset
536 newCfName.length, // CF length
537 kv.getQualifierArray(), // qualifier buffer
538 kv.getQualifierOffset(), // qualifier offset
539 kv.getQualifierLength(), // qualifier length
540 kv.getTimestamp(),// timestamp
541 KeyValue.Type.codeToType(kv.getTypeByte()), // KV Type
542 kv.getValueArray(), // value buffer
543 kv.getValueOffset(), // value offset
544 kv.getValueLength(), // value length
545 kv.getTagsArray(),// tag buffer
546 kv.getTagsOffset(), // tag offset
547 kv.getTagsLength()); // tag length
548 }
ExtendedCellWrapper the right name? MapReduceCell?
> Deprecate KV Usage in MR to move to Cells in 3.0
>
>
> Key: HBASE-18649
> URL: https://issues.apache.org/jira/browse/HBASE-18649
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Affects Versions: 2.0.0-alpha-2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0, 3.0.0, 2.1.0
>
> Attachments: HBASE-18649_master_2.patch, HBASE-18649_master.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159678#comment-16159678 ] stack commented on HBASE-16478: --- [~enis] Yes sir. It seems to work. Lets push this set of patches home. > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18453) CompactionRequest should not be exposed to user directly
[ https://issues.apache.org/jira/browse/HBASE-18453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159677#comment-16159677 ] stack commented on HBASE-18453: --- Sounds good. Thanks for digging [~Apache9] > CompactionRequest should not be exposed to user directly > > > Key: HBASE-18453 > URL: https://issues.apache.org/jira/browse/HBASE-18453 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > > It is an implementation class. And we need to find another to let user know > the compaction start and end. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159669#comment-16159669 ] Enis Soztutar commented on HBASE-16478: --- bq. I rebased patch and tried reading an old hbase1 WAL with the amended code. It works. Lets get this in. Thanks Stack. This is a rename of PB class, so it should work (the wire format does not depend on the class name). > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18773) Add utility to determine if a TableName is a meta table
[ https://issues.apache.org/jira/browse/HBASE-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159645#comment-16159645 ] stack commented on HBASE-18773: --- Sounds good. Can you corral isMetaTableNameWithSuffix so it is hard-to-find unless you are doing this Read Replica Cluster feature -- put it in a utility class or whatever (no harm if not as long as it has good javadoc... to distingush it from 'normal' function). > Add utility to determine if a TableName is a meta table > --- > > Key: HBASE-18773 > URL: https://issues.apache.org/jira/browse/HBASE-18773 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18773.HBASE-18477.001.patch > > > HBASE-18444 adds a method of specifying a meta table suffix. To continue work > on HBASE-18477, we need a way to determine if a TableName is a meta table. > This patch adds this method and a unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16478: -- Attachment: HBASE-16478.master.001.patch Retry. > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18771) Incorrect StoreFileRefresh leading to split and compaction failures
[ https://issues.apache.org/jira/browse/HBASE-18771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159633#comment-16159633 ] Ashu Pachauri commented on HBASE-18771: --- [~yuzhih...@gmail.com] Yes, we should do this as a follow up of HBASE-18186, i.e. once we get rid of FNFEs that have silently crept in over time due to other changes. I have filed HBASE-18786 to figure out what's the right way to approach this. > Incorrect StoreFileRefresh leading to split and compaction failures > --- > > Key: HBASE-18771 > URL: https://issues.apache.org/jira/browse/HBASE-18771 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Blocker > Fix For: 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18771.branch-1.3.001.patch, > HBASE-18771.branch-1.3.002.patch, HBASE-18771.branch-1.3.003.patch, > HBASE-18771.master.001.patch > > > We ran into issues of compaction and split failures with 1.3 similar to > HBASE-18186 and HBASE-17406. Here's what i believe is happening - > Lets say we have 4 store files that are compacted to form a new one. At this > point we now have 5 store files, however only 1(the newly formed) is open now > for the store and rest are waiting to get archived by HFileArchiver > Now before the files are archived we get a FNFE in a scanner. This results in > HRegion.RegionScannerImpl.handleFileNotFound(FileNotFoundException fnfe) > being called which results in region.refreshStoreFiles(true) -> > HStore.refreshStoreFiles() > HStore.refreshStoreFiles now checks the hdfs dir and adds the previously > compacted files back to the store, however these files are also present in > StoreFileManager's compactedFiles list. Now at this point HFileArchiver runs, > checks compactedFiles list and moves these files into the archive directory. > Now when compaction runs it gets: > 2017-09-04 12:30:13,899 ERROR [ctions-1504505399609] > regionserver.CompactSplitThread - Compaction selection failed regionName = > , storeName = 0, priority = 26, time = 1504528213899 > java.io.FileNotFoundException: File does not exist: hdfs:// > at > org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1337) > at > org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1329) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1329) > at > org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:422) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getReferencedFileStatus(StoreFileInfo.java:342) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getFileStatus(StoreFileInfo.java:355) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getModificationTime(StoreFileInfo.java:360) > at > org.apache.hadoop.hbase.regionserver.StoreFile.getModificationTimeStamp(StoreFile.java:325) > at > org.apache.hadoop.hbase.regionserver.StoreUtils.getLowestTimestamp(StoreUtils.java:63) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.shouldPerformMajorCompaction(RatioBasedCompactionPolicy.java:65) > at > org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy.selectCompaction(SortedCompactionPolicy.java:82) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.select(DefaultStoreEngine.java:107) > at > org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1679) > Similarly if a split happens after archival we fail after PONR while opening > daughter regions due to FNFE. This results in parent offline and daughters > also in a limbo since they're unable to open. Since we get the error after > PONR we also end up aborting the RS. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18786) FileNotFoundException should not be silently handled for primary region replicas
Ashu Pachauri created HBASE-18786: - Summary: FileNotFoundException should not be silently handled for primary region replicas Key: HBASE-18786 URL: https://issues.apache.org/jira/browse/HBASE-18786 Project: HBase Issue Type: Sub-task Reporter: Ashu Pachauri This is a follow up for HBASE-18186. FileNotFoundException while scanning from a primary region replica can be indicative of a more severe problem. Handling them silently can cause many underlying issues go undetected. We should either 1. Hard fail the regionserver if there is a FNFE on a primary region replica, OR 2. Report these exceptions as some region / server level metric so that these can be proactively investigated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16478) Rename WALKey in PB to WALEdit
[
https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159629#comment-16159629
]
Hadoop QA commented on HBASE-16478:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 27s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
0m 53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 42s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}120m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| | org.apache.hadoop.hbase.regionserver.TestRowTooBig |
| | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.client.TestAsyncTableScanAll |
| | org.apache.hadoop.hbase.client.TestMultipleTimestamps |
| | org.apache.hadoop.hbase.client.TestFromClientSide |
| | org.apache.hadoop.hbase.regionserver.wal.TestFSHLog |
| | org.apache.hadoop.hbase.regionserver.TestRemoveRegionMetrics |
| | org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| |
[jira] [Commented] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[
https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159594#comment-16159594
]
xinxin fan commented on HBASE-18090:
[[mailto:Mikhail Antonov]] Thanks for your review!
{quote}Before I go in reviews..opening regions in read-only mode for snapshots
seems reasonable. That change would only affect MR over snapshots codebase or
some other paths too?{quote}
I think the read-only regions only affect MR over snapshots codebase.
{quote} if we set readonly flag we skip replaying WAL and don't create those
tmp files. {quote}
It seem that primary regions even opened in read only mode should replay the
edits, just see HRegion.#initializeRegionInternals:
{code:java}
if (ServerRegionReplicaUtil.shouldReplayRecoveredEdits(this)) {
// Recover any edits if available.
maxSeqId = Math.max(maxSeqId,
replayRecoveredEditsIfAny(this.fs.getRegionDir(), maxSeqIdInStores,
reporter, status));
// Make sure mvcc is up to max.
this.mvcc.advanceTo(maxSeqId);
}
{code}
{quote}Will that work for snapshots created with skipFlush option? Is it always
safe to skip WAL in that case?{quote}
The MR just work on the snapshot store files, so i think it make no different
if the region is read-only or not. How do you think?
> Improve TableSnapshotInputFormat to allow more multiple mappers per region
> --
>
> Key: HBASE-18090
> URL: https://issues.apache.org/jira/browse/HBASE-18090
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Affects Versions: 1.4.0
>Reporter: Mikhail Antonov
>Assignee: xinxin fan
> Attachments: HBASE-18090-branch-1.3-v1.patch,
> HBASE-18090-branch-1.3-v2.patch
>
>
> TableSnapshotInputFormat runs one map task per region in the table snapshot.
> This places unnecessary restriction that the region layout of the original
> table needs to take the processing resources available to MR job into
> consideration. Allowing to run multiple mappers per region (assuming
> reasonably even key distribution) would be useful.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[
https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159594#comment-16159594
]
xinxin fan edited comment on HBASE-18090 at 9/9/17 12:38 AM:
-
Thanks for your review!
{quote}Before I go in reviews..opening regions in read-only mode for snapshots
seems reasonable. That change would only affect MR over snapshots codebase or
some other paths too?{quote}
I think the read-only regions only affect MR over snapshots codebase.
{quote} if we set readonly flag we skip replaying WAL and don't create those
tmp files. {quote}
It seem that primary regions even opened in read only mode should replay the
edits, just see HRegion.#initializeRegionInternals:
{code:java}
if (ServerRegionReplicaUtil.shouldReplayRecoveredEdits(this)) {
// Recover any edits if available.
maxSeqId = Math.max(maxSeqId,
replayRecoveredEditsIfAny(this.fs.getRegionDir(), maxSeqIdInStores,
reporter, status));
// Make sure mvcc is up to max.
this.mvcc.advanceTo(maxSeqId);
}
{code}
{quote}Will that work for snapshots created with skipFlush option? Is it always
safe to skip WAL in that case?{quote}
The MR just work on the snapshot store files, so i think it make no different
if the region is read-only or not. How do you think?
was (Author: xinxin fan):
[[mailto:Mikhail Antonov]] Thanks for your review!
{quote}Before I go in reviews..opening regions in read-only mode for snapshots
seems reasonable. That change would only affect MR over snapshots codebase or
some other paths too?{quote}
I think the read-only regions only affect MR over snapshots codebase.
{quote} if we set readonly flag we skip replaying WAL and don't create those
tmp files. {quote}
It seem that primary regions even opened in read only mode should replay the
edits, just see HRegion.#initializeRegionInternals:
{code:java}
if (ServerRegionReplicaUtil.shouldReplayRecoveredEdits(this)) {
// Recover any edits if available.
maxSeqId = Math.max(maxSeqId,
replayRecoveredEditsIfAny(this.fs.getRegionDir(), maxSeqIdInStores,
reporter, status));
// Make sure mvcc is up to max.
this.mvcc.advanceTo(maxSeqId);
}
{code}
{quote}Will that work for snapshots created with skipFlush option? Is it always
safe to skip WAL in that case?{quote}
The MR just work on the snapshot store files, so i think it make no different
if the region is read-only or not. How do you think?
> Improve TableSnapshotInputFormat to allow more multiple mappers per region
> --
>
> Key: HBASE-18090
> URL: https://issues.apache.org/jira/browse/HBASE-18090
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Affects Versions: 1.4.0
>Reporter: Mikhail Antonov
>Assignee: xinxin fan
> Attachments: HBASE-18090-branch-1.3-v1.patch,
> HBASE-18090-branch-1.3-v2.patch
>
>
> TableSnapshotInputFormat runs one map task per region in the table snapshot.
> This places unnecessary restriction that the region layout of the original
> table needs to take the processing resources available to MR job into
> consideration. Allowing to run multiple mappers per region (assuming
> reasonably even key distribution) would be useful.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18771) Incorrect StoreFileRefresh leading to split and compaction failures
[ https://issues.apache.org/jira/browse/HBASE-18771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159586#comment-16159586 ] Ted Yu commented on HBASE-18771: bq. that HStore#refreshStoreFiles should never be triggered for primary region replicas How about adding check in HStore#refreshStoreFiles() for the above (can be done in another JIRA) ? > Incorrect StoreFileRefresh leading to split and compaction failures > --- > > Key: HBASE-18771 > URL: https://issues.apache.org/jira/browse/HBASE-18771 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Blocker > Fix For: 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18771.branch-1.3.001.patch, > HBASE-18771.branch-1.3.002.patch, HBASE-18771.branch-1.3.003.patch, > HBASE-18771.master.001.patch > > > We ran into issues of compaction and split failures with 1.3 similar to > HBASE-18186 and HBASE-17406. Here's what i believe is happening - > Lets say we have 4 store files that are compacted to form a new one. At this > point we now have 5 store files, however only 1(the newly formed) is open now > for the store and rest are waiting to get archived by HFileArchiver > Now before the files are archived we get a FNFE in a scanner. This results in > HRegion.RegionScannerImpl.handleFileNotFound(FileNotFoundException fnfe) > being called which results in region.refreshStoreFiles(true) -> > HStore.refreshStoreFiles() > HStore.refreshStoreFiles now checks the hdfs dir and adds the previously > compacted files back to the store, however these files are also present in > StoreFileManager's compactedFiles list. Now at this point HFileArchiver runs, > checks compactedFiles list and moves these files into the archive directory. > Now when compaction runs it gets: > 2017-09-04 12:30:13,899 ERROR [ctions-1504505399609] > regionserver.CompactSplitThread - Compaction selection failed regionName = > , storeName = 0, priority = 26, time = 1504528213899 > java.io.FileNotFoundException: File does not exist: hdfs:// > at > org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1337) > at > org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1329) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1329) > at > org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:422) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getReferencedFileStatus(StoreFileInfo.java:342) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getFileStatus(StoreFileInfo.java:355) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getModificationTime(StoreFileInfo.java:360) > at > org.apache.hadoop.hbase.regionserver.StoreFile.getModificationTimeStamp(StoreFile.java:325) > at > org.apache.hadoop.hbase.regionserver.StoreUtils.getLowestTimestamp(StoreUtils.java:63) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.shouldPerformMajorCompaction(RatioBasedCompactionPolicy.java:65) > at > org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy.selectCompaction(SortedCompactionPolicy.java:82) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.select(DefaultStoreEngine.java:107) > at > org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1679) > Similarly if a split happens after archival we fail after PONR while opening > daughter regions due to FNFE. This results in parent offline and daughters > also in a limbo since they're unable to open. Since we get the error after > PONR we also end up aborting the RS. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18750) Cleanup the docs saying "HTable use write buffer"
[
https://issues.apache.org/jira/browse/HBASE-18750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159583#comment-16159583
]
stack commented on HBASE-18750:
---
+1
Change this on commit ... "... don't use BufferedMutator " to "... .does
not use BufferedMutator ..."
Nice.
> Cleanup the docs saying "HTable use write buffer"
> -
>
> Key: HBASE-18750
> URL: https://issues.apache.org/jira/browse/HBASE-18750
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18750.v0.patch
>
>
> Cleanup the docs saying "HTable use write buffer"
> {code}
> Default size of the HTable client write buffer in bytes. A bigger buffer
> takes more memory — on both the client and server side since server
> instantiates the passed write buffer to process it — but a larger buffer size
> reduces the number of RPCs made. For an estimate of server-side memory-used,
> evaluate hbase.client.write.buffer * hbase.regionserver.handler.count
> {code}
> {code}
> Put either adds new rows to a table (if the key is new) or can update
> existing rows (if the key already exists). Puts are executed via Table.put
> (writeBuffer) or Table.batch (non-writeBuffer).
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18771) Incorrect StoreFileRefresh leading to split and compaction failures
[ https://issues.apache.org/jira/browse/HBASE-18771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159582#comment-16159582 ] Ashu Pachauri commented on HBASE-18771: --- On a parallel note, echoing everyone else's opinion here that HStore#refreshStoreFiles should never be triggered for primary region replicas. This not only indicates an underlying problem (FNFE on scan path) but also may cause one to silently overlook some nasty dataloss scenarios. > Incorrect StoreFileRefresh leading to split and compaction failures > --- > > Key: HBASE-18771 > URL: https://issues.apache.org/jira/browse/HBASE-18771 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Blocker > Fix For: 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18771.branch-1.3.001.patch, > HBASE-18771.branch-1.3.002.patch, HBASE-18771.branch-1.3.003.patch, > HBASE-18771.master.001.patch > > > We ran into issues of compaction and split failures with 1.3 similar to > HBASE-18186 and HBASE-17406. Here's what i believe is happening - > Lets say we have 4 store files that are compacted to form a new one. At this > point we now have 5 store files, however only 1(the newly formed) is open now > for the store and rest are waiting to get archived by HFileArchiver > Now before the files are archived we get a FNFE in a scanner. This results in > HRegion.RegionScannerImpl.handleFileNotFound(FileNotFoundException fnfe) > being called which results in region.refreshStoreFiles(true) -> > HStore.refreshStoreFiles() > HStore.refreshStoreFiles now checks the hdfs dir and adds the previously > compacted files back to the store, however these files are also present in > StoreFileManager's compactedFiles list. Now at this point HFileArchiver runs, > checks compactedFiles list and moves these files into the archive directory. > Now when compaction runs it gets: > 2017-09-04 12:30:13,899 ERROR [ctions-1504505399609] > regionserver.CompactSplitThread - Compaction selection failed regionName = > , storeName = 0, priority = 26, time = 1504528213899 > java.io.FileNotFoundException: File does not exist: hdfs:// > at > org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1337) > at > org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1329) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1329) > at > org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:422) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getReferencedFileStatus(StoreFileInfo.java:342) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getFileStatus(StoreFileInfo.java:355) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getModificationTime(StoreFileInfo.java:360) > at > org.apache.hadoop.hbase.regionserver.StoreFile.getModificationTimeStamp(StoreFile.java:325) > at > org.apache.hadoop.hbase.regionserver.StoreUtils.getLowestTimestamp(StoreUtils.java:63) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.shouldPerformMajorCompaction(RatioBasedCompactionPolicy.java:65) > at > org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy.selectCompaction(SortedCompactionPolicy.java:82) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.select(DefaultStoreEngine.java:107) > at > org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1679) > Similarly if a split happens after archival we fail after PONR while opening > daughter regions due to FNFE. This results in parent offline and daughters > also in a limbo since they're unable to open. Since we get the error after > PONR we also end up aborting the RS. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18753) Introduce the unsynchronized TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159575#comment-16159575 ] stack commented on HBASE-18753: --- +1 > Introduce the unsynchronized TimeRangeTracker > - > > Key: HBASE-18753 > URL: https://issues.apache.org/jira/browse/HBASE-18753 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18753.v0.patch, HBASE-18753.v1.patch, > HBASE-18753.v2.patch, HBASE-18753.v3.patch > > > If HBASE-18752 is pushed, it will be better to introduce the unsync > TimeRangeTracker for reducing the cost of recalculation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18771) Incorrect StoreFileRefresh leading to split and compaction failures
[
https://issues.apache.org/jira/browse/HBASE-18771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159571#comment-16159571
]
Ashu Pachauri commented on HBASE-18771:
---
Nice find! The patch looks generally good. There is one nitpick in the tests
though:
{code:java}
+ // Split at this point should not result in the RS being aborted
+
assertEquals(util.getMiniHBaseCluster().getLiveRegionServerThreads().size(), 3);
{code}
The RS abort may not happen every time a split fails depending on where in the
process the split transaction failed (or it rolled back). Also, the RS may come
back up after aborting at the exact same time when this check happens and abort
again. A more robust check is on the number of online regions for the table,
which should be exactly one more than the previously online regions.
> Incorrect StoreFileRefresh leading to split and compaction failures
> ---
>
> Key: HBASE-18771
> URL: https://issues.apache.org/jira/browse/HBASE-18771
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
>Priority: Blocker
> Fix For: 1.4.0, 1.3.2, 1.5.0
>
> Attachments: HBASE-18771.branch-1.3.001.patch,
> HBASE-18771.branch-1.3.002.patch, HBASE-18771.branch-1.3.003.patch,
> HBASE-18771.master.001.patch
>
>
> We ran into issues of compaction and split failures with 1.3 similar to
> HBASE-18186 and HBASE-17406. Here's what i believe is happening -
> Lets say we have 4 store files that are compacted to form a new one. At this
> point we now have 5 store files, however only 1(the newly formed) is open now
> for the store and rest are waiting to get archived by HFileArchiver
> Now before the files are archived we get a FNFE in a scanner. This results in
> HRegion.RegionScannerImpl.handleFileNotFound(FileNotFoundException fnfe)
> being called which results in region.refreshStoreFiles(true) ->
> HStore.refreshStoreFiles()
> HStore.refreshStoreFiles now checks the hdfs dir and adds the previously
> compacted files back to the store, however these files are also present in
> StoreFileManager's compactedFiles list. Now at this point HFileArchiver runs,
> checks compactedFiles list and moves these files into the archive directory.
> Now when compaction runs it gets:
> 2017-09-04 12:30:13,899 ERROR [ctions-1504505399609]
> regionserver.CompactSplitThread - Compaction selection failed regionName =
> , storeName = 0, priority = 26, time = 1504528213899
> java.io.FileNotFoundException: File does not exist: hdfs://
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1337)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1329)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1329)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:422)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.getReferencedFileStatus(StoreFileInfo.java:342)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.getFileStatus(StoreFileInfo.java:355)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.getModificationTime(StoreFileInfo.java:360)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.getModificationTimeStamp(StoreFile.java:325)
> at
> org.apache.hadoop.hbase.regionserver.StoreUtils.getLowestTimestamp(StoreUtils.java:63)
> at
> org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.shouldPerformMajorCompaction(RatioBasedCompactionPolicy.java:65)
> at
> org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy.selectCompaction(SortedCompactionPolicy.java:82)
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.select(DefaultStoreEngine.java:107)
> at
> org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1679)
> Similarly if a split happens after archival we fail after PONR while opening
> daughter regions due to FNFE. This results in parent offline and daughters
> also in a limbo since they're unable to open. Since we get the error after
> PONR we also end up aborting the RS.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18621) Refactor ClusterOptions before applying to code base
[ https://issues.apache.org/jira/browse/HBASE-18621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159574#comment-16159574 ] Hudson commented on HBASE-18621: FAILURE: Integrated in Jenkins build HBase-2.0 #480 (See [https://builds.apache.org/job/HBase-2.0/480/]) HBASE-18621 Refactor ClusterOptions before applying to code base (chia7712: rev 6970c1e9eb650cb9ec4e87433449cabbbe4351aa) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java * (edit) hbase-protocol-shaded/src/main/protobuf/Master.proto * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (edit) hbase-protocol-shaded/src/main/protobuf/ClusterStatus.proto * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * (edit) hbase-protocol/src/main/protobuf/ClusterStatus.proto * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientClusterStatus.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java > Refactor ClusterOptions before applying to code base > > > Key: HBASE-18621 > URL: https://issues.apache.org/jira/browse/HBASE-18621 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18621.master.001.patch, > HBASE-18621.master.002.patch, HBASE-18621.master.003.patch, > HBASE-18621.master.004.patch, HBASE-18621.master.005.patch, > HBASE-18621.master.006.patch, HBASE-18621.master.007.patch, > HBASE-18621.master.008.patch, HBASE-18621.master.009.patch, > HBASE-18621.master.010.patch, HBASE-18621.master.011.patch > > > So far, ClusterStatus.Options is not so clean that can be applied to code > base. > Refactoring it before next move. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18641) Include block content verification logic used in lruCache in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159572#comment-16159572 ] Hudson commented on HBASE-18641: FAILURE: Integrated in Jenkins build HBase-2.0 #480 (See [https://builds.apache.org/job/HBase-2.0/480/]) HBASE-18641 Include block content verification logic used in lruCache in (tedyu: rev 9a50203679f835000ff4df40fd9f601fdee84a77) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCacheUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java > Include block content verification logic used in lruCache in bucketCache > > > Key: HBASE-18641 > URL: https://issues.apache.org/jira/browse/HBASE-18641 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair >Priority: Minor > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18641-branch-1.PATCH, HBASE-18641.PATCH, > HBASE-18641-V1.0.PATCH, HBASE-18641-WIP.PATCH > > > With off-heap/bucketCache being used to cache data blocks without going > through on-heap cache, the logic used in lruCache to check the content of > already cached block need to be included in bucketCache. Please see this > [discussion|https://mail-archives.apache.org/mod_mbox/hbase-dev/201708.mbox/%3cCAO40JLCnXLw3=0bbUaXdDx=w2fklljefvgj6-uvj_2jhfvo...@mail.gmail.com%3e] > for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18106) Redo ProcedureInfo and LockInfo
[ https://issues.apache.org/jira/browse/HBASE-18106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159573#comment-16159573 ] Hudson commented on HBASE-18106: FAILURE: Integrated in Jenkins build HBase-2.0 #480 (See [https://builds.apache.org/job/HBase-2.0/480/]) HBASE-18106 Redo ProcedureInfo and LockInfo (stack: rev c48dc02b7611dca305940b737cbddd07039dd30b) * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureScheduler.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureSuspended.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * (edit) hbase-shell/src/main/ruby/shell/commands/list_procedures.rb * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MobCompactionChore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RecoverMetaProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/locking/TestLockManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/locking/TestLockProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/locking/LockProcedure.java * (edit) hbase-shell/src/main/ruby/hbase/admin.rb * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java * (add) hbase-shell/src/test/ruby/shell/list_procedures_test.rb * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteColumnFamilyProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java * (add) hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufMessageConverter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java * (delete) hbase-common/src/main/java/org/apache/hadoop/hbase/procedure2/LockInfo.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * (delete) hbase-common/src/main/java/org/apache/hadoop/hbase/ProcedureInfo.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RestoreSnapshotProcedure.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java * (edit) hbase-protocol-shaded/src/main/protobuf/Master.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java * (add) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/LockedResourceType.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureDescriber.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestProcedureAdmin.java * (add) hbase-common/src/main/java/org/apache/hadoop/hbase/util/JRubyFormat.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureEvents.java * (edit) hbase-server/src/main/resources/hbase-webapps/master/procedures.jsp * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureReplayOrder.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateNamespaceProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyNamespaceProcedure.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/SimpleProcedureScheduler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/locking/LockManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/shaded/protobuf/TestProtobufUtil.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ProcedureDescriber.java * (edit) hbase-protocol-shaded/src/main/protobuf/LockService.proto * (edit) hbase-shell/src/main/ruby/shell/commands/list_locks.rb * (add)
[jira] [Commented] (HBASE-18750) Cleanup the docs saying "HTable use write buffer"
[
https://issues.apache.org/jira/browse/HBASE-18750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159566#comment-16159566
]
Hadoop QA commented on HBASE-18750:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
48s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 9s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
30s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}110m 40s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}184m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestBlockEvictionFromClient |
| | hadoop.hbase.security.token.TestZKSecretWatcher |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18750 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886166/HBASE-18750.v0.patch |
| Optional Tests | asflicense javac javadoc unit xml |
| uname | Linux cff8b415ec20 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 77ca743 |
| Default Java | 1.8.0_144 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/8527/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8527/testReport/ |
| modules | C: hbase-common . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8527/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Cleanup the docs saying "HTable use write buffer"
> -
>
> Key: HBASE-18750
> URL: https://issues.apache.org/jira/browse/HBASE-18750
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18750.v0.patch
>
>
> Cleanup the docs saying "HTable use write buffer"
> {code}
> Default size of the HTable client write buffer in bytes. A bigger buffer
> takes
[jira] [Commented] (HBASE-18621) Refactor ClusterOptions before applying to code base
[ https://issues.apache.org/jira/browse/HBASE-18621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159565#comment-16159565 ] Hudson commented on HBASE-18621: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3682 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3682/]) HBASE-18621 Refactor ClusterOptions before applying to code base (chia7712: rev 77ca743d095de11bbe30cdc230d8198f0c3d7d4d) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java * (edit) hbase-protocol-shaded/src/main/protobuf/Master.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * (edit) hbase-protocol-shaded/src/main/protobuf/ClusterStatus.proto * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientClusterStatus.java * (edit) hbase-protocol/src/main/protobuf/ClusterStatus.proto > Refactor ClusterOptions before applying to code base > > > Key: HBASE-18621 > URL: https://issues.apache.org/jira/browse/HBASE-18621 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18621.master.001.patch, > HBASE-18621.master.002.patch, HBASE-18621.master.003.patch, > HBASE-18621.master.004.patch, HBASE-18621.master.005.patch, > HBASE-18621.master.006.patch, HBASE-18621.master.007.patch, > HBASE-18621.master.008.patch, HBASE-18621.master.009.patch, > HBASE-18621.master.010.patch, HBASE-18621.master.011.patch > > > So far, ClusterStatus.Options is not so clean that can be applied to code > base. > Refactoring it before next move. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18641) Include block content verification logic used in lruCache in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159564#comment-16159564 ] Hudson commented on HBASE-18641: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3682 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3682/]) HBASE-18641 Include block content verification logic used in lruCache in (tedyu: rev af2157242759e5e50ddeab3c1c6b03b41080372f) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCacheUtil.java > Include block content verification logic used in lruCache in bucketCache > > > Key: HBASE-18641 > URL: https://issues.apache.org/jira/browse/HBASE-18641 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair >Priority: Minor > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18641-branch-1.PATCH, HBASE-18641.PATCH, > HBASE-18641-V1.0.PATCH, HBASE-18641-WIP.PATCH > > > With off-heap/bucketCache being used to cache data blocks without going > through on-heap cache, the logic used in lruCache to check the content of > already cached block need to be included in bucketCache. Please see this > [discussion|https://mail-archives.apache.org/mod_mbox/hbase-dev/201708.mbox/%3cCAO40JLCnXLw3=0bbUaXdDx=w2fklljefvgj6-uvj_2jhfvo...@mail.gmail.com%3e] > for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12556) Create a golden file for testing client API source/binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159559#comment-16159559 ] stack commented on HBASE-12556: --- Moving out. Discussion unresolved. JACC has annotation support now, at least in 2.3 -- we ship 2.1 at mo (though I'm not sure it is the same as @busbey desires in his first comment above). Sounds like the 'golden file' maker would be useful and a process run at build time doing compares ([~dimaspivak] suggestion). This is not being worked on so doesn't look like it will make 2.0.0. Moving it out. Meantime I did the old-school manual compare over in HBASE-18622 > Create a golden file for testing client API source/binary compatibility > --- > > Key: HBASE-12556 > URL: https://issues.apache.org/jira/browse/HBASE-12556 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-12556-wip.patch > > > [~lhofhansl] had a suggestion (in some other jira I forgot) for doing a > golden file for the HBase API so that we can compare between releases to > ensure that we are keeping source and binary compatibility as defined in this > document : > https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit > > I think we can generate a file, commit it to the repo, and create a unit test > to check whether any API's are broken. Adding new InterfaceAudience.Public > interfaces has to modify this file so that it becomes an explicit decision. > The downside is that we have to modify the file every time we add a new API, > but it should be fine since it will force us to think more before committing > to supporting new interfaces within the same major versions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-12556) Create a golden file for testing client API source/binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12556: -- Fix Version/s: (was: 2.0.0-alpha-3) > Create a golden file for testing client API source/binary compatibility > --- > > Key: HBASE-12556 > URL: https://issues.apache.org/jira/browse/HBASE-12556 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-12556-wip.patch > > > [~lhofhansl] had a suggestion (in some other jira I forgot) for doing a > golden file for the HBase API so that we can compare between releases to > ensure that we are keeping source and binary compatibility as defined in this > document : > https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit > > I think we can generate a file, commit it to the repo, and create a unit test > to check whether any API's are broken. Adding new InterfaceAudience.Public > interfaces has to modify this file so that it becomes an explicit decision. > The downside is that we have to modify the file every time we add a new API, > but it should be fine since it will force us to think more before committing > to supporting new interfaces within the same major versions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[
https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159543#comment-16159543
]
xinxin fan commented on HBASE-18090:
{quote}This is a relatively large patch.
Can you create patch for master branch and put it on review board ?{quote}
Sure, i will create patch for master soon
> Improve TableSnapshotInputFormat to allow more multiple mappers per region
> --
>
> Key: HBASE-18090
> URL: https://issues.apache.org/jira/browse/HBASE-18090
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Affects Versions: 1.4.0
>Reporter: Mikhail Antonov
>Assignee: xinxin fan
> Attachments: HBASE-18090-branch-1.3-v1.patch,
> HBASE-18090-branch-1.3-v2.patch
>
>
> TableSnapshotInputFormat runs one map task per region in the table snapshot.
> This places unnecessary restriction that the region layout of the original
> table needs to take the processing resources available to MR job into
> consideration. Allowing to run multiple mappers per region (assuming
> reasonably even key distribution) would be useful.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159539#comment-16159539 ] Hudson commented on HBASE-18652: FAILURE: Integrated in Jenkins build HBase-1.4 #910 (See [https://builds.apache.org/job/HBase-1.4/910/]) HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (tedyu: rev 59312575c81c029cac437d5dce8458869a3a3b3d) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH, > HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159531#comment-16159531 ] Hudson commented on HBASE-18652: FAILURE: Integrated in Jenkins build HBase-1.5 #54 (See [https://builds.apache.org/job/HBase-1.5/54/]) HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (tedyu: rev b6a12c843e3eca5b90065a6d045df77fd7164494) * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH, > HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18732) [compat 1-2] HBASE-14047 removed Cell methods w/o deprecation cycle
[ https://issues.apache.org/jira/browse/HBASE-18732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-18732. --- Resolution: Not A Problem Resolving as 'not a problem' after all. > [compat 1-2] HBASE-14047 removed Cell methods w/o deprecation cycle > --- > > Key: HBASE-18732 > URL: https://issues.apache.org/jira/browse/HBASE-18732 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack > Fix For: 2.0.0-alpha-3 > > > This might be ok. We should just eat this. Cell was useless in 1.0.0. Opening > this just so we have a bit of a discussion about it. See HBASE-14047 where > removal was done. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18732) [compat 1-2] HBASE-14047 removed Cell methods w/o deprecation cycle
[ https://issues.apache.org/jira/browse/HBASE-18732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159523#comment-16159523 ] stack commented on HBASE-18732: --- Thanks boys. Excellent. [~ashish singhi] We are ok. The deprecation for getMvccVersion went in before 1.0.0 was release so it has been deprecated for the life of a major version; it is safe to purge in 2.0.0. Thanks both for doing the research here. I made note up in incompat doc. > [compat 1-2] HBASE-14047 removed Cell methods w/o deprecation cycle > --- > > Key: HBASE-18732 > URL: https://issues.apache.org/jira/browse/HBASE-18732 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack > Fix For: 2.0.0-alpha-3 > > > This might be ok. We should just eat this. Cell was useless in 1.0.0. Opening > this just so we have a bit of a discussion about it. See HBASE-14047 where > removal was done. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18733) [compat 1-2] Hide WALKey
[ https://issues.apache.org/jira/browse/HBASE-18733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159517#comment-16159517 ] stack commented on HBASE-18733: --- FYI [~busbey], would be good to get HBASE-16473 in too... If you think we should shutdown WALKey access, I can disappear it... using the [~enis] patches > [compat 1-2] Hide WALKey > > > Key: HBASE-18733 > URL: https://issues.apache.org/jira/browse/HBASE-18733 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-3 > > > It has all the below issues and it uses our internal PB. See head of class > where it says it should go away... > hbase-server-1.0.0.jar, WALEdit.class > package org.apache.hadoop.hbase.regionserver.wal > [−] WALEdit.getCompaction ( Cell kv ) [static] : > WALProtos.CompactionDescriptor 1 > org/apache/hadoop/hbase/regionserver/wal/WALEdit.getCompaction:(Lorg/apache/hadoop/hbase/Cell;)Lorg/apache/hadoop/hbase/protobuf/generated/WALProtos$CompactionDescriptor; > ChangeEffect > 1 Return value type has been changed from > org.apache.hadoop.hbase.protobuf.generated.WALProtos.CompactionDescriptor to > org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor. > This method has been removed because the return type is part of the method > signature. A client program may be interrupted by NoSuchMethodError exception. > [+] WALEdit.getFlushDescriptor ( Cell cell ) [static] : > WALProtos.FlushDescriptor 1 > [+] WALEdit.getRegionEventDescriptor ( Cell cell ) [static] : > WALProtos.RegionEventDescriptor 1 > hbase-server-1.0.0.jar, WALKey.class > package org.apache.hadoop.hbase.wal > [+] WALKey.getBuilder ( WALCellCodec.ByteStringCompressor compressor ) : > WALProtos.WALKey.Builder 1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16478: -- Hadoop Flags: Reviewed Status: Patch Available (was: Open) I rebased patch and tried reading an old hbase1 WAL with the amended code. It works. Lets get this in. > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16478: -- Attachment: HBASE-16478.master.001.patch > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18727) [C++] Don't link JNI/JVM stuff into libHbaseClient
[
https://issues.apache.org/jira/browse/HBASE-18727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159466#comment-16159466
]
Sudeep Sunthankar commented on HBASE-18727:
---
Hi [~enis],
The changes look good. Couple of points:-
# test-util directory consists code related to standalone and mini cluster.
Instead of excluding the sources in EXCLUDE_SRC we can remove the test-util
folder from the MODULES variable itself as follows:
{code}
MODULES := connection core exceptions security serde utils
{code}
# Also I have noticed that we have a unit-test by the name of
connection/rpc-test-server-handler.cc. We should either change the below
variable as follows to exclude it from the library or rename the file.
{code}
EXCLUDE_SRC := $(foreach sdir,$(SRC_DIR),$(wildcard $(sdir)/*-test*.cc))
{code}
Thanks.
> [C++] Don't link JNI/JVM stuff into libHbaseClient
> --
>
> Key: HBASE-18727
> URL: https://issues.apache.org/jira/browse/HBASE-18727
> Project: HBase
> Issue Type: Sub-task
>Reporter: Scott Hunt
>Priority: Minor
> Fix For: HBASE-14850
>
> Attachments: hbase-18727-v2.patch, Makefile_HBASE_18727.patch
>
>
> It doesn't seem like we should be linking the objects from the test-util
> directory into the library, as those [I believe] are just for testing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-13868) Correct "Disable automatic splitting" section in HBase book
[
https://issues.apache.org/jira/browse/HBASE-13868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159459#comment-16159459
]
Enis Soztutar commented on HBASE-13868:
---
+1.
> Correct "Disable automatic splitting" section in HBase book
> ---
>
> Key: HBASE-13868
> URL: https://issues.apache.org/jira/browse/HBASE-13868
> Project: HBase
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Chia-Ping Tsai
>Priority: Trivial
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-13868.v0.patch
>
>
> This recommendation is not correct for
> *IncreasingToUpperBoundRegionSplitPolicy* (which is default now)
> {quote}
> Disable Automatic Splitting
> To disable automatic splitting, set hbase.hregion.max.filesize to a very
> large value, such as 100 GB It is not recommended to set it to its absolute
> maximum value of Long.MAX_VALUE.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18783) Declare the builder of ClusterStatus as IA.Private; Remove the Writables from ClusterStatus
[
https://issues.apache.org/jira/browse/HBASE-18783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159446#comment-16159446
]
Hadoop QA commented on HBASE-18783:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 18s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
41s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18783 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886164/HBASE-18783.v0.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux b6704ca225ea 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 77ca743 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8528/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8528/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Declare the builder of ClusterStatus as IA.Private; Remove the Writables from
> ClusterStatus
> ---
>
> Key: HBASE-18783
> URL: https://issues.apache.org/jira/browse/HBASE-18783
> Project: HBase
> Issue
[jira] [Commented] (HBASE-18753) Introduce the unsynchronized TimeRangeTracker
[
https://issues.apache.org/jira/browse/HBASE-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159441#comment-16159441
]
Hadoop QA commented on HBASE-18753:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
40m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
41s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}118m
1s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
28s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}214m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886135/HBASE-18753.v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 8a2f598be025 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / af21572 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8524/testReport/ |
| modules | C: hbase-common hbase-server hbase-mapreduce U: . |
| Console output |
[jira] [Updated] (HBASE-17106) Wrap arguments to Coprocessor method invocations in Context objects
[
https://issues.apache.org/jira/browse/HBASE-17106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17106:
--
Fix Version/s: (was: 2.0.0)
2.0.0-alpha-4
> Wrap arguments to Coprocessor method invocations in Context objects
> ---
>
> Key: HBASE-17106
> URL: https://issues.apache.org/jira/browse/HBASE-17106
> Project: HBase
> Issue Type: New Feature
>Reporter: Enis Soztutar
> Fix For: 2.0.0-alpha-4
>
>
> As discussed in various contexts (and recently in
> https://issues.apache.org/jira/browse/HBASE-16962?focusedCommentId=15648512=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15648512)
> we have a very large number of deprecated methods in RegionObserver (and
> possibly others) which are due to the fact that the method signatures like:
> {code}
> @Deprecated
> InternalScanner preFlushScannerOpen(final
> ObserverContext c,
> final Store store, final KeyValueScanner memstoreScanner, final
> InternalScanner s)
> throws IOException;
> {code}
> depend inherently on the internal method signatures which gets changed
> frequently.
> We should look into wrapping the method arguments for such declerations in
> the RegionObserver interface so that we can evolve and add new arguments
> without breaking existing coprocessors between minor versions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[ https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159428#comment-16159428 ] Mikhail Antonov commented on HBASE-18090: - So i reviewed the v2; looks good and actually 1.3 line compatibly at the glance..will give another look once it's up on the review board. > Improve TableSnapshotInputFormat to allow more multiple mappers per region > -- > > Key: HBASE-18090 > URL: https://issues.apache.org/jira/browse/HBASE-18090 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 1.4.0 >Reporter: Mikhail Antonov >Assignee: xinxin fan > Attachments: HBASE-18090-branch-1.3-v1.patch, > HBASE-18090-branch-1.3-v2.patch > > > TableSnapshotInputFormat runs one map task per region in the table snapshot. > This places unnecessary restriction that the region layout of the original > table needs to take the processing resources available to MR job into > consideration. Allowing to run multiple mappers per region (assuming > reasonably even key distribution) would be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-13868) Correct "Disable automatic splitting" section in HBase book
[
https://issues.apache.org/jira/browse/HBASE-13868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-13868:
---
Status: Patch Available (was: Open)
> Correct "Disable automatic splitting" section in HBase book
> ---
>
> Key: HBASE-13868
> URL: https://issues.apache.org/jira/browse/HBASE-13868
> Project: HBase
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Chia-Ping Tsai
>Priority: Trivial
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-13868.v0.patch
>
>
> This recommendation is not correct for
> *IncreasingToUpperBoundRegionSplitPolicy* (which is default now)
> {quote}
> Disable Automatic Splitting
> To disable automatic splitting, set hbase.hregion.max.filesize to a very
> large value, such as 100 GB It is not recommended to set it to its absolute
> maximum value of Long.MAX_VALUE.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-13868) Correct "Disable automatic splitting" section in HBase book
[
https://issues.apache.org/jira/browse/HBASE-13868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-13868:
---
Attachment: HBASE-13868.v0.patch
> Correct "Disable automatic splitting" section in HBase book
> ---
>
> Key: HBASE-13868
> URL: https://issues.apache.org/jira/browse/HBASE-13868
> Project: HBase
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Chia-Ping Tsai
>Priority: Trivial
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-13868.v0.patch
>
>
> This recommendation is not correct for
> *IncreasingToUpperBoundRegionSplitPolicy* (which is default now)
> {quote}
> Disable Automatic Splitting
> To disable automatic splitting, set hbase.hregion.max.filesize to a very
> large value, such as 100 GB It is not recommended to set it to its absolute
> maximum value of Long.MAX_VALUE.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18785) Blocker doc issues
stack created HBASE-18785: - Summary: Blocker doc issues Key: HBASE-18785 URL: https://issues.apache.org/jira/browse/HBASE-18785 Project: HBase Issue Type: Task Reporter: stack Priority: Blocker Fix For: 2.0.0 Hang blocker documentation issues off this one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
[ https://issues.apache.org/jira/browse/HBASE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159426#comment-16159426 ] stack commented on HBASE-18767: --- bq. The issues which aim to "correct" our docs should be declared as Blocker. For example, incorrect link and wrong guideline. Ok. I made an umbrella issue in meantime under which we can hang 2.0.0 doc blockers. HBASE-18785 > Release hbase-2.0.0-alpha-3; Theme "Scrubbed API" > - > > Key: HBASE-18767 > URL: https://issues.apache.org/jira/browse/HBASE-18767 > Project: HBase > Issue Type: Task >Reporter: stack > Fix For: 2.0.0-alpha-3 > > > From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API. > I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/ > the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18781) Accessing table.jsp directly returns Exception to user.
[ https://issues.apache.org/jira/browse/HBASE-18781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159423#comment-16159423 ] stack commented on HBASE-18781: --- [~gubjanos] So I should apply this and HBASE-17436 to test? Thanks. > Accessing table.jsp directly returns Exception to user. > --- > > Key: HBASE-18781 > URL: https://issues.apache.org/jira/browse/HBASE-18781 > Project: HBase > Issue Type: Bug >Reporter: Janos Gub >Assignee: Janos Gub > Attachments: HBASE-18781.patch > > > Discovered by [~stack] while accessing table.jsp directly. > Moved a function up introduced in > https://issues.apache.org/jira/browse/HBASE-15675. Also changed error message > to be more descriptive. See more on > https://issues.apache.org/jira/browse/HBASE-17436?focusedCommentId=16142040=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16142040 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-13868) Correct "Disable automatic splitting" section in HBase book
[
https://issues.apache.org/jira/browse/HBASE-13868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-13868:
---
Fix Version/s: 2.0.0-alpha-3
> Correct "Disable automatic splitting" section in HBase book
> ---
>
> Key: HBASE-13868
> URL: https://issues.apache.org/jira/browse/HBASE-13868
> Project: HBase
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Chia-Ping Tsai
>Priority: Trivial
> Fix For: 2.0.0-alpha-3
>
>
> This recommendation is not correct for
> *IncreasingToUpperBoundRegionSplitPolicy* (which is default now)
> {quote}
> Disable Automatic Splitting
> To disable automatic splitting, set hbase.hregion.max.filesize to a very
> large value, such as 100 GB It is not recommended to set it to its absolute
> maximum value of Long.MAX_VALUE.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
[ https://issues.apache.org/jira/browse/HBASE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159418#comment-16159418 ] Chia-Ping Tsai commented on HBASE-18767: bq. Should we add a doc blocker on hbase 2? The issues which aim to "correct" our docs should be declared as Blocker. For example, incorrect link and wrong guideline. > Release hbase-2.0.0-alpha-3; Theme "Scrubbed API" > - > > Key: HBASE-18767 > URL: https://issues.apache.org/jira/browse/HBASE-18767 > Project: HBase > Issue Type: Task >Reporter: stack > Fix For: 2.0.0-alpha-3 > > > From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API. > I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/ > the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18784) Use of filesystem that requires hflush / hsync / append / etc should query outputstream capabilities
Sean Busbey created HBASE-18784: --- Summary: Use of filesystem that requires hflush / hsync / append / etc should query outputstream capabilities Key: HBASE-18784 URL: https://issues.apache.org/jira/browse/HBASE-18784 Project: HBase Issue Type: Improvement Components: Filesystem Integration Affects Versions: 2.0.0-alpha-2, 1.4.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Fix For: 2.1.0, 1.5.0 In places where we rely on the underlying filesystem holding up the promises of hflush/hsync (most importantly the WAL), we should use the new interfaces provided by HDFS-11644 to fail loudly when they are not present (e.g. on S3, on EC mounts, etc). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18784) Use of filesystem that requires hflush / hsync / append / etc should query outputstream capabilities
[ https://issues.apache.org/jira/browse/HBASE-18784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159395#comment-16159395 ] Sean Busbey commented on HBASE-18784: - If we make a jira for "move Hadoop 3 to S in the support matrix" then this should be a blocker for that. > Use of filesystem that requires hflush / hsync / append / etc should query > outputstream capabilities > > > Key: HBASE-18784 > URL: https://issues.apache.org/jira/browse/HBASE-18784 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration >Affects Versions: 1.4.0, 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.1.0, 1.5.0 > > > In places where we rely on the underlying filesystem holding up the promises > of hflush/hsync (most importantly the WAL), we should use the new interfaces > provided by HDFS-11644 to fail loudly when they are not present (e.g. on S3, > on EC mounts, etc). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-13868) Correct "Disable automatic splitting" section in HBase book
[
https://issues.apache.org/jira/browse/HBASE-13868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai reassigned HBASE-13868:
--
Assignee: Chia-Ping Tsai
> Correct "Disable automatic splitting" section in HBase book
> ---
>
> Key: HBASE-13868
> URL: https://issues.apache.org/jira/browse/HBASE-13868
> Project: HBase
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Chia-Ping Tsai
>Priority: Trivial
>
> This recommendation is not correct for
> *IncreasingToUpperBoundRegionSplitPolicy* (which is default now)
> {quote}
> Disable Automatic Splitting
> To disable automatic splitting, set hbase.hregion.max.filesize to a very
> large value, such as 100 GB It is not recommended to set it to its absolute
> maximum value of Long.MAX_VALUE.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
[ https://issues.apache.org/jira/browse/HBASE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159380#comment-16159380 ] stack commented on HBASE-18767: --- [~chia7712] You are right. Should we add a doc blocker on hbase 2? > Release hbase-2.0.0-alpha-3; Theme "Scrubbed API" > - > > Key: HBASE-18767 > URL: https://issues.apache.org/jira/browse/HBASE-18767 > Project: HBase > Issue Type: Task >Reporter: stack > Fix For: 2.0.0-alpha-3 > > > From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API. > I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/ > the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18622) Mitigate compatibility concerns between branch-1 and branch-2
[
https://issues.apache.org/jira/browse/HBASE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159371#comment-16159371
]
stack commented on HBASE-18622:
---
InterfaceAudience.Private is saving our butt all over the place.
HColumnDescriptor went from this in 1.0.0
{code}
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class HColumnDescriptor implements WritableComparable
{
{code}
to this in 2.0
{code}
@InterfaceAudience.Public
@Deprecated // remove it in 3.0
public class HColumnDescriptor implements ColumnFamilyDescriptor,
Comparable {
{code}
Figuring this ok.
Otherwise, filed subtasks here and updated the incompatibility section in the
'hbase2 book':
https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.723jjn18p2jr
> Mitigate compatibility concerns between branch-1 and branch-2
> -
>
> Key: HBASE-18622
> URL: https://issues.apache.org/jira/browse/HBASE-18622
> Project: HBase
> Issue Type: Bug
> Components: API
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-alpha-3
>
> Attachments: report.1.2_2.0.html.gz
>
>
> This project is to do what [~apurtell] did in the issue "HBASE-18431 Mitigate
> compatibility concerns between branch-1.3 and branch-1.4" only do it between
> branch-1 and branch-2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
[ https://issues.apache.org/jira/browse/HBASE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159373#comment-16159373 ] Chia-Ping Tsai commented on HBASE-18767: Do we intend to revise reference guide before 2.0 release? There are 100+ issues about the docs. Personally, we will give users a negative impression of the quality if the official documents provide many of stale information... > Release hbase-2.0.0-alpha-3; Theme "Scrubbed API" > - > > Key: HBASE-18767 > URL: https://issues.apache.org/jira/browse/HBASE-18767 > Project: HBase > Issue Type: Task >Reporter: stack > Fix For: 2.0.0-alpha-3 > > > From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API. > I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/ > the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-10944) Remove all kv.getBuffer() and kv.getRow() references existing in the code
[ https://issues.apache.org/jira/browse/HBASE-10944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159365#comment-16159365 ] stack commented on HBASE-10944: --- [~ram_krish] See above comment sir. > Remove all kv.getBuffer() and kv.getRow() references existing in the code > - > > Key: HBASE-10944 > URL: https://issues.apache.org/jira/browse/HBASE-10944 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 1.5.0, 2.0.0-alpha-3 > > > kv.getRow() and kv.getBuffers() are still used in places to form key byte[] > and row byte[]. Removing all such instances including testcases will make > the usage of Cell complete. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reopened HBASE-18652: Biju: >From jenkins build, this caused unit test failure in branch-1. Mind attaching updated patch ? > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH, > HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18718) Document the coprocessor.Export
[ https://issues.apache.org/jira/browse/HBASE-18718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159361#comment-16159361 ] stack commented on HBASE-18718: --- +1 Excellent. On commit, change below The Export can run under the Endpoint or MapReduce. The Export can be run via a Coprocessor Endpoint or MapReduce. > Document the coprocessor.Export > --- > > Key: HBASE-18718 > URL: https://issues.apache.org/jira/browse/HBASE-18718 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, documentation, tooling >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > Attachments: HBASE-18718.v0.patch, HBASE-18718.v0.png, > HBASE-18718.v1.patch, HBASE-18718.v1.patch, HBASE-18718.v1.png > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[ https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159352#comment-16159352 ] Mikhail Antonov commented on HBASE-18090: - heads up to [~apurtell] I'd be interesting to get this in branch-1; possibly in 1.3.2 as it seems backward compatible and fairly isolated change. Thoughts? > Improve TableSnapshotInputFormat to allow more multiple mappers per region > -- > > Key: HBASE-18090 > URL: https://issues.apache.org/jira/browse/HBASE-18090 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 1.4.0 >Reporter: Mikhail Antonov >Assignee: xinxin fan > Attachments: HBASE-18090-branch-1.3-v1.patch, > HBASE-18090-branch-1.3-v2.patch > > > TableSnapshotInputFormat runs one map task per region in the table snapshot. > This places unnecessary restriction that the region layout of the original > table needs to take the processing resources available to MR job into > consideration. Allowing to run multiple mappers per region (assuming > reasonably even key distribution) would be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18750) Cleanup the docs saying "HTable use write buffer"
[
https://issues.apache.org/jira/browse/HBASE-18750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18750:
---
Status: Patch Available (was: Open)
> Cleanup the docs saying "HTable use write buffer"
> -
>
> Key: HBASE-18750
> URL: https://issues.apache.org/jira/browse/HBASE-18750
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18750.v0.patch
>
>
> Cleanup the docs saying "HTable use write buffer"
> {code}
> Default size of the HTable client write buffer in bytes. A bigger buffer
> takes more memory — on both the client and server side since server
> instantiates the passed write buffer to process it — but a larger buffer size
> reduces the number of RPCs made. For an estimate of server-side memory-used,
> evaluate hbase.client.write.buffer * hbase.regionserver.handler.count
> {code}
> {code}
> Put either adds new rows to a table (if the key is new) or can update
> existing rows (if the key already exists). Puts are executed via Table.put
> (writeBuffer) or Table.batch (non-writeBuffer).
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18750) Cleanup the docs saying "HTable use write buffer"
[
https://issues.apache.org/jira/browse/HBASE-18750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18750:
---
Attachment: HBASE-18750.v0.patch
> Cleanup the docs saying "HTable use write buffer"
> -
>
> Key: HBASE-18750
> URL: https://issues.apache.org/jira/browse/HBASE-18750
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18750.v0.patch
>
>
> Cleanup the docs saying "HTable use write buffer"
> {code}
> Default size of the HTable client write buffer in bytes. A bigger buffer
> takes more memory — on both the client and server side since server
> instantiates the passed write buffer to process it — but a larger buffer size
> reduces the number of RPCs made. For an estimate of server-side memory-used,
> evaluate hbase.client.write.buffer * hbase.regionserver.handler.count
> {code}
> {code}
> Put either adds new rows to a table (if the key is new) or can update
> existing rows (if the key already exists). Puts are executed via Table.put
> (writeBuffer) or Table.batch (non-writeBuffer).
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18783) Declare the builder of ClusterStatus as IA.Private; Remove the Writables from ClusterStatus
[ https://issues.apache.org/jira/browse/HBASE-18783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159349#comment-16159349 ] stack commented on HBASE-18783: --- +1 Makes sense to me. > Declare the builder of ClusterStatus as IA.Private; Remove the Writables from > ClusterStatus > --- > > Key: HBASE-18783 > URL: https://issues.apache.org/jira/browse/HBASE-18783 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18783.v0.patch > > > To clarify the usage of ClusterStatus API. > # It makes no sense that user want to create custom ClusterStatus; As a > result, we should declare the builder of ClusterStatus as IA.Private. > # HBASE-6038 removed the write/readFields from ClusterStatus; thus the > Writable is meaningless for ClusterStatus now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18770) We should not allow RegionObserver.preBulkLoadHFile to bypass the default behavior
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18770: -- Fix Version/s: (was: 2.0.0-alpha-3) 2.0.0-alpha-4 > We should not allow RegionObserver.preBulkLoadHFile to bypass the default > behavior > -- > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang > Fix For: 2.0.0-alpha-4 > > > As now we do not allow users to create a StoreFile instance. Users can still > select the files to be bulk loaded by modifying the familyPaths passed in. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18747) Introduce new example and helper classes to tell CP users how to do filtering on scanners
[
https://issues.apache.org/jira/browse/HBASE-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18747:
--
Fix Version/s: (was: 2.0.0-alpha-3)
2.0.0-alpha-4
> Introduce new example and helper classes to tell CP users how to do filtering
> on scanners
> -
>
> Key: HBASE-18747
> URL: https://issues.apache.org/jira/browse/HBASE-18747
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Duo Zhang
> Fix For: 2.0.0-alpha-4
>
>
> Finally we decided that CP users should not have the ability to create
> {{StoreScanner}} or {{StoreFileScanner}}, so it is impossible for them to
> filter out some cells when flush or compaction by simply provide a filter
> when constructing {{StoreScanner}}.
> But I think filtering out some cells is a very important usage for CP users,
> so we need to provide the ability in another way. Theoretically it can be
> done with wrapping an {{InternalScanner}}, but I think we need to give an
> example, or even some helper classes to help CP users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18783) Declare the builder of ClusterStatus as IA.Private; Remove the Writables from ClusterStatus
[ https://issues.apache.org/jira/browse/HBASE-18783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18783: --- Status: Patch Available (was: Open) > Declare the builder of ClusterStatus as IA.Private; Remove the Writables from > ClusterStatus > --- > > Key: HBASE-18783 > URL: https://issues.apache.org/jira/browse/HBASE-18783 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18783.v0.patch > > > To clarify the usage of ClusterStatus API. > # It makes no sense that user want to create custom ClusterStatus; As a > result, we should declare the builder of ClusterStatus as IA.Private. > # HBASE-6038 removed the write/readFields from ClusterStatus; thus the > Writable is meaningless for ClusterStatus now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18783) Declare the builder of ClusterStatus as IA.Private; Remove the Writables from ClusterStatus
[ https://issues.apache.org/jira/browse/HBASE-18783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18783: --- Attachment: HBASE-18783.v0.patch > Declare the builder of ClusterStatus as IA.Private; Remove the Writables from > ClusterStatus > --- > > Key: HBASE-18783 > URL: https://issues.apache.org/jira/browse/HBASE-18783 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18783.v0.patch > > > To clarify the usage of ClusterStatus API. > # It makes no sense that user want to create custom ClusterStatus; As a > result, we should declare the builder of ClusterStatus as IA.Private. > # HBASE-6038 removed the write/readFields from ClusterStatus; thus the > Writable is meaningless for ClusterStatus now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18703) Inconsistent behavior for preBatchMutate in doMiniBatchMutate and processRowsWithLocks
[ https://issues.apache.org/jira/browse/HBASE-18703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18703: -- Fix Version/s: (was: 2.0.0-alpha-3) 2.0.0-alpha-4 > Inconsistent behavior for preBatchMutate in doMiniBatchMutate and > processRowsWithLocks > -- > > Key: HBASE-18703 > URL: https://issues.apache.org/jira/browse/HBASE-18703 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Priority: Critical > Fix For: 2.0.0-alpha-4 > > > In doMiniBatchMutate, the preBatchMutate is called before building WAL, but > in processRowsWithLocks, we suggest the RowProcessor implementation to build > WAL in process method, which is ahead of preBatchMutate. > If a CP modifies the mutations, especially if it removes some cells from the > mutations, then the behavior of processRowsWithLocks is broken. The changes > applied to memstore and WAL will be different. And there is no way to remove > entries from a WALEdit through CP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18169) Coprocessor fix and cleanup before 2.0.0 release
[ https://issues.apache.org/jira/browse/HBASE-18169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18169: -- Fix Version/s: (was: 2.0.0-alpha-3) 2.0.0-alpha-4 > Coprocessor fix and cleanup before 2.0.0 release > > > Key: HBASE-18169 > URL: https://issues.apache.org/jira/browse/HBASE-18169 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 2.0.0-alpha-1 >Reporter: Duo Zhang >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > > As discussed in HBASE-18038. In RegionServerServices, Region and StoreFile > interfaces we expose too many unnecessary methods. We need to find a way to > not expose these methods to CP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16894) Create more than 1 split per region, generalize HBASE-12590
[
https://issues.apache.org/jira/browse/HBASE-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16894:
--
Fix Version/s: (was: 2.0.0-alpha-3)
(was: 3.0.0)
2.0.0
> Create more than 1 split per region, generalize HBASE-12590
> ---
>
> Key: HBASE-16894
> URL: https://issues.apache.org/jira/browse/HBASE-16894
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Fix For: 2.0.0
>
> Attachments: HBASE-12590-v1.patch, ImplementaionAndSomeQuestion.docx
>
>
> A common request from users is to be able to better control how many map
> tasks are created per region. Right now, it is always 1 region = 1 input
> split = 1 map task. Same goes for Spark since it uses the TIF. With region
> sizes as large as 50 GBs, it is desirable to be able to create more than 1
> split per region.
> HBASE-12590 adds a config property for MR jobs to be able to handle skew in
> region sizes. The algorithm is roughly:
> {code}
> If (region size >= average size*ratio) : cut the region into two MR input
> splits
> If (average size <= region size < average size*ratio) : one region as one MR
> input split
> If (sum of several continuous regions size < average size * ratio): combine
> these regions into one MR input split.
> {code}
> Although we can set data skew ratio to be 0.5 or something to abuse
> HBASE-12590 into creating more than 1 split task per region, it is not ideal.
> But there is no way to create more with the patch as it is. For example we
> cannot create more than 2 tasks per region.
> If we want to fix this properly, we should extend the approach in
> HBASE-12590, and make it so that the client can specify the desired num of
> mappers, or desired split size, and the TIF generates the splits based on the
> current region sizes very similar to the algorithm in HBASE-12590, but a more
> generic way. This also would eliminate the hand tuning of data skew ratio.
> We also can think about the guidepost approach that Phoenix has in the stats
> table which is used for exactly this purpose. Right now, the region can be
> split into powers of two assuming uniform distribution within the region.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16769: -- Fix Version/s: (was: 2.0.0-alpha-3) 2.0.0-alpha-4 > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18298: -- Fix Version/s: (was: 2.0.0-alpha-3) 2.0.0-alpha-4 > RegionServerServices Interface cleanup for CP expose > > > Key: HBASE-18298 > URL: https://issues.apache.org/jira/browse/HBASE-18298 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, > HBASE-18298_V3.patch, HBASE-18298_V4.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18446) Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix)
[ https://issues.apache.org/jira/browse/HBASE-18446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18446: -- Fix Version/s: (was: 2.0.0-alpha-3) 2.0.0-alpha-4 > Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix) > --- > > Key: HBASE-18446 > URL: https://issues.apache.org/jira/browse/HBASE-18446 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18446.patch > > > Do not see any reason why it is marked as IA.LimitedPrivate. It is not > referenced in any CPs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18782) Module untangling work
[
https://issues.apache.org/jira/browse/HBASE-18782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159326#comment-16159326
]
Hadoop QA commented on HBASE-18782:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
1s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-spark-it {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
15s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-spark-it {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
20s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 2s{color}
| {color:red} hbase-shell in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 58s{color}
| {color:red} hbase-endpoint in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
48s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
42s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color}
[jira] [Commented] (HBASE-18780) Remove HLogPrettyPrinter and hlog command
[ https://issues.apache.org/jira/browse/HBASE-18780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159315#comment-16159315 ] stack commented on HBASE-18780: --- Tests pass for me locally. And this patch removed an unused file so can't have triggered above test fails. > Remove HLogPrettyPrinter and hlog command > - > > Key: HBASE-18780 > URL: https://issues.apache.org/jira/browse/HBASE-18780 > Project: HBase > Issue Type: Sub-task > Components: documentation, wal >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Labels: beginner > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18780.v0.patch > > > WALPrettyPrinter was substituted for HLogPrettyPrinter. Remove it for 2.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
