[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[
https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293687#comment-16293687
]
Hadoop QA commented on HBASE-19526:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 15m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0-beta1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 57s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19526 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12902483/HBASE-19526.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux 743872c0a31b 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / d276c3275e |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/10490/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10490/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10490/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Update hadoop version to 3.0 GA
> ---
>
> Key: HBASE-19526
> URL: https://issues.apache.org/jira/browse/HBASE-19526
> Project: HBase
> Issue Type: Task
> Components: build, dependencies
>Reporter: Mike Drob
>Assignee: Appy
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19526.master.001.patch,
> HBASE-19526.master.001.patch,
[jira] [Commented] (HBASE-19527) Make ExecutorService threads daemon=true.
[
https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293683#comment-16293683
]
Hadoop QA commented on HBASE-19527:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
6s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
30s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
38s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0-beta1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}105m 22s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}142m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19527 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12902482/HBASE-19527.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 01a5908bc6a4 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d276c3275e |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/10491/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10491/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10491/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was
[jira] [Updated] (HBASE-19522) The complete order may be wrong in AsyncBufferedMutatorImpl
[
https://issues.apache.org/jira/browse/HBASE-19522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-19522:
---
Resolution: Fixed
Fix Version/s: 2.0.0-beta-1
Status: Resolved (was: Patch Available)
Pushed to master and branch-2.
> The complete order may be wrong in AsyncBufferedMutatorImpl
> ---
>
> Key: HBASE-19522
> URL: https://issues.apache.org/jira/browse/HBASE-19522
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19522.master.001.patch,
> HBASE-19522.master.002.patch
>
>
> {code}
> List toComplete = this.futures;
> assert toSend.size() == toComplete.size();
> this.mutations = new ArrayList<>();
> this.futures = new ArrayList<>();
> bufferedSize = 0L;
> Iterator toCompleteIter = toComplete.iterator();
> for (CompletableFuture future : table.batch(toSend)) {
> future.whenComplete((r, e) -> {
> CompletableFuture f = toCompleteIter.next(); // Call next in
> callback, so the complete order may different with the future order
> if (e != null) {
> f.completeExceptionally(e);
> } else {
> f.complete(null);
> }
> });
> }
> {code}
> Here we call table.batch to get a list of CompleteFuture for each mutation.
> Then we register a call back for each future. But the problem is we call
> toCompleteIter.next() in the callback. So we may complete the future by a
> wrong order(not same with the mutation order). Meanwhile, as ArrayList is not
> thread safe, so different thread may get same future by toCompleteIter.next().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19112) Suspect methods on Cell to be deprecated
[
https://issues.apache.org/jira/browse/HBASE-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293672#comment-16293672
]
Chia-Ping Tsai commented on HBASE-19112:
+1 on commit (it seems the patch is stale). More refactor for RawCellBuilder
can be in follow-up.
> Suspect methods on Cell to be deprecated
>
>
> Key: HBASE-19112
> URL: https://issues.apache.org/jira/browse/HBASE-19112
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Josh Elser
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19112_branch-2.patch,
> HBASE-19112_branch-2_1.patch, HBASE-19112_master.patch,
> HBASE-19112_master_1.patch, HBASE-19112_master_1.patch,
> HBASE-19112_master_2.patch, HBASE-19112_master_3.patch
>
>
> [~chia7712] suggested on the [mailing
> list|https://lists.apache.org/thread.html/e6de9af26d9b888a358ba48bf74655ccd893573087c032c0fcf01585@%3Cdev.hbase.apache.org%3E]
> that we have some methods on Cell which should be deprecated for removal:
> * {{#getType()}}
> * {{#getTimestamp()}}
> * {{#getTag()}}
> * {{#getSequenceId()}}
> Let's make a pass over these (and maybe the rest) to make sure that there
> aren't others which are either implementation details or methods returning
> now-private-marked classes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19522) The complete order may be wrong in AsyncBufferedMutatorImpl
[
https://issues.apache.org/jira/browse/HBASE-19522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293671#comment-16293671
]
Guanghao Zhang commented on HBASE-19522:
The failed ut not related. Will commit it shortly.
> The complete order may be wrong in AsyncBufferedMutatorImpl
> ---
>
> Key: HBASE-19522
> URL: https://issues.apache.org/jira/browse/HBASE-19522
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-19522.master.001.patch,
> HBASE-19522.master.002.patch
>
>
> {code}
> List toComplete = this.futures;
> assert toSend.size() == toComplete.size();
> this.mutations = new ArrayList<>();
> this.futures = new ArrayList<>();
> bufferedSize = 0L;
> Iterator toCompleteIter = toComplete.iterator();
> for (CompletableFuture future : table.batch(toSend)) {
> future.whenComplete((r, e) -> {
> CompletableFuture f = toCompleteIter.next(); // Call next in
> callback, so the complete order may different with the future order
> if (e != null) {
> f.completeExceptionally(e);
> } else {
> f.complete(null);
> }
> });
> }
> {code}
> Here we call table.batch to get a list of CompleteFuture for each mutation.
> Then we register a call back for each future. But the problem is we call
> toCompleteIter.next() in the callback. So we may complete the future by a
> wrong order(not same with the mutation order). Meanwhile, as ArrayList is not
> thread safe, so different thread may get same future by toCompleteIter.next().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only
[
https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293669#comment-16293669
]
Duo Zhang commented on HBASE-14790:
---
Let me do it. Thanks the reminder sir [~stack].
> Implement a new DFSOutputStream for logging WAL only
>
>
> Key: HBASE-14790
> URL: https://issues.apache.org/jira/browse/HBASE-14790
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0-beta-1
>
>
> The original {{DFSOutputStream}} is very powerful and aims to serve all
> purposes. But in fact, we do not need most of the features if we only want to
> log WAL. For example, we do not need pipeline recovery since we could just
> close the old logger and open a new one. And also, we do not need to write
> multiple blocks since we could also open a new logger if the old file is too
> large.
> And the most important thing is that, it is hard to handle all the corner
> cases to avoid data loss or data inconsistency(such as HBASE-14004) when
> using original DFSOutputStream due to its complicated logic. And the
> complicated logic also force us to use some magical tricks to increase
> performance. For example, we need to use multiple threads to call {{hflush}}
> when logging, and now we use 5 threads. But why 5 not 10 or 100?
> So here, I propose we should implement our own {{DFSOutputStream}} when
> logging WAL. For correctness, and also for performance.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293666#comment-16293666 ] Hadoop QA commented on HBASE-19526: --- (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HBASE-Build/10492/console in case of problems. > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch, > HBASE-19526.master.001.patch, HBASE-19526.master.002.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19526: - Attachment: HBASE-19526.master.002.patch > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch, > HBASE-19526.master.001.patch, HBASE-19526.master.002.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19530) New regions should always be added with state CLOSED
[ https://issues.apache.org/jira/browse/HBASE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19530: - Resolution: Fixed Assignee: Appy Fix Version/s: 2.0.0-beta-1 Status: Resolved (was: Patch Available) > New regions should always be added with state CLOSED > > > Key: HBASE-19530 > URL: https://issues.apache.org/jira/browse/HBASE-19530 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19530.master.001.patch > > > We shouldn't add regions with state null. In case of failures and recovery, > it's not possible to determine what did it mean and things become uncertain. > All operations should add regions in a well defined state. > For now, we'll set the default to CLOSED, since whatever ops are adding new > regions, they would anyways be enabling them explicitly if needed. > fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19530) New regions should always be added with state CLOSED
[ https://issues.apache.org/jira/browse/HBASE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293663#comment-16293663 ] Appy commented on HBASE-19530: -- Btw, i predict that this patch might make "fix" TestTruncateTableProcedure. I quote fix because, those failures are result of two assumption collectively resulting in a failure (region state = null --> assume OFFLINE, table state = null --> assume ENABLED). This is break the first one of them and test might start passing. But we still need to address the second one, and that will be done in HBASE-19529. > New regions should always be added with state CLOSED > > > Key: HBASE-19530 > URL: https://issues.apache.org/jira/browse/HBASE-19530 > Project: HBase > Issue Type: Bug >Reporter: Appy > Attachments: HBASE-19530.master.001.patch > > > We shouldn't add regions with state null. In case of failures and recovery, > it's not possible to determine what did it mean and things become uncertain. > All operations should add regions in a well defined state. > For now, we'll set the default to CLOSED, since whatever ops are adding new > regions, they would anyways be enabling them explicitly if needed. > fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19530) New regions should always be added with state CLOSED
[ https://issues.apache.org/jira/browse/HBASE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293663#comment-16293663 ] Appy edited comment on HBASE-19530 at 12/16/17 5:26 AM: Btw, i predict that this patch might make "fix" TestTruncateTableProcedure. I quote fix because, those failures are result of two assumption collectively resulting in a failure (region state = null --> assume OFFLINE, table state = null --> assume ENABLED). This will break the first one and test might start passing. But we still need to address the second one, and that will be done in HBASE-19529. was (Author: appy): Btw, i predict that this patch might make "fix" TestTruncateTableProcedure. I quote fix because, those failures are result of two assumption collectively resulting in a failure (region state = null --> assume OFFLINE, table state = null --> assume ENABLED). This is break the first one of them and test might start passing. But we still need to address the second one, and that will be done in HBASE-19529. > New regions should always be added with state CLOSED > > > Key: HBASE-19530 > URL: https://issues.apache.org/jira/browse/HBASE-19530 > Project: HBase > Issue Type: Bug >Reporter: Appy > Attachments: HBASE-19530.master.001.patch > > > We shouldn't add regions with state null. In case of failures and recovery, > it's not possible to determine what did it mean and things become uncertain. > All operations should add regions in a well defined state. > For now, we'll set the default to CLOSED, since whatever ops are adding new > regions, they would anyways be enabling them explicitly if needed. > fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17730: -- Fix Version/s: (was: 2.0.0-beta-1) 2.0.0-beta-2 > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0-beta-2 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293661#comment-16293661 ] stack commented on HBASE-17730: --- This is a doc and recipe issue. Moved to beta2 > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0-beta-2 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18622) Mitigate API compatibility concerns between branch-1 and branch-2
[ https://issues.apache.org/jira/browse/HBASE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18622: -- Fix Version/s: (was: 2.0.0-beta-1) 2.0.0-beta-2 > Mitigate API compatibility concerns between branch-1 and branch-2 > - > > Key: HBASE-18622 > URL: https://issues.apache.org/jira/browse/HBASE-18622 > Project: HBase > Issue Type: Bug > Components: API >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0-beta-2 > > Attachments: report.1.2_2.0.html.gz > > > This project is to do what [~apurtell] did in the issue "HBASE-18431 Mitigate > compatibility concerns between branch-1.3 and branch-1.4" only do it between > branch-1 and branch-2. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19530) New regions should always be added with state CLOSED
[ https://issues.apache.org/jira/browse/HBASE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293660#comment-16293660 ] Appy commented on HBASE-19530: -- Let's try it then! Pushing to master and branch-2. > New regions should always be added with state CLOSED > > > Key: HBASE-19530 > URL: https://issues.apache.org/jira/browse/HBASE-19530 > Project: HBase > Issue Type: Bug >Reporter: Appy > Attachments: HBASE-19530.master.001.patch > > > We shouldn't add regions with state null. In case of failures and recovery, > it's not possible to determine what did it mean and things become uncertain. > All operations should add regions in a well defined state. > For now, we'll set the default to CLOSED, since whatever ops are adding new > regions, they would anyways be enabling them explicitly if needed. > fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19527) Make ExecutorService threads daemon=true.
[ https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293659#comment-16293659 ] Appy commented on HBASE-19527: -- bq. Making the threads it runs be daemon seems fine but there may be some assumptions we're breaking w/ this change That was my biggest concern too. If a clean shutdown meant letting worker threads check for server status at fixed safe/check points and quitting by themselves, that would no longer happen now. They would abruptly end even for a clean server shutdown. For the case of server crash, they would have abruptly ended anyways, so all necessary recovery should already be in place. So all boils down to: Is it fine to abruptly quit whatever work these threads are doing in case of a clean shutdown? You mentioned region open/close. Those are regulated by master and RS are supposed to explicitly ack back success. So abruptly ending them should be fine (since master would detect lack of ack and take actions). What other cases? > Make ExecutorService threads daemon=true. > - > > Key: HBASE-19527 > URL: https://issues.apache.org/jira/browse/HBASE-19527 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19527.master.001.patch, > HBASE-19527.master.001.patch, HBASE-19527.master.001.patch > > > Let me try this. ExecutorService runs OPENs, CLOSE, etc. If Server is going > down, no point in these threads sticking around (I think). Let me try this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19515) Region server left in online servers list forever if it went down after registering to master and before creating ephemeral node
[ https://issues.apache.org/jira/browse/HBASE-19515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293657#comment-16293657 ] ramkrishna.s.vasudevan commented on HBASE-19515: Thanks [~saint@gmail.com]. Got it. > Region server left in online servers list forever if it went down after > registering to master and before creating ephemeral node > > > Key: HBASE-19515 > URL: https://issues.apache.org/jira/browse/HBASE-19515 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > > This one is interesting. It was supposedly fixed long time ago back in > HBASE-9593 (The issue has same subject as this one) but there was a problem > w/ the fix reported later, post-commit, long after the issue was closed. The > 'fix' was registering ephemeral node in ZK BEFORE reporting in to the Master > for the first time. The problem w/ this approach is that the Master tells the > RS what name it should use reporting in. If we register in ZK before we talk > to the Master, the name in ZK and the one the RS ends up using could deviate. > In hbase2, we do the right thing registering the ephemeral node after we > report to the Master. So, the issue reported in HBASE-9593, that a RS that > dies between reporting to master and registering up in ZK, stays registered > at the Master for ever is back; we'll keep trying to assign it regions. Its a > real problem. > That hbase2 has this issue has been suppressed up until now. The test that > was written for HBASE-9593, TestRSKilledWhenInitializing, is a good test but > a little sloppy. It puts up two RSs aborting one only after registering at > the Master before posting to ZK. That leaves one healthy server up. It is > hosting hbase:meta. This is enough for the test to bluster through. The only > assign it does is namespace table. It goes to the hbase:meta server. If the > test created a new table and did roundrobin, it'd fail. > After HBASE-18946, where we do round robin on table create -- a desirable > attribute -- via the balancer so all is kosher, the test > TestRSKilledWhenInitializing now starts to fail because we chose the hobbled > server most of the time. > So, this issue is about fixing the original issue properly for hbase2. We > don't have a timeout on assign in AMv2, not yet, that might be the fix, or > perhaps a double report before we online a server with the second report > coming in after ZK goes up (or we stop doing ephemeral nodes for RS up in ZK > and just rely on heartbeats). > Making this a critical issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293652#comment-16293652 ] Hudson commented on HBASE-19498: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4233 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4233/]) HBASE-19498 Fix findbugs and error-prone warnings in hbase-client (stack: rev 59529a78f039965a4c08805c9087d82d64621c20) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/TestHColumnDescriptor.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestTableDescriptorBuilder.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueExcludeFilter.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/security/TestSaslUtil.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/NotServingRegionException.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestBufferedMutatorParams.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnCountGetFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/DependentColumnFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/RegionLocations.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/CompareFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/RandomRowFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/RowFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FirstKeyOnlyFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/BitComparator.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/security/SaslUtil.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncMasterRequestRpcRetryingCaller.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/BinaryComparator.java * (edit) hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKUtil.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/MultiRowRangeFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnRangeFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FirstKeyValueMatchingQualifiersFilter.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestRegionInfoDisplay.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ValueFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SubstringComparator.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorParams.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterBase.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/InclusiveStopFilter.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientScanner.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ServerName.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/PerClientRandomNonceGenerator.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcClient.java * (edit) hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKLeaderManager.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Increment.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/util/PoolMap.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/InvalidFamilyOperationException.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java * (edit) hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Append.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FamilyFilter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PageFilter.java * (edit)
[jira] [Commented] (HBASE-19497) Fix findbugs and error-prone warnings in hbase-common (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293653#comment-16293653 ] Hudson commented on HBASE-19497: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4233 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4233/]) HBASE-19497 Fix findbugs and error-prone warnings in hbase-common (stack: rev f9f869f60a334982b739420a614b5d44adf26af0) * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/OrderedInt16.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/OrderedFloat64.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/RawShort.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/RawString.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBufferListOutputStream.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/StructIterator.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/TestCellUtil.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/io/TestTagCompressionContext.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/codec/CellCodec.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/CryptoCipherProvider.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/AbstractDataBlockEncoder.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBufferOutputStream.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/TestIndividualBytesFieldCell.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/compress/ReusableStreamGzipCodec.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/TestByteBufferKeyValue.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/CellScanner.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/CompressionState.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/AbstractHBaseToolTest.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileContextBuilder.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/OrderedBlobVar.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/RedundantKVGenerator.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/aes/CommonsCryptoAESEncryptor.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/OrderedInt8.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/aes/CommonsCryptoAESDecryptor.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/RawFloat.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestLoadTestKVGenerator.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/OrderedString.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/OrderedInt64.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestJRubyFormat.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/RetryCounterFactory.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestVersionInfo.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/JSONMetricUtil.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/HasThread.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/nio/TestMultiByteBuff.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/compress/Compression.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/io/crypto/TestEncryption.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileContext.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/ReflectionUtils.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/exceptions/UnexpectedStateException.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/types/StructBuilder.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/aes/AESEncryptor.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/aes/CommonsCryptoAES.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestByteRangeWithKVSerialization.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/NoTagsByteBufferKeyValue.java * (edit)
[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293651#comment-16293651 ] Appy commented on HBASE-19526: -- v2. Updated stale references to older versions in others places too. As long as hadoopcheck gives +1 it should be fine since the default version is 2.7.4 and tests are running against it. I think at this point it makes sense to do nightly tests against hadoop3 so that we can get enough confidence with it sooner rather than later. All it'll require is pushing another branch with 1 line change (say branch-2_exp_hadoop_3). The real issue would be keeping it in sync with branch-2. > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch, > HBASE-19526.master.001.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19148) Reevaluate default values of configurations
[
https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293650#comment-16293650
]
Hadoop QA commented on HBASE-19148:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
3s{color} | {color:red} hbase-server: The patch generated 4 new + 3 unchanged -
0 fixed = 7 total (was 3) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0-beta1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
10s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}107m 40s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}150m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19148 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12902479/0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch
|
| Optional Tests | asflicense javac javadoc unit xml findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 00b25e6374b1 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64
[jira] [Updated] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19526: -- Attachment: HBASE-19526.master.001.patch > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch, > HBASE-19526.master.001.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293639#comment-16293639 ] stack commented on HBASE-19526: --- There is this location too under dev-tools in hbase-personality for yetus... hbase_hadoop3_versions="3.0.0-beta1" The failed test has odd failure... but doesn't seem related.. Let me retry. Stacktrace java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions: Sat Dec 16 01:22:00 UTC 2017, , java.io.IOException: java.io.IOException: offset=-1 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:462) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:325) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:305) Caused by: java.lang.IllegalArgumentException: offset=-1 at org.apache.hadoop.hbase.client.RegionInfo.getTable(RegionInfo.java:309) at org.apache.hadoop.hbase.HRegionInfo.getTable(HRegionInfo.java:432) at org.apache.hadoop.hbase.ClusterStatus.getLastMajorCompactionTsForTable(ClusterStatus.java:324) at org.apache.hadoop.hbase.master.HMaster.getLastMajorCompactionTimestamp(HMaster.java:3174) at org.apache.hadoop.hbase.master.MasterRpcServices.getLastMajorCompactionTimestamp(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:403) ... 3 more > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19527) Make ExecutorService threads daemon=true.
[ https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19527: -- Attachment: HBASE-19527.master.001.patch The failures are random. May be because of this change. Any opinions out there? We put up instances of ExecutorService in loads of places. Making the threads it runs be daemon seems fine but there may be some assumptions we're breaking w/ this change (should be fine though). > Make ExecutorService threads daemon=true. > - > > Key: HBASE-19527 > URL: https://issues.apache.org/jira/browse/HBASE-19527 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19527.master.001.patch, > HBASE-19527.master.001.patch, HBASE-19527.master.001.patch > > > Let me try this. ExecutorService runs OPENs, CLOSE, etc. If Server is going > down, no point in these threads sticking around (I think). Let me try this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19530) New regions should always be added with state CLOSED
[ https://issues.apache.org/jira/browse/HBASE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293634#comment-16293634 ] stack commented on HBASE-19530: --- Patch looks great [~appy] +1 > New regions should always be added with state CLOSED > > > Key: HBASE-19530 > URL: https://issues.apache.org/jira/browse/HBASE-19530 > Project: HBase > Issue Type: Bug >Reporter: Appy > Attachments: HBASE-19530.master.001.patch > > > We shouldn't add regions with state null. In case of failures and recovery, > it's not possible to determine what did it mean and things become uncertain. > All operations should add regions in a well defined state. > For now, we'll set the default to CLOSED, since whatever ops are adding new > regions, they would anyways be enabling them explicitly if needed. > fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19522) The complete order may be wrong in AsyncBufferedMutatorImpl
[
https://issues.apache.org/jira/browse/HBASE-19522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293633#comment-16293633
]
Hadoop QA commented on HBASE-19522:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 49s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0-beta1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
35s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 56s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19522 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12902477/HBASE-19522.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7cda2e590912 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f9f869f60a |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
[jira] [Commented] (HBASE-19527) Make ExecutorService threads daemon=true.
[
https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293632#comment-16293632
]
Hadoop QA commented on HBASE-19527:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0-beta1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}100m 48s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}140m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestRegionsOnMasterOptions |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19527 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12902472/HBASE-19527.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux db228335800a 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f9f869f60a |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/10487/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10487/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-19530) New regions should always be added with state CLOSED
[
https://issues.apache.org/jira/browse/HBASE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293572#comment-16293572
]
Hadoop QA commented on HBASE-19530:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
35s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 23s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0-beta1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19530 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12902476/HBASE-19530.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6511d8b091c6 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / f9f869f60a |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10486/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10486/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> New regions should always be added with state CLOSED
>
[jira] [Updated] (HBASE-19148) Reevaluate default values of configurations
[ https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19148: -- Status: Patch Available (was: In Progress) > Reevaluate default values of configurations > --- > > Key: HBASE-19148 > URL: https://issues.apache.org/jira/browse/HBASE-19148 > Project: HBase > Issue Type: Bug > Components: defaults >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: > 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch, > HBASE-19148.master.001.patch > > > Remove cruft and mythologies. Make descriptions more digestible. Change > defaults given experience. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19148) Reevaluate default values of configurations
[ https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19148: -- Status: In Progress (was: Patch Available) > Reevaluate default values of configurations > --- > > Key: HBASE-19148 > URL: https://issues.apache.org/jira/browse/HBASE-19148 > Project: HBase > Issue Type: Bug > Components: defaults >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: > 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch, > HBASE-19148.master.001.patch > > > Remove cruft and mythologies. Make descriptions more digestible. Change > defaults given experience. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19148) Reevaluate default values of configurations
[ https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19148: -- Attachment: 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch Small amount of change. Guanghao did important ones in a linked issue. > Reevaluate default values of configurations > --- > > Key: HBASE-19148 > URL: https://issues.apache.org/jira/browse/HBASE-19148 > Project: HBase > Issue Type: Bug > Components: defaults >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: > 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch, > HBASE-19148.master.001.patch > > > Remove cruft and mythologies. Make descriptions more digestible. Change > defaults given experience. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[
https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293556#comment-16293556
]
Hadoop QA commented on HBASE-19526:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 14m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
21m 41s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0-beta1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}122m 4s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}175m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAsyncTableAdminApi |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19526 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12902461/HBASE-19526.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux 9fd811cf8002 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 20b42d2d70 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/10485/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10485/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10485/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Update hadoop version to 3.0 GA
> ---
>
> Key: HBASE-19526
> URL: https://issues.apache.org/jira/browse/HBASE-19526
> Project: HBase
> Issue Type: Task
> Components: build, dependencies
>Reporter: Mike Drob
>Assignee: Appy
> Fix For:
[jira] [Updated] (HBASE-19509) RSGroupAdminEndpoint#preCreateTable triggers TableNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19509:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> RSGroupAdminEndpoint#preCreateTable triggers TableNotFoundException
> ---
>
> Key: HBASE-19509
> URL: https://issues.apache.org/jira/browse/HBASE-19509
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Ted Yu
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 1.4.1, 1.5.0, 2.0.0-beta-1
>
> Attachments: HBASE-19509-branch-1.patch, HBASE-19509.patch
>
>
> In a cluster where RS Group endpoint is installed, I noticed the following in
> master log:
> {code}
> 2017-12-13 21:42:14,230 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
> master.TableStateManager: Unable to get table ltt-diff state
> org.apache.hadoop.hbase.TableNotFoundException: ltt-diff
> at
> org.apache.hadoop.hbase.master.TableStateManager.getTableState(TableStateManager.java:175)
> at
> org.apache.hadoop.hbase.master.TableStateManager.isTableState(TableStateManager.java:132)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.isTableDisabled(AssignmentManager.java:345)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTables(RSGroupAdminServer.java:409)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.assignTableToGroup(RSGroupAdminEndpoint.java:334)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.preCreateTable(RSGroupAdminEndpoint.java:346)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost$11.call(MasterCoprocessorHost.java:319)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost$11.call(MasterCoprocessorHost.java:316)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:599)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:672)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.preCreateTable(MasterCoprocessorHost.java:316)
> at org.apache.hadoop.hbase.master.HMaster$3.run(HMaster.java:1738)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at
> org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1734)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:559)
> {code}
> It seems RSGroupAdminEndpoint#preCreateTable should take into account that
> the table doesn't exist and avoid the extraneous TableNotFoundException .
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19509) RSGroupAdminEndpoint#preCreateTable triggers TableNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293554#comment-16293554
]
Andrew Purtell commented on HBASE-19509:
Repeated runs of the RSGroups tests look good, going to commit.
> RSGroupAdminEndpoint#preCreateTable triggers TableNotFoundException
> ---
>
> Key: HBASE-19509
> URL: https://issues.apache.org/jira/browse/HBASE-19509
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Ted Yu
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 1.4.1, 1.5.0, 2.0.0-beta-1
>
> Attachments: HBASE-19509-branch-1.patch, HBASE-19509.patch
>
>
> In a cluster where RS Group endpoint is installed, I noticed the following in
> master log:
> {code}
> 2017-12-13 21:42:14,230 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
> master.TableStateManager: Unable to get table ltt-diff state
> org.apache.hadoop.hbase.TableNotFoundException: ltt-diff
> at
> org.apache.hadoop.hbase.master.TableStateManager.getTableState(TableStateManager.java:175)
> at
> org.apache.hadoop.hbase.master.TableStateManager.isTableState(TableStateManager.java:132)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.isTableDisabled(AssignmentManager.java:345)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTables(RSGroupAdminServer.java:409)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.assignTableToGroup(RSGroupAdminEndpoint.java:334)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.preCreateTable(RSGroupAdminEndpoint.java:346)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost$11.call(MasterCoprocessorHost.java:319)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost$11.call(MasterCoprocessorHost.java:316)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:599)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:672)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.preCreateTable(MasterCoprocessorHost.java:316)
> at org.apache.hadoop.hbase.master.HMaster$3.run(HMaster.java:1738)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at
> org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1734)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:559)
> {code}
> It seems RSGroupAdminEndpoint#preCreateTable should take into account that
> the table doesn't exist and avoid the extraneous TableNotFoundException .
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19148) Reevaluate default values of configurations
[ https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293552#comment-16293552 ] stack commented on HBASE-19148: --- [~davelatham] some questions sir I think 'hbase.master.loadbalance.bytable=true' makes sense as default on big cluster where tables are big, less so on small cluster (i think). Looking at the stochastic balancer, it has this in the class comment: * This balancer is best used with hbase.master.loadbalance.bytable set to false * so that the balancer gets the full picture of all loads on the cluster. Let me add this config to hbase-default so folks see at least it is there. hbase.regionserver.fileSplitTimeout=3 in branch-2. This better? bq. hbase.regionserver.logroll.multiplier=0.5 - With the default of 0.95, whenever write load was significant, we saw many log files that had a little bit more than 1 HDFS block worth of data. Seemed a waste. So, we'd run over the block size because 0.95 doesn't give us enough wiggly room within which to close the WAL? We could come down from 0.95. 0.5 seems way down. We'd make lots of files -- a pain in itself. Set it at 0.8? Should we up the default block size? > Reevaluate default values of configurations > --- > > Key: HBASE-19148 > URL: https://issues.apache.org/jira/browse/HBASE-19148 > Project: HBase > Issue Type: Bug > Components: defaults >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19148.master.001.patch > > > Remove cruft and mythologies. Make descriptions more digestible. Change > defaults given experience. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19522) The complete order may be wrong in AsyncBufferedMutatorImpl
[
https://issues.apache.org/jira/browse/HBASE-19522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293549#comment-16293549
]
Guanghao Zhang commented on HBASE-19522:
Will commit this after see HADOOP QA result. Thanks [~tedyu] for reviewing.
> The complete order may be wrong in AsyncBufferedMutatorImpl
> ---
>
> Key: HBASE-19522
> URL: https://issues.apache.org/jira/browse/HBASE-19522
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-19522.master.001.patch,
> HBASE-19522.master.002.patch
>
>
> {code}
> List toComplete = this.futures;
> assert toSend.size() == toComplete.size();
> this.mutations = new ArrayList<>();
> this.futures = new ArrayList<>();
> bufferedSize = 0L;
> Iterator toCompleteIter = toComplete.iterator();
> for (CompletableFuture future : table.batch(toSend)) {
> future.whenComplete((r, e) -> {
> CompletableFuture f = toCompleteIter.next(); // Call next in
> callback, so the complete order may different with the future order
> if (e != null) {
> f.completeExceptionally(e);
> } else {
> f.complete(null);
> }
> });
> }
> {code}
> Here we call table.batch to get a list of CompleteFuture for each mutation.
> Then we register a call back for each future. But the problem is we call
> toCompleteIter.next() in the callback. So we may complete the future by a
> wrong order(not same with the mutation order). Meanwhile, as ArrayList is not
> thread safe, so different thread may get same future by toCompleteIter.next().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19522) The complete order may be wrong in AsyncBufferedMutatorImpl
[
https://issues.apache.org/jira/browse/HBASE-19522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293548#comment-16293548
]
Guanghao Zhang commented on HBASE-19522:
Add a new ut in 002 patch.
> The complete order may be wrong in AsyncBufferedMutatorImpl
> ---
>
> Key: HBASE-19522
> URL: https://issues.apache.org/jira/browse/HBASE-19522
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-19522.master.001.patch,
> HBASE-19522.master.002.patch
>
>
> {code}
> List toComplete = this.futures;
> assert toSend.size() == toComplete.size();
> this.mutations = new ArrayList<>();
> this.futures = new ArrayList<>();
> bufferedSize = 0L;
> Iterator toCompleteIter = toComplete.iterator();
> for (CompletableFuture future : table.batch(toSend)) {
> future.whenComplete((r, e) -> {
> CompletableFuture f = toCompleteIter.next(); // Call next in
> callback, so the complete order may different with the future order
> if (e != null) {
> f.completeExceptionally(e);
> } else {
> f.complete(null);
> }
> });
> }
> {code}
> Here we call table.batch to get a list of CompleteFuture for each mutation.
> Then we register a call back for each future. But the problem is we call
> toCompleteIter.next() in the callback. So we may complete the future by a
> wrong order(not same with the mutation order). Meanwhile, as ArrayList is not
> thread safe, so different thread may get same future by toCompleteIter.next().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19522) The complete order may be wrong in AsyncBufferedMutatorImpl
[
https://issues.apache.org/jira/browse/HBASE-19522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-19522:
---
Attachment: HBASE-19522.master.002.patch
> The complete order may be wrong in AsyncBufferedMutatorImpl
> ---
>
> Key: HBASE-19522
> URL: https://issues.apache.org/jira/browse/HBASE-19522
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-19522.master.001.patch,
> HBASE-19522.master.002.patch
>
>
> {code}
> List toComplete = this.futures;
> assert toSend.size() == toComplete.size();
> this.mutations = new ArrayList<>();
> this.futures = new ArrayList<>();
> bufferedSize = 0L;
> Iterator toCompleteIter = toComplete.iterator();
> for (CompletableFuture future : table.batch(toSend)) {
> future.whenComplete((r, e) -> {
> CompletableFuture f = toCompleteIter.next(); // Call next in
> callback, so the complete order may different with the future order
> if (e != null) {
> f.completeExceptionally(e);
> } else {
> f.complete(null);
> }
> });
> }
> {code}
> Here we call table.batch to get a list of CompleteFuture for each mutation.
> Then we register a call back for each future. But the problem is we call
> toCompleteIter.next() in the callback. So we may complete the future by a
> wrong order(not same with the mutation order). Meanwhile, as ArrayList is not
> thread safe, so different thread may get same future by toCompleteIter.next().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19530) New regions should always be added with state CLOSED
[ https://issues.apache.org/jira/browse/HBASE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19530: - Status: Patch Available (was: Open) > New regions should always be added with state CLOSED > > > Key: HBASE-19530 > URL: https://issues.apache.org/jira/browse/HBASE-19530 > Project: HBase > Issue Type: Bug >Reporter: Appy > Attachments: HBASE-19530.master.001.patch > > > We shouldn't add regions with state null. In case of failures and recovery, > it's not possible to determine what did it mean and things become uncertain. > All operations should add regions in a well defined state. > For now, we'll set the default to CLOSED, since whatever ops are adding new > regions, they would anyways be enabling them explicitly if needed. > fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19530) New regions should always be added with state CLOSED
[ https://issues.apache.org/jira/browse/HBASE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19530: - Attachment: HBASE-19530.master.001.patch > New regions should always be added with state CLOSED > > > Key: HBASE-19530 > URL: https://issues.apache.org/jira/browse/HBASE-19530 > Project: HBase > Issue Type: Bug >Reporter: Appy > Attachments: HBASE-19530.master.001.patch > > > We shouldn't add regions with state null. In case of failures and recovery, > it's not possible to determine what did it mean and things become uncertain. > All operations should add regions in a well defined state. > For now, we'll set the default to CLOSED, since whatever ops are adding new > regions, they would anyways be enabling them explicitly if needed. > fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19527) Make ExecutorService threads daemon=true.
[ https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19527: -- Attachment: HBASE-19527.master.001.patch Retry. TestHRegion passes for me locally. > Make ExecutorService threads daemon=true. > - > > Key: HBASE-19527 > URL: https://issues.apache.org/jira/browse/HBASE-19527 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19527.master.001.patch, > HBASE-19527.master.001.patch > > > Let me try this. ExecutorService runs OPENs, CLOSE, etc. If Server is going > down, no point in these threads sticking around (I think). Let me try this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19527) Make ExecutorService threads daemon=true.
[
https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293468#comment-16293468
]
Hadoop QA commented on HBASE-19527:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
38s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0-beta1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}105m 27s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}143m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegion |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19527 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12902451/HBASE-19527.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 50d7bfe7ab21 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 20b42d2d70 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/10483/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10483/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293464#comment-16293464 ] Hudson commented on HBASE-14614: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4232 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4232/]) HBASE-19272 Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK (stack: rev 89e2869e225d204b6b19df5b0aee003a61fd10d4) * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.014.patch, > HBASE-14614.master.015.patch, HBASE-14614.master.017.patch, > HBASE-14614.master.018.patch, HBASE-14614.master.019.patch, > HBASE-14614.master.020.patch, HBASE-14614.master.022.patch, > HBASE-14614.master.023.patch, HBASE-14614.master.024.patch, > HBASE-14614.master.025.patch, HBASE-14614.master.026.patch, > HBASE-14614.master.027.patch, HBASE-14614.master.028.patch, > HBASE-14614.master.029.patch, HBASE-14614.master.030.patch, > HBASE-14614.master.033.patch, HBASE-14614.master.038.patch, > HBASE-14614.master.039.patch, HBASE-14614.master.040.patch, > HBASE-14614.master.041.patch, HBASE-14614.master.042.patch, > HBASE-14614.master.043.patch, HBASE-14614.master.044.patch, > HBASE-14614.master.045.patch, HBASE-14614.master.045.patch, > HBASE-14614.master.046.patch, HBASE-14614.master.047.patch, > HBASE-14614.master.048.patch, HBASE-14614.master.049.patch, > HBASE-14614.master.050.patch, HBASE-14614.master.051.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19497) Fix findbugs and error-prone warnings in hbase-common (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19497: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master and branch-2. Thanks [~psomogyi] and [~appy] (And [~balazs.meszaros]) I fixed javadoc on commit. > Fix findbugs and error-prone warnings in hbase-common (branch-2) > > > Key: HBASE-19497 > URL: https://issues.apache.org/jira/browse/HBASE-19497 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-4 >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19497.master.001.patch, > HBASE-19497.master.001.patch, HBASE-19497.master.002.patch, > HBASE-19497.master.003.patch, HBASE-19497.master.004.patch > > > In hbase-common fix important findbugs and error-prone warnings on branch-2 / > master. Start with a forward port pass from HBASE-19239. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18352) Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293465#comment-16293465 ] Hudson commented on HBASE-18352: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4232 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4232/]) HBASE-18352 Enable (stack: rev 20b42d2d70f17d7f49a684514a1b094c40cd3b6d) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterOperationsForRegionReplicas.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java > Enable > TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas > disabled by Proc-V2 AM in HBASE-14614 > -- > > Key: HBASE-18352 > URL: https://issues.apache.org/jira/browse/HBASE-18352 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: huaxiang sun > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18352.master.001.patch, > HBASE-18352.master.002.patch, HBASE-18946_1.patch > > > The following replica tests were disabled by Core Proc-V2 AM in HBASE-14614: > - Disabled parts of...testCreateTableWithMultipleReplicas in > TestMasterOperationsForRegionReplicas There is an issue w/ assigning more > replicas if number of replicas is changed on us. See '/* DISABLED! FOR > NOW'. > ** NOTE We moved fixing of the below two tests out to HBASE-19268 > - Disabled testRegionReplicasOnMidClusterHighReplication in > TestStochasticLoadBalancer2 > - Disabled testFlushAndCompactionsInPrimary in TestRegionReplicas > This JIRA tracks the work to enable them (or modify/remove if not applicable). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18838) shaded artifacts are incorrect when built against hadoop 3
[ https://issues.apache.org/jira/browse/HBASE-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293462#comment-16293462 ] Hudson commented on HBASE-18838: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4232 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4232/]) HBASE-18838 Fix hadoop3 check-shaded-invariants (mdrob: rev 75f512bd717a14e0c7b7bbe5594de9270759706e) * (edit) hbase-rest/pom.xml * (edit) hbase-procedure/pom.xml * (edit) hbase-thrift/pom.xml * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/JSONMetricUtil.java * (edit) hbase-shell/pom.xml * (edit) hbase-it/pom.xml * (edit) pom.xml * (edit) hbase-backup/pom.xml * (edit) hbase-shaded/pom.xml * (edit) hbase-examples/pom.xml * (edit) hbase-mapreduce/pom.xml * (edit) hbase-replication/pom.xml * (edit) hbase-rsgroup/pom.xml * (edit) hbase-endpoint/pom.xml * (edit) hbase-client/pom.xml * (edit) hbase-server/pom.xml * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-testing-util/pom.xml > shaded artifacts are incorrect when built against hadoop 3 > -- > > Key: HBASE-18838 > URL: https://issues.apache.org/jira/browse/HBASE-18838 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0-alpha-3 >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18838-WIP.v2.patch, HBASE-18838.WIP.patch, > HBASE-18838.v3.patch, HBASE-18838.v4.patch, HBASE-18838.v5.patch > > > Building master/branch-2 against the hadoop-3 profile results in > check-invariants screaming about unrelocated dependencies. will list details > in comment. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19272) Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again...
[ https://issues.apache.org/jira/browse/HBASE-19272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293463#comment-16293463 ] Hudson commented on HBASE-19272: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4232 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4232/]) HBASE-19272 Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK (stack: rev 89e2869e225d204b6b19df5b0aee003a61fd10d4) * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java > Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again... > -- > > Key: HBASE-19272 > URL: https://issues.apache.org/jira/browse/HBASE-19272 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack >Assignee: stack > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19272.master.001.patch > > > DIsabled by HBASE-14614, enabling AMv2. See HBASE-18110. > Here is the list: > * TestHBaseFsckTwoRS > * TestOfflineMetaRebuildBase > * TestHBaseFsckReplicas > * TestOfflineMetaRebuildOverlap > * TestHBaseFsckOneRS > * TestOfflineMetaRebuildHole -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19528) Major Compaction Tool
[ https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293458#comment-16293458 ] stack commented on HBASE-19528: --- And you've seen the CompactionTool we already have in the code base FYI > Major Compaction Tool > -- > > Key: HBASE-19528 > URL: https://issues.apache.org/jira/browse/HBASE-19528 > Project: HBase > Issue Type: New Feature >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 3.0.0 > > > The basic overview of how this tool works is: > Parameters: > Table > Stores > ClusterConcurrency > Timestamp > So you input a table, desired concurrency and the list of stores you wish to > major compact. The tool first checks the filesystem to see which stores need > compaction based on the timestamp you provide (default is current time). It > takes that list of stores that require compaction and executes those requests > concurrently with at most N distinct RegionServers compacting at a given > time. Each thread waits for the compaction to complete before moving to the > next queue. If a region split, merge or move happens this tool ensures those > regions get major compacted as well. > This helps us in two ways, we can limit how much I/O bandwidth we are using > for major compaction cluster wide and we are guaranteed after the tool > completes that all requested compactions complete regardless of moves, merges > and splits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19528) Major Compaction Tool
[ https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293439#comment-16293439 ] stack commented on HBASE-19528: --- How you tell the region about the newly compacted files [~churromorales]? [~anastas] FYI > Major Compaction Tool > -- > > Key: HBASE-19528 > URL: https://issues.apache.org/jira/browse/HBASE-19528 > Project: HBase > Issue Type: New Feature >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 3.0.0 > > > The basic overview of how this tool works is: > Parameters: > Table > Stores > ClusterConcurrency > Timestamp > So you input a table, desired concurrency and the list of stores you wish to > major compact. The tool first checks the filesystem to see which stores need > compaction based on the timestamp you provide (default is current time). It > takes that list of stores that require compaction and executes those requests > concurrently with at most N distinct RegionServers compacting at a given > time. Each thread waits for the compaction to complete before moving to the > next queue. If a region split, merge or move happens this tool ensures those > regions get major compacted as well. > This helps us in two ways, we can limit how much I/O bandwidth we are using > for major compaction cluster wide and we are guaranteed after the tool > completes that all requested compactions complete regardless of moves, merges > and splits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19529) Handle null states in AM
[ https://issues.apache.org/jira/browse/HBASE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293437#comment-16293437 ] stack commented on HBASE-19529: --- Minimally, we'd doc this in code. > Handle null states in AM > > > Key: HBASE-19529 > URL: https://issues.apache.org/jira/browse/HBASE-19529 > Project: HBase > Issue Type: Bug >Reporter: Appy > > From debugging in HBASE-19457, found some questions that need concrete > answers: > 1) What does a region state of null in meta means? Currently AM treats it as > OFFLINE > 2) What does a table state of null in meta means? Currently TSM treats it as > ENABLED. > More importantly, we need to fix holes in AM so that our state machine is > well defined and doesn't end up in these uncertainties. > Figuring out answers to above questions will help in that direction. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19530) New regions should always be added with state CLOSED
[ https://issues.apache.org/jira/browse/HBASE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293436#comment-16293436 ] stack commented on HBASE-19530: --- Sounds good. Can we try it against hadoopqa and see what breaks? > New regions should always be added with state CLOSED > > > Key: HBASE-19530 > URL: https://issues.apache.org/jira/browse/HBASE-19530 > Project: HBase > Issue Type: Bug >Reporter: Appy > > We shouldn't add regions with state null. In case of failures and recovery, > it's not possible to determine what did it mean and things become uncertain. > All operations should add regions in a well defined state. > For now, we'll set the default to CLOSED, since whatever ops are adding new > regions, they would anyways be enabling them explicitly if needed. > fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19457) Debugging flaky TestTruncateTableProcedure#testRecoveryAndDoubleExecutionPreserveSplits
[ https://issues.apache.org/jira/browse/HBASE-19457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293435#comment-16293435 ] stack commented on HBASE-19457: --- I can't find a case of three tiers of proc. Would need to try it. I don't see why not. Yeah, its turning up some greens now when it used to be solid red: https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html The new failure types -- 2 out 5 -- seem different. As you say, lets keep an eye on it. Thanks for new JIRAs. Yeah, lets sort out state in meta. > Debugging flaky > TestTruncateTableProcedure#testRecoveryAndDoubleExecutionPreserveSplits > --- > > Key: HBASE-19457 > URL: https://issues.apache.org/jira/browse/HBASE-19457 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19457.master.001.patch, patch1, test-output.txt > > > Trying to explain the bug in a more general way where understanding of > ProcedureV2 is not required. > Truncating table operation: > > delete region states from meta > delete table state from meta > > add new regions to meta with state null. > crash > recovery: TableStateManager treats table with null state as ENABLED. AM > treats regions with null state as offline. Combined result - AM starts > assigning the new regions from incomplete truncate operation. > Fix: Mark table as disabled instead of deleting it's state. > > *patch1* > Just added some logging to help with debugging: > - 60s was too less time, increased timeout > - Added some useful log statements -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293427#comment-16293427 ] stack commented on HBASE-19526: --- +1 if the patch works. Does it? Do we have to add the repo that has the 3.0.0 RC in it? > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19498: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-2 and master. Thanks [~psomogyi]. Good discussion [~appy] > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293425#comment-16293425 ] stack commented on HBASE-19498: --- [~psomogyi] Appy and I had questions up on RB when you have a chance thanks. > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293422#comment-16293422 ] stack commented on HBASE-19498: --- bq. But if we really want to move to specifying charset manually, then it's better to have a util function StringtoBytesUtf8 and use it everywhere then current approach. This we have and we use it everywhere. See Bytes.toString. bq. I also found https://stackoverflow.com/questions/361975/setting-the-default-java-character-encoding While happy at first, but it might not be good idea to change default charset globally. Might have very bad consequence in some places. Yeah, thats about setting the platform coding. We have setting the charset encoding Strings when we write to zk or the db since near-to day-one as UTF-8 (see Bytes class). When we write out web pages, they have at the head that they are encoded as UTF-8. We don't advertise it and we should but we log data in UTF-8. When we dump on the console its usually UTF-8 (this might clash w/ terminal charset). This patch adds UTF-8 specification in remaining places where it goes unspecified. Some is silly (tests), some makes sense, dumping mbean output, and while there are cases where if data had been encoded with one charset and read back w/ another, i don't see instances in this patch. I was going to commit to get it in because it will rot if we leave it hang. Having it in also makes backport to 1.4 run smoother since it already has these patches. Good on you [~appy] > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19527) Make ExecutorService threads daemon=true.
[ https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293408#comment-16293408 ] stack commented on HBASE-19527: --- A whole class to set a flag! Who ever wrote that was doing make work.  Am setting flag on factory here Iirc so wouldn't work. Is it even used in Hadoop code base outside of the patch that committed it? > Make ExecutorService threads daemon=true. > - > > Key: HBASE-19527 > URL: https://issues.apache.org/jira/browse/HBASE-19527 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19527.master.001.patch > > > Let me try this. ExecutorService runs OPENs, CLOSE, etc. If Server is going > down, no point in these threads sticking around (I think). Let me try this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19527) Make ExecutorService threads daemon=true.
[ https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293404#comment-16293404 ] Appy edited comment on HBASE-19527 at 12/15/17 11:24 PM: - Came across this a while back. https://hadoop.apache.org/docs/r1.2.1/api/org/apache/hadoop/util/Daemon.html Nice syntactic sugar. Can we use it if not too much change? was (Author: appy): Came across this a while back. https://hadoop.apache.org/docs/r1.2.1/api/org/apache/hadoop/util/Daemon.html Nice syntactic sugar. Can we use it too? > Make ExecutorService threads daemon=true. > - > > Key: HBASE-19527 > URL: https://issues.apache.org/jira/browse/HBASE-19527 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19527.master.001.patch > > > Let me try this. ExecutorService runs OPENs, CLOSE, etc. If Server is going > down, no point in these threads sticking around (I think). Let me try this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19527) Make ExecutorService threads daemon=true.
[ https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293404#comment-16293404 ] Appy commented on HBASE-19527: -- Came across this a while back. https://hadoop.apache.org/docs/r1.2.1/api/org/apache/hadoop/util/Daemon.html Nice syntactic sugar. Can we use it too? > Make ExecutorService threads daemon=true. > - > > Key: HBASE-19527 > URL: https://issues.apache.org/jira/browse/HBASE-19527 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19527.master.001.patch > > > Let me try this. ExecutorService runs OPENs, CLOSE, etc. If Server is going > down, no point in these threads sticking around (I think). Let me try this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293401#comment-16293401 ] Appy commented on HBASE-19498: -- Is there anything in here that's urgent to get into beta1? If yes, commit then followup with refactor. If not, let's introduce the util function as part of this patch itself. > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293386#comment-16293386 ] Appy edited comment on HBASE-19498 at 12/15/17 11:20 PM: - I also found https://stackoverflow.com/questions/361975/setting-the-default-java-character-encoding While happy at first, but it might not be good idea to change default charset globally. Might have very bad consequence in some places. was (Author: appy): I also found https://stackoverflow.com/questions/361975/setting-the-default-java-character-encoding While happy at first, it might not be good idea to change default charset globally. Might have very bad consequence in some places. > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293391#comment-16293391 ] Appy commented on HBASE-19526: -- Yeah, it's just metrics stuff. Let me take a look of amount of work required. > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19526: - Status: Patch Available (was: Open) > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy reassigned HBASE-19526: Assignee: Appy > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293390#comment-16293390 ] Appy commented on HBASE-19526: -- Ran locally, compiles. Let's see what QA has to say about tests. > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19526: - Attachment: HBASE-19526.master.001.patch > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19526.master.001.patch > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293386#comment-16293386 ] Appy commented on HBASE-19498: -- I also found https://stackoverflow.com/questions/361975/setting-the-default-java-character-encoding While happy at first, it might not be good idea to change default charset globally. Might have very bad consequence in some places. > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19457) Debugging flaky TestTruncateTableProcedure#testRecoveryAndDoubleExecutionPreserveSplits
[ https://issues.apache.org/jira/browse/HBASE-19457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293380#comment-16293380 ] Appy edited comment on HBASE-19457 at 12/15/17 10:43 PM: - We discussed few things, here's the summary: - we have procs spawing subprocs, but not sure if there's an example where this tree's depth > 2. If yes, we can change truncate proc to just delete proc + create proc. bq. As a step in truncate before we create the new? Wonder why this needs it and CreateTable doesnt (I think you ask this above). Both have ADD_TO_META step where they add regions to meta. But when we fail after that: in case of truncate proc, there's a table row in meta with state null --> gets assumed as enabled --> AM starts interfering in case of create proc, there's no table row at all --> AM ignores those new regions New stuff: Stack recently committed HBASE-18946 which fixes issues around balancer and assigning. After it went in, we see more greens for TestTruncateTableProcedure in flaky dashboard. A word on that: When AM interfered on recovery (see "...recovery: TableStateManager treats table with null state as ENABLED. AM treats regions with null state as offline. Combined result - AM starts assigning the new " in description), it started Assign procs. But they got stuck for some reason (which i didn't care to debug as part of this test fix since it's unrelated). His patch makes that case better. But the real fix here should be to correctly handle state in TTP so that AM doesn't interfere. We'll keep an eye on dashboard, see the new failures, and then decide verdict on this jira's patch. In meantime opened this new jira to discuss other questions HBASE-19529, HBASE-19530 was (Author: appy): We discussed few things, here's the summary: - we have procs spawing subprocs, but not sure if there's an example where this tree's depth > 2. If yes, we can change truncate proc to just delete proc + create proc. bq. As a step in truncate before we create the new? Wonder why this needs it and CreateTable doesnt (I think you ask this above). Both have ADD_TO_META step where they add regions to meta. But when we fail after that: in case of truncate proc, there's a table row in meta with state null --> gets assumed as enabled --> AM starts interfering in case of create proc, there's no table row at all --> AM ignores those new regions New stuff: Stack recently committed HBASE-18946 which fixes issues around balancer and assigning. After it went in, we see more greens for TestTruncateTableProcedure in flaky dashboard. A word on that: When AM interfered on recovery (see "...recovery: TableStateManager treats table with null state as ENABLED. AM treats regions with null state as offline. Combined result - AM starts assigning the new " in description), it started Assign procs. But they got stuck for some reason (which i didn't care to debug as part of this test fix since it's unrelated). His patch makes that case better. But the real fix here should be to correctly handle state in TTP so that AM doesn't interfere. We'll keep an eye on dashboard, see the new failures, and then decide verdict on this patch. In meantime opened this new jira to discuss other questions HBASE-19529, HBASE-19530 > Debugging flaky > TestTruncateTableProcedure#testRecoveryAndDoubleExecutionPreserveSplits > --- > > Key: HBASE-19457 > URL: https://issues.apache.org/jira/browse/HBASE-19457 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19457.master.001.patch, patch1, test-output.txt > > > Trying to explain the bug in a more general way where understanding of > ProcedureV2 is not required. > Truncating table operation: > > delete region states from meta > delete table state from meta > > add new regions to meta with state null. > crash > recovery: TableStateManager treats table with null state as ENABLED. AM > treats regions with null state as offline. Combined result - AM starts > assigning the new regions from incomplete truncate operation. > Fix: Mark table as disabled instead of deleting it's state. > > *patch1* > Just added some logging to help with debugging: > - 60s was too less time, increased timeout > - Added some useful log statements -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19457) Debugging flaky TestTruncateTableProcedure#testRecoveryAndDoubleExecutionPreserveSplits
[ https://issues.apache.org/jira/browse/HBASE-19457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293380#comment-16293380 ] Appy commented on HBASE-19457: -- We discussed few things, here's the summary: - we have procs spawing subprocs, but not sure if there's an example where this tree's depth > 2. If yes, we can change truncate proc to just delete proc + create proc. bq. As a step in truncate before we create the new? Wonder why this needs it and CreateTable doesnt (I think you ask this above). Both have ADD_TO_META step where they add regions to meta. But when we fail after that: in case of truncate proc, there's a table row in meta with state null --> gets assumed as enabled --> AM starts interfering in case of create proc, there's no table row at all --> AM ignores those new regions New stuff: Stack recently committed HBASE-18946 which fixes issues around balancer and assigning. After it went in, we see more greens for TestTruncateTableProcedure in flaky dashboard. A word on that: When AM interfered on recovery (see "...recovery: TableStateManager treats table with null state as ENABLED. AM treats regions with null state as offline. Combined result - AM starts assigning the new " in description), it started Assign procs. But they got stuck for some reason (which i didn't care to debug as part of this test fix since it's unrelated). His patch makes that case better. But the real fix here should be to correctly handle state in TTP so that AM doesn't interfere. We'll keep an eye on dashboard, see the new failures, and then decide verdict on this patch. In meantime opened this new jira to discuss other questions HBASE-19529, HBASE-19530 > Debugging flaky > TestTruncateTableProcedure#testRecoveryAndDoubleExecutionPreserveSplits > --- > > Key: HBASE-19457 > URL: https://issues.apache.org/jira/browse/HBASE-19457 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19457.master.001.patch, patch1, test-output.txt > > > Trying to explain the bug in a more general way where understanding of > ProcedureV2 is not required. > Truncating table operation: > > delete region states from meta > delete table state from meta > > add new regions to meta with state null. > crash > recovery: TableStateManager treats table with null state as ENABLED. AM > treats regions with null state as offline. Combined result - AM starts > assigning the new regions from incomplete truncate operation. > Fix: Mark table as disabled instead of deleting it's state. > > *patch1* > Just added some logging to help with debugging: > - 60s was too less time, increased timeout > - Added some useful log statements -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19530) New regions should always be added with state CLOSED
Appy created HBASE-19530: Summary: New regions should always be added with state CLOSED Key: HBASE-19530 URL: https://issues.apache.org/jira/browse/HBASE-19530 Project: HBase Issue Type: Bug Reporter: Appy We shouldn't add regions with state null. In case of failures and recovery, it's not possible to determine what did it mean and things become uncertain. All operations should add regions in a well defined state. For now, we'll set the default to CLOSED, since whatever ops are adding new regions, they would anyways be enabling them explicitly if needed. fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19529) Handle null states in AM
Appy created HBASE-19529: Summary: Handle null states in AM Key: HBASE-19529 URL: https://issues.apache.org/jira/browse/HBASE-19529 Project: HBase Issue Type: Bug Reporter: Appy >From debugging in HBASE-19457, found some questions that need concrete answers: 1) What does a region state of null in meta means? Currently AM treats it as OFFLINE 2) What does a table state of null in meta means? Currently TSM treats it as ENABLED. More importantly, we need to fix holes in AM so that our state machine is well defined and doesn't end up in these uncertainties. Figuring out answers to above questions will help in that direction. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19528) Major Compaction Tool
churro morales created HBASE-19528: -- Summary: Major Compaction Tool Key: HBASE-19528 URL: https://issues.apache.org/jira/browse/HBASE-19528 Project: HBase Issue Type: New Feature Reporter: churro morales Assignee: churro morales Fix For: 2.0.0, 3.0.0 The basic overview of how this tool works is: Parameters: Table Stores ClusterConcurrency Timestamp So you input a table, desired concurrency and the list of stores you wish to major compact. The tool first checks the filesystem to see which stores need compaction based on the timestamp you provide (default is current time). It takes that list of stores that require compaction and executes those requests concurrently with at most N distinct RegionServers compacting at a given time. Each thread waits for the compaction to complete before moving to the next queue. If a region split, merge or move happens this tool ensures those regions get major compacted as well. This helps us in two ways, we can limit how much I/O bandwidth we are using for major compaction cluster wide and we are guaranteed after the tool completes that all requested compactions complete regardless of moves, merges and splits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19527) Make ExecutorService threads daemon=true.
[ https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19527: -- Assignee: stack Status: Patch Available (was: Open) Trying hadoopqa. > Make ExecutorService threads daemon=true. > - > > Key: HBASE-19527 > URL: https://issues.apache.org/jira/browse/HBASE-19527 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19527.master.001.patch > > > Let me try this. ExecutorService runs OPENs, CLOSE, etc. If Server is going > down, no point in these threads sticking around (I think). Let me try this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293314#comment-16293314 ] Appy commented on HBASE-19498: -- Took a look at usages with [~stack] again. The uses are either: - in tests - printing to logs - on md5 checksum of encoded region name so these are fine. But if we really want to move to specifying charset manually, then it's better to have a util function StringtoBytesUtf8 and use it everywhere then current approach. > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19527) Make ExecutorService threads daemon=true.
[ https://issues.apache.org/jira/browse/HBASE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19527: -- Attachment: HBASE-19527.master.001.patch > Make ExecutorService threads daemon=true. > - > > Key: HBASE-19527 > URL: https://issues.apache.org/jira/browse/HBASE-19527 > Project: HBase > Issue Type: Bug >Reporter: stack > Attachments: HBASE-19527.master.001.patch > > > Let me try this. ExecutorService runs OPENs, CLOSE, etc. If Server is going > down, no point in these threads sticking around (I think). Let me try this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19527) Make ExecutorService threads daemon=true.
stack created HBASE-19527: - Summary: Make ExecutorService threads daemon=true. Key: HBASE-19527 URL: https://issues.apache.org/jira/browse/HBASE-19527 Project: HBase Issue Type: Bug Reporter: stack Let me try this. ExecutorService runs OPENs, CLOSE, etc. If Server is going down, no point in these threads sticking around (I think). Let me try this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293281#comment-16293281 ] stack commented on HBASE-19526: --- That's be nice. Compat was where we were to put all the differences but we never did. Its just host for different metrics systems if I understand right. How hard to purge? > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob > Fix For: 2.0.0-beta-1 > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293272#comment-16293272 ] stack commented on HBASE-19498: --- I took a look at usage here too. Yeah. Strings or dumping stuff on UI or test. UTF-8 is the systems charset. Picking up default charset of the OS we do not want. I hear you on perhaps something that worked will now throw odd errors but I don't see a case in the patch. What you reckon [~appy] > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19497) Fix findbugs and error-prone warnings in hbase-common (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293267#comment-16293267 ] stack commented on HBASE-19497: --- I went through the patch. We add utf-8 when getting a String to display or when dumping stuff on console or ui or its in tests. Fine I'd say. > Fix findbugs and error-prone warnings in hbase-common (branch-2) > > > Key: HBASE-19497 > URL: https://issues.apache.org/jira/browse/HBASE-19497 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-4 >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19497.master.001.patch, > HBASE-19497.master.001.patch, HBASE-19497.master.002.patch, > HBASE-19497.master.003.patch, HBASE-19497.master.004.patch > > > In hbase-common fix important findbugs and error-prone warnings on branch-2 / > master. Start with a forward port pass from HBASE-19239. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293259#comment-16293259 ] stack commented on HBASE-19498: --- It'll be different for characters > position 128. Otherwise its ASCII. Specifying the charset was on PrintStream and toString, stuff we are going to dump on console or in UI... in this case UTF-8 would be better (probably harmless -- though famous last words). Thanks [~appy] > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19358) Improve the stability of splitting log when do fail over
[
https://issues.apache.org/jira/browse/HBASE-19358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293257#comment-16293257
]
Ted Yu commented on HBASE-19358:
https://builds.apache.org/job/PreCommit-HBASE-Build/10482/console :
{code}
21:20:45 HBASE-19358 patch is being downloaded at Fri Dec 15 21:20:45 UTC 2017
from
21:20:46
https://issues.apache.org/jira/secure/attachment/12902235/split-logic-new.jpg
-> Downloaded
21:20:46 ERROR: Unsure how to process HBASE-19358.
{code}
The above run was triggered by me.
The previous (10481) failed in the same way.
> Improve the stability of splitting log when do fail over
>
>
> Key: HBASE-19358
> URL: https://issues.apache.org/jira/browse/HBASE-19358
> Project: HBase
> Issue Type: Improvement
> Components: MTTR
>Affects Versions: 0.98.24
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
> Attachments: HBASE-19358.patch, split-1-log.png, split-logic-new.jpg,
> split-logic-old.jpg, split-table.png, split_test_result.png
>
>
> The way we splitting log now is like the following figure:
> !https://issues.apache.org/jira/secure/attachment/12902234/split-logic-old.jpg!
> The problem is the OutputSink will write the recovered edits during splitting
> log, which means it will create one WriterAndPath for each region and retain
> it until the end. If the cluster is small and the number of regions per rs is
> large, it will create too many HDFS streams at the same time. Then it is
> prone to failure since each datanode need to handle too many streams.
> Thus I come up with a new way to split log.
> !https://issues.apache.org/jira/secure/attachment/12902235/split-logic-new.jpg!
> We try to cache all the recovered edits, but if it exceeds the MaxHeapUsage,
> we will pick the largest EntryBuffer and write it to a file (close the writer
> after finish). Then after we read all entries into memory, we will start a
> writeAndCloseThreadPool, it starts a certain number of threads to write all
> buffers to files. Thus it will not create HDFS streams more than
> *_hbase.regionserver.hlog.splitlog.writer.threads_* we set.
> The biggest benefit is we can control the number of streams we create during
> splitting log,
> it will not exceeds *_hbase.regionserver.wal.max.splitters *
> hbase.regionserver.hlog.splitlog.writer.threads_*, but before it is
> *_hbase.regionserver.wal.max.splitters * the number of region the hlog
> contains_*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19498) Fix findbugs and error-prone warnings in hbase-client (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293229#comment-16293229 ] Appy commented on HBASE-19498: -- (Same comment as on HABSE-19497) i am not sure about charactersets thing. What was the old default? If it was different, then we shouldn't do it until we have made sure that this stuff is not serialized and stored anywhere. > Fix findbugs and error-prone warnings in hbase-client (branch-2) > > > Key: HBASE-19498 > URL: https://issues.apache.org/jira/browse/HBASE-19498 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19498.master.001.patch, > HBASE-19498.master.002.patch, HBASE-19498.master.003.patch, > HBASE-19498.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19497) Fix findbugs and error-prone warnings in hbase-common (branch-2)
[ https://issues.apache.org/jira/browse/HBASE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293228#comment-16293228 ] Appy commented on HBASE-19497: -- i am not sure about charactersets thing. What was the old default? If it was different, then we shouldn't do it until we have made sure that this stuff is not serialized and stored anywhere. > Fix findbugs and error-prone warnings in hbase-common (branch-2) > > > Key: HBASE-19497 > URL: https://issues.apache.org/jira/browse/HBASE-19497 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-4 >Reporter: Peter Somogyi >Assignee: Peter Somogyi > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19497.master.001.patch, > HBASE-19497.master.001.patch, HBASE-19497.master.002.patch, > HBASE-19497.master.003.patch, HBASE-19497.master.004.patch > > > In hbase-common fix important findbugs and error-prone warnings on branch-2 / > master. Start with a forward port pass from HBASE-19239. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18352) Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18352: -- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master and branch-2. > Enable > TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas > disabled by Proc-V2 AM in HBASE-14614 > -- > > Key: HBASE-18352 > URL: https://issues.apache.org/jira/browse/HBASE-18352 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: huaxiang sun > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18352.master.001.patch, > HBASE-18352.master.002.patch, HBASE-18946_1.patch > > > The following replica tests were disabled by Core Proc-V2 AM in HBASE-14614: > - Disabled parts of...testCreateTableWithMultipleReplicas in > TestMasterOperationsForRegionReplicas There is an issue w/ assigning more > replicas if number of replicas is changed on us. See '/* DISABLED! FOR > NOW'. > ** NOTE We moved fixing of the below two tests out to HBASE-19268 > - Disabled testRegionReplicasOnMidClusterHighReplication in > TestStochasticLoadBalancer2 > - Disabled testFlushAndCompactionsInPrimary in TestRegionReplicas > This JIRA tracks the work to enable them (or modify/remove if not applicable). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18352) Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293221#comment-16293221 ] stack commented on HBASE-18352: --- The whole suite ran. This test passed. We've picked up some timing out tests. Will address elsewhere. > Enable > TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas > disabled by Proc-V2 AM in HBASE-14614 > -- > > Key: HBASE-18352 > URL: https://issues.apache.org/jira/browse/HBASE-18352 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: huaxiang sun > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18352.master.001.patch, > HBASE-18352.master.002.patch, HBASE-18946_1.patch > > > The following replica tests were disabled by Core Proc-V2 AM in HBASE-14614: > - Disabled parts of...testCreateTableWithMultipleReplicas in > TestMasterOperationsForRegionReplicas There is an issue w/ assigning more > replicas if number of replicas is changed on us. See '/* DISABLED! FOR > NOW'. > ** NOTE We moved fixing of the below two tests out to HBASE-19268 > - Disabled testRegionReplicasOnMidClusterHighReplication in > TestStochasticLoadBalancer2 > - Disabled testFlushAndCompactionsInPrimary in TestRegionReplicas > This JIRA tracks the work to enable them (or modify/remove if not applicable). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-14620) Procedure V2: Update HBCK to incorporate the Proc-V2-based assignment
[ https://issues.apache.org/jira/browse/HBASE-14620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14620: -- Resolution: Won't Fix Status: Resolved (was: Patch Available) Resolving won't fix. We'll make a new hbck for hbase2 rather than bring along the old hbck1. > Procedure V2: Update HBCK to incorporate the Proc-V2-based assignment > - > > Key: HBASE-14620 > URL: https://issues.apache.org/jira/browse/HBASE-14620 > Project: HBase > Issue Type: Sub-task > Components: hbck, proc-v2 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > Attachments: HBASE-14620.v1-master.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19526) Update hadoop version to 3.0 GA
[ https://issues.apache.org/jira/browse/HBASE-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293214#comment-16293214 ] Appy commented on HBASE-19526: -- And after the change, if hadoop2-compat builds fine against both hadoop2 and hadoop3, then: - we don't need any compat stuff in first place, right? That's non-trivial stuff and shouldn't be in code base if not needed. - can get rid of hbase-hadoop-compat module? [~stack]? > Update hadoop version to 3.0 GA > --- > > Key: HBASE-19526 > URL: https://issues.apache.org/jira/browse/HBASE-19526 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Mike Drob > Fix For: 2.0.0-beta-1 > > > We're still building against hadoop 3.0-beta1, while GA is recently released. > We should update, hopefully no surprises. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19231) Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate
[ https://issues.apache.org/jira/browse/HBASE-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19231: -- Fix Version/s: (was: 2.0.0-beta-1) 2.0.0 > Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate > - > > Key: HBASE-19231 > URL: https://issues.apache.org/jira/browse/HBASE-19231 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > Also #testHBaseFsckWithFewerMetaReplicas, > #testHBaseFsckWithExcessMetaReplicas, and > #testHBaseFsckWithFewerMetaReplicaZnodes in TestMetaWithReplicas. Both depend > on HBCK for validation. Disabled for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19231) Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate
[ https://issues.apache.org/jira/browse/HBASE-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19231: -- Summary: Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate (was: Reenable TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate) > Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate > - > > Key: HBASE-19231 > URL: https://issues.apache.org/jira/browse/HBASE-19231 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > Also #testHBaseFsckWithFewerMetaReplicas, > #testHBaseFsckWithExcessMetaReplicas, and > #testHBaseFsckWithFewerMetaReplicaZnodes in TestMetaWithReplicas. Both depend > on HBCK for validation. Disabled for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-19231) Reenable TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate
[ https://issues.apache.org/jira/browse/HBASE-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-19231: - Assignee: stack > Reenable TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate > - > > Key: HBASE-19231 > URL: https://issues.apache.org/jira/browse/HBASE-19231 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > Also #testHBaseFsckWithFewerMetaReplicas, > #testHBaseFsckWithExcessMetaReplicas, and > #testHBaseFsckWithFewerMetaReplicaZnodes in TestMetaWithReplicas. Both depend > on HBCK for validation. Disabled for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19231) Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate
[ https://issues.apache.org/jira/browse/HBASE-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293211#comment-16293211 ] stack commented on HBASE-19231: --- Or rather, let me just move this out of beta-1. Leave it open. No harm in recasting at least one of these Ignored tests so it works in new context. Moving this out. > Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate > - > > Key: HBASE-19231 > URL: https://issues.apache.org/jira/browse/HBASE-19231 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > Also #testHBaseFsckWithFewerMetaReplicas, > #testHBaseFsckWithExcessMetaReplicas, and > #testHBaseFsckWithFewerMetaReplicaZnodes in TestMetaWithReplicas. Both depend > on HBCK for validation. Disabled for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19231) Reenable TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate
[ https://issues.apache.org/jira/browse/HBASE-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293204#comment-16293204 ] stack commented on HBASE-19231: --- I removed testHBaseFsckWithFewerMetaReplicas from this test. It does this: HBaseFsckRepair.closeRegionSilentlyAndWait You can't do silent close in hbase2. Not allowed. Then uses hbck to do repair after doing this damage. Removing it. testHBaseFsckWithFewerMetaReplicaZnodes does same silent close. Removing it. Removed testHBaseFsckWithExcessMetaReplicas. It adds replica then uses hbck to repair too many replicas. We probably need such a thing in hbase2 somehow. Ok. Not going to fix. Just removing the above. There are a few tests of replicas on meta still in the suite. > Reenable TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate > - > > Key: HBASE-19231 > URL: https://issues.apache.org/jira/browse/HBASE-19231 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack > Fix For: 2.0.0-beta-1 > > > Also #testHBaseFsckWithFewerMetaReplicas, > #testHBaseFsckWithExcessMetaReplicas, and > #testHBaseFsckWithFewerMetaReplicaZnodes in TestMetaWithReplicas. Both depend > on HBCK for validation. Disabled for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19121) HBCK for AMv2
[ https://issues.apache.org/jira/browse/HBASE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293191#comment-16293191 ] stack commented on HBASE-19121: --- Also, see TestMetaWithReplicas in branch-1. Some of the tests in the suite have been removed in branch-2 because they used hbck. At least the last one, where hbck was repairing too many replicas looks like something we should integrate into hbase or redo in any hbck2 (See branch-1 because removed from branch-2). > HBCK for AMv2 > - > > Key: HBASE-19121 > URL: https://issues.apache.org/jira/browse/HBASE-19121 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: stack > Fix For: 2.0.0-beta-2 > > > We don't have an hbck for the new AM. Old hbck may actually do damage going > against AMv2. > Fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18946) Stochastic load balancer assigns replica regions to the same RS
[ https://issues.apache.org/jira/browse/HBASE-18946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293190#comment-16293190 ] Hudson commented on HBASE-18946: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4231 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4231/]) HBASE-18946 Stochastic load balancer assigns replica regions to the same (stack: rev 010012cbcb3064b78b9e184a2808bbd26ea80903) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RecoverMetaProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/AbstractTestDLS.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicasWithRestartScenarios.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ZKNamespaceManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestAssignmentManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSKilledWhenInitializing.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CloneSnapshotProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestAssignProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java > Stochastic load balancer assigns replica regions to the same RS > --- > > Key: HBASE-18946 > URL: https://issues.apache.org/jira/browse/HBASE-18946 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: stack > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18946.master.001.patch, > HBASE-18946.master.002.patch, HBASE-18946.master.003.patch, > HBASE-18946.master.004.patch, HBASE-18946.master.005.patch, > HBASE-18946.master.006.patch, HBASE-18946.master.007.patch, > HBASE-18946.master.008.patch, HBASE-18946.master.009.patch, > HBASE-18946.master.010.patch, HBASE-18946.master.011.patch, > HBASE-18946.master.012.patch, HBASE-18946.patch, HBASE-18946.patch, > HBASE-18946_2.patch, HBASE-18946_2.patch, HBASE-18946_simple_7.patch, > HBASE-18946_simple_8.patch, TestRegionReplicasWithRestartScenarios.java > > > Trying out region replica and its assignment I can see that some times the > default LB Stocahstic load balancer assigns replica regions to the same RS. > This happens when we have 3 RS checked in and we have a table with 3 > replicas. When a RS goes down then the replicas being assigned to same RS is > acceptable but the case when we have enough RS to assign this behaviour is > undesirable and does not solve the purpose of replicas. > [~huaxiang] and [~enis]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18352) Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas disabled by Proc-V2 AM in HBASE-14614
[
https://issues.apache.org/jira/browse/HBASE-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293183#comment-16293183
]
Hadoop QA commented on HBASE-18352:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
46s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
9s{color} | {color:red} hbase-server: The patch generated 5 new + 295 unchanged
- 3 fixed = 300 total (was 298) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
25s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 53s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0-beta1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}107m 44s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}146m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18352 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12902420/HBASE-18352.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ceeecba4d0ec 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 010012cbcb |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/10480/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/10480/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10480/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10480/console |
| Powered by |
[jira] [Resolved] (HBASE-19272) Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again...
[ https://issues.apache.org/jira/browse/HBASE-19272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-19272. --- Resolution: Fixed Assignee: stack I pushed .001 to branch-2 and master. It just removes the named tests. > Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again... > -- > > Key: HBASE-19272 > URL: https://issues.apache.org/jira/browse/HBASE-19272 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack >Assignee: stack > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19272.master.001.patch > > > DIsabled by HBASE-14614, enabling AMv2. See HBASE-18110. > Here is the list: > * TestHBaseFsckTwoRS > * TestOfflineMetaRebuildBase > * TestHBaseFsckReplicas > * TestOfflineMetaRebuildOverlap > * TestHBaseFsckOneRS > * TestOfflineMetaRebuildHole -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19272) Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again...
[ https://issues.apache.org/jira/browse/HBASE-19272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19272: -- Summary: Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again... (was: Reenable HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again...) > Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again... > -- > > Key: HBASE-19272 > URL: https://issues.apache.org/jira/browse/HBASE-19272 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19272.master.001.patch > > > DIsabled by HBASE-14614, enabling AMv2. See HBASE-18110. > Here is the list: > * TestHBaseFsckTwoRS > * TestOfflineMetaRebuildBase > * TestHBaseFsckReplicas > * TestOfflineMetaRebuildOverlap > * TestHBaseFsckOneRS > * TestOfflineMetaRebuildHole -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19272) Reenable HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again...
[ https://issues.apache.org/jira/browse/HBASE-19272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19272: -- Attachment: HBASE-19272.master.001.patch > Reenable HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again... > - > > Key: HBASE-19272 > URL: https://issues.apache.org/jira/browse/HBASE-19272 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19272.master.001.patch > > > DIsabled by HBASE-14614, enabling AMv2. See HBASE-18110. > Here is the list: > * TestHBaseFsckTwoRS > * TestOfflineMetaRebuildBase > * TestHBaseFsckReplicas > * TestOfflineMetaRebuildOverlap > * TestHBaseFsckOneRS > * TestOfflineMetaRebuildHole -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19121) HBCK for AMv2
[ https://issues.apache.org/jira/browse/HBASE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293135#comment-16293135 ] stack commented on HBASE-19121: --- For 'repairs' an hbck2 might want to effect, see current hbck. See also the old tests listed in HBASE-19272 that are on branch-1 but now removed from branch-2. See what sort of 'repairs' the tests were verifying. > HBCK for AMv2 > - > > Key: HBASE-19121 > URL: https://issues.apache.org/jira/browse/HBASE-19121 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: stack > Fix For: 2.0.0-beta-2 > > > We don't have an hbck for the new AM. Old hbck may actually do damage going > against AMv2. > Fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19272) Reenable HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again...
[ https://issues.apache.org/jira/browse/HBASE-19272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293131#comment-16293131 ] stack commented on HBASE-19272: --- TestHBaseFsckTwoRS has to be rewritten. Manipulates the AM directly. Not allowed anymore. And it needs HBCKv1 verification, not happening in hbase2. Remove. TestOfflineMetaRebuildBase is an hbck test. Remove from hbase2. TestHBaseFsckReplicas is all about running hbck for various replica scenarios. Needs recast. Removing. TestOfflineMetaRebuildOverlap does something you can't do anymore -- operating on an offlined meta.. Need to have master online changing meta now. Removing. TestHBaseFsckOneRS is an HBCK test. Remove. Ditto TestOfflineMetaRebuildHole. Going to just remove these tests. > Reenable HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again... > - > > Key: HBASE-19272 > URL: https://issues.apache.org/jira/browse/HBASE-19272 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: stack > Fix For: 2.0.0-beta-1 > > > DIsabled by HBASE-14614, enabling AMv2. See HBASE-18110. > Here is the list: > * TestHBaseFsckTwoRS > * TestOfflineMetaRebuildBase > * TestHBaseFsckReplicas > * TestOfflineMetaRebuildOverlap > * TestHBaseFsckOneRS > * TestOfflineMetaRebuildHole -- This message was sent by Atlassian JIRA (v6.4.14#64029)
