[jira] [Commented] (HBASE-21310) Split TestCloneSnapshotFromClient
[
https://issues.apache.org/jira/browse/HBASE-21310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649716#comment-16649716
]
Hadoop QA commented on HBASE-21310:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} hbase-server: The patch generated 0 new + 6
unchanged - 5 fixed = 6 total (was 11) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}129m
13s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21310 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943857/HBASE-21310-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 73faf7eceb32 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / dde336f6ef |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14695/testReport/ |
| Max. process+thread count | 5459 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/14695/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This
[jira] [Commented] (HBASE-21260) The whole balancer plans might be aborted if there are more than one plans to move a same region
[
https://issues.apache.org/jira/browse/HBASE-21260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649714#comment-16649714
]
Hadoop QA commented on HBASE-21260:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
40s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 46s{color}
| {color:red} hbase-server generated 1 new + 187 unchanged - 1 fixed = 188
total (was 188) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 1s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}130m 20s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:42ca976 |
| JIRA Issue | HBASE-21260 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943178/HBASE-21260.branch-2.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux a766b49b3c1c 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 6125872f48 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| javac |
[jira] [Commented] (HBASE-21291) Add a test for bypassing stuck state-machine procedures
[
https://issues.apache.org/jira/browse/HBASE-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649695#comment-16649695
]
Jingyun Tian commented on HBASE-21291:
--
[~allan163] Could you help commit this patch?
> Add a test for bypassing stuck state-machine procedures
> ---
>
> Key: HBASE-21291
> URL: https://issues.apache.org/jira/browse/HBASE-21291
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Attachments: HBASE-21291.master.001.patch,
> HBASE-21291.master.002.patch, HBASE-21291.master.003.patch,
> HBASE-21291.master.004.patch, HBASE-21291.master.005.patch
>
>
> {code}
> if (!procedure.isFailed()) {
> if (subprocs != null) {
> if (subprocs.length == 1 && subprocs[0] == procedure) {
> // Procedure returned itself. Quick-shortcut for a state
> machine-like procedure;
> // i.e. we go around this loop again rather than go back out on
> the scheduler queue.
> subprocs = null;
> reExecute = true;
> LOG.trace("Short-circuit to next step on pid={}",
> procedure.getProcId());
> } else {
> // Yield the current procedure, and make the subprocedure runnable
> // subprocs may come back 'null'.
> subprocs = initializeChildren(procStack, procedure, subprocs);
> LOG.info("Initialized subprocedures=" +
> (subprocs == null? null:
> Stream.of(subprocs).map(e -> "{" + e.toString() + "}").
> collect(Collectors.toList()).toString()));
> }
> } else if (procedure.getState() == ProcedureState.WAITING_TIMEOUT) {
> LOG.debug("Added to timeoutExecutor {}", procedure);
> timeoutExecutor.add(procedure);
> } else if (!suspended) {
> // No subtask, so we are done
> procedure.setState(ProcedureState.SUCCESS);
> }
> }
> {code}
> Currently implementation of ProcedureExecutor will set the reExcecute to true
> for state machine like procedure. Then if this procedure is stuck at one
> certain state, it will loop forever.
> {code}
> IdLock.Entry lockEntry =
> procExecutionLock.getLockEntry(proc.getProcId());
> try {
> executeProcedure(proc);
> } catch (AssertionError e) {
> LOG.info("ASSERT pid=" + proc.getProcId(), e);
> throw e;
> } finally {
> procExecutionLock.releaseLockEntry(lockEntry);
> {code}
> Since procedure will get the IdLock and release it after execution done,
> state machine procedure will never release IdLock until it is finished.
> Then bypassProcedure doesn't work because is will try to grab the IdLock at
> first.
> {code}
> IdLock.Entry lockEntry =
> procExecutionLock.tryLockEntry(procedure.getProcId(), lockWait);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21315) The getActiveMinProcId and getActiveMaxProcId of BitSetNode are incorrect if there are no active procedure
[ https://issues.apache.org/jira/browse/HBASE-21315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21315: -- Issue Type: Sub-task (was: Bug) Parent: HBASE-20828 > The getActiveMinProcId and getActiveMaxProcId of BitSetNode are incorrect if > there are no active procedure > -- > > Key: HBASE-21315 > URL: https://issues.apache.org/jira/browse/HBASE-21315 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > > Found by [~allan163]. If no active procedure, we will return the start as the > min proc id, and getEnd() as the max proc id. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21315) The getActiveMinProcId and getActiveMaxProcId of BitSetNode are incorrect if there are no active procedure
Duo Zhang created HBASE-21315: - Summary: The getActiveMinProcId and getActiveMaxProcId of BitSetNode are incorrect if there are no active procedure Key: HBASE-21315 URL: https://issues.apache.org/jira/browse/HBASE-21315 Project: HBase Issue Type: Bug Reporter: Duo Zhang Found by [~allan163]. If no active procedure, we will return the start as the min proc id, and getEnd() as the max proc id. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20623) Introduce the helper method "getCellBuilder()" to Mutation
[
https://issues.apache.org/jira/browse/HBASE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649686#comment-16649686
]
Chia-Ping Tsai commented on HBASE-20623:
bq. the cell doesn't have a getRow method for me to assert,so in the UT
Consider using CellUtil.cloneRow.
bq. I cell.setRow(Bytes.toBytes("123")) and get no this row to check setRow is
useless.WDYT?
LGTM. Please do the same check for data type.
> Introduce the helper method "getCellBuilder()" to Mutation
> --
>
> Key: HBASE-20623
> URL: https://issues.apache.org/jira/browse/HBASE-20623
> Project: HBase
> Issue Type: Task
> Components: API
>Reporter: Chia-Ping Tsai
>Assignee: maoling
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20623.master.001.patch,
> HBASE-20623.master.002.patch, HBASE-20623.master.003.patch,
> HBASE-20623.master.004.patch, HBASE-20623.master.005.patch,
> HBASE-20623.master.006.patch, HBASE-20623.master.007.patch
>
>
> see
> [https://lists.apache.org/thread.html/d05bfaa0134502a47f6e1aca56cb0b096d4dd32ddefbbdf28db4952a@%3Cdev.hbase.apache.org%3E]
> for more details.
> {code:java}
> How about a "getCellBuilder" or "getCellBuilderFactory" method for
> Mutation implementations that gives you a CellBuilder instance that
> already has relevant parts set? Like for a Put instance it should be
> able to already have the Type and Row set.{code}
> mentioned a day or so ago by [~busbey]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20359) why Comparator extends comparable?
[ https://issues.apache.org/jira/browse/HBASE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649682#comment-16649682 ] Nick.han edited comment on HBASE-20359 at 10/15/18 3:20 AM: [~maoling] sorry,I reviewed this question,and it missed something,my bad,what I mean is class ByteArrayComparable and it's subclass LongComparator,BinaryComparator ...etc,ByteArrayComparable is not named well beacuse it's subclass is Comparator, a comparator is a tool,it's not right that a comparator extends a comparable. was (Author: nick.han): sorry,I reviewed this question,and it missed something,my bad,what I mean is class ByteArrayComparable and it's subclass LongComparator,BinaryComparator ...etc,ByteArrayComparable is not named well beacuse it's subclass is Comparator, a comparator is a tool,it's not right that a comparator extends a comparable. > why Comparator extends comparable? > -- > > Key: HBASE-20359 > URL: https://issues.apache.org/jira/browse/HBASE-20359 > Project: HBase > Issue Type: Improvement >Reporter: Nick.han >Priority: Minor > > Comparable is an interface,object implements this interface can be compare > with some other object, Comparator is a tool for compare object,the tool > itself can be compare is useless, so why a tool implement the comparable > interface?Do we have to improve this? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20359) why Comparator extends comparable?
[ https://issues.apache.org/jira/browse/HBASE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649682#comment-16649682 ] Nick.han commented on HBASE-20359: -- sorry,I reviewed this question,and it missed something,my bad,what I mean is class ByteArrayComparable and it's subclass LongComparator,BinaryComparator ...etc,ByteArrayComparable is not named well beacuse it's subclass is Comparator, a comparator is a tool,it's not right that a comparator extends a comparable. > why Comparator extends comparable? > -- > > Key: HBASE-20359 > URL: https://issues.apache.org/jira/browse/HBASE-20359 > Project: HBase > Issue Type: Improvement >Reporter: Nick.han >Priority: Minor > > Comparable is an interface,object implements this interface can be compare > with some other object, Comparator is a tool for compare object,the tool > itself can be compare is useless, so why a tool implement the comparable > interface?Do we have to improve this? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21314) The implementation of BitSetNode is not efficient
Duo Zhang created HBASE-21314: - Summary: The implementation of BitSetNode is not efficient Key: HBASE-21314 URL: https://issues.apache.org/jira/browse/HBASE-21314 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang As the MAX_NODE_SIZE is the same with BITS_PER_WORD, which means that we could only have one word(long) for each BitSetNode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21313) TransitRegionStateProcedure should not fail with FAILED_OPEN when acting as a sub procedure
Duo Zhang created HBASE-21313: - Summary: TransitRegionStateProcedure should not fail with FAILED_OPEN when acting as a sub procedure Key: HBASE-21313 URL: https://issues.apache.org/jira/browse/HBASE-21313 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang Since it will cause the parent procedure to fail, and maybe the parent procedure has already passed the PONR. Maybe here we should reply on the bypass logic. The procedure will hang there forever, until we bypass it, and use HBCK2 to recover the problems. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20528) Revise collections copying from iteration to built-in function
[
https://issues.apache.org/jira/browse/HBASE-20528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649673#comment-16649673
]
Chia-Ping Tsai commented on HBASE-20528:
{code}
-for (int i = 0; i < this.curFunctionCosts.length; i++) {
- curFunctionCosts[i] = tempFunctionCosts[i];
-}
+curFunctionCosts = Arrays.copyOf(tempFunctionCosts,
this.curFunctionCosts.length);
{code}
seems this change makes a duplicate clone of curFunctionCosts...(since
curFunctionCosts is initialized by StochasticLoadBalancer.setConf). Could you
please revert it? sorry for my previous comments :(
> Revise collections copying from iteration to built-in function
> --
>
> Key: HBASE-20528
> URL: https://issues.apache.org/jira/browse/HBASE-20528
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Hua-Yi Ho
>Assignee: Hua-Yi Ho
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-20528.master.001.patch
>
>
> Some collection codes in file
> StochasticLoadBalancer.java, AbstractHBaseTool.java, HFileInputFormat.java,
> Result.java, and WalPlayer.java, using iterations to copy whole data in
> collections. The iterations can just replace by just Colletions.addAll and
> Arrays.copyOf.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21278:
--
Issue Type: Sub-task (was: Bug)
Parent: HBASE-20828
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278-v1.patch, HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21256) Improve IntegrationTestBigLinkedList for testing huge data
[ https://issues.apache.org/jira/browse/HBASE-21256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21256: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Improve IntegrationTestBigLinkedList for testing huge data > -- > > Key: HBASE-21256 > URL: https://issues.apache.org/jira/browse/HBASE-21256 > Project: HBase > Issue Type: Improvement > Components: integration tests >Affects Versions: 3.0.0 >Reporter: Zephyr Guo >Assignee: Zephyr Guo >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.2.0 > > Attachments: HBASE-21256-branch-1.patch, HBASE-21256-v1.patch, > HBASE-21256-v10.patch, HBASE-21256-v11.patch, HBASE-21256-v12.patch, > HBASE-21256-v2.patch, HBASE-21256-v3.patch, HBASE-21256-v4.patch, > ITBLL-1.png, ITBLL-2.png > > > Recently, I use ITBLL to test some features in our company. I have > encountered the following problems: > > 1. Generator is too slow at the generating stage, the root cause is > SecureRandom. There is a global lock in SecureRandom( See the following > picture). I use Random instead of SecureRandom, and it could speed up this > stage(500% up with 20 mapper). SecureRandom was brought by HBASE-13382, but > speaking of generating random bytes, in my opnion, > it is the same with Random. > !ITBLL-1.png! > 2. VerifyReducer have a cpu cost of 14% on format method. This is cause by > create keyString variable. However, keyString may never be used if test > result is correct.(and that's in most cases). Just delay creating keyString > can yield huge performance boost in verifing stage. > !ITBLL-2.png! > 3.Arguments check is needed, because there's constraint between arguments. If > we broken this constraint, we can not get a correct circular list. > > 4.Let big family value size could be configured. > > 5.Avoid starting RS at backup master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21256) Improve IntegrationTestBigLinkedList for testing huge data
[ https://issues.apache.org/jira/browse/HBASE-21256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649671#comment-16649671 ] Duo Zhang commented on HBASE-21256: --- Pushed to branch-1 after fixing the checkstyle issues. Thanks [~apurtell] for the backporting. Thanks [~gzh1992n] for contributing. > Improve IntegrationTestBigLinkedList for testing huge data > -- > > Key: HBASE-21256 > URL: https://issues.apache.org/jira/browse/HBASE-21256 > Project: HBase > Issue Type: Improvement > Components: integration tests >Affects Versions: 3.0.0 >Reporter: Zephyr Guo >Assignee: Zephyr Guo >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.2.0 > > Attachments: HBASE-21256-branch-1.patch, HBASE-21256-v1.patch, > HBASE-21256-v10.patch, HBASE-21256-v11.patch, HBASE-21256-v12.patch, > HBASE-21256-v2.patch, HBASE-21256-v3.patch, HBASE-21256-v4.patch, > ITBLL-1.png, ITBLL-2.png > > > Recently, I use ITBLL to test some features in our company. I have > encountered the following problems: > > 1. Generator is too slow at the generating stage, the root cause is > SecureRandom. There is a global lock in SecureRandom( See the following > picture). I use Random instead of SecureRandom, and it could speed up this > stage(500% up with 20 mapper). SecureRandom was brought by HBASE-13382, but > speaking of generating random bytes, in my opnion, > it is the same with Random. > !ITBLL-1.png! > 2. VerifyReducer have a cpu cost of 14% on format method. This is cause by > create keyString variable. However, keyString may never be used if test > result is correct.(and that's in most cases). Just delay creating keyString > can yield huge performance boost in verifing stage. > !ITBLL-2.png! > 3.Arguments check is needed, because there's constraint between arguments. If > we broken this constraint, we can not get a correct circular list. > > 4.Let big family value size could be configured. > > 5.Avoid starting RS at backup master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649637#comment-16649637
]
Duo Zhang commented on HBASE-21278:
---
If no other concerns let's commit this? Will prepare the patch for branch-2.0 &
branch-2.1 in another issue, as the assign/unassign are a bit different...
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278-v1.patch, HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21260) The whole balancer plans might be aborted if there are more than one plans to move a same region
[
https://issues.apache.org/jira/browse/HBASE-21260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21260:
--
Fix Version/s: 2.2.0
3.0.0
Affects Version/s: (was: 2.1.0)
(was: 2.0.0)
Status: Patch Available (was: Open)
> The whole balancer plans might be aborted if there are more than one plans to
> move a same region
> -
>
> Key: HBASE-21260
> URL: https://issues.apache.org/jira/browse/HBASE-21260
> Project: HBase
> Issue Type: Bug
> Components: Balancer, master
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21260.branch-2.001.patch,
> HBASE-21260.branch-2.002.patch
>
>
> In SimpleLoadBalancer, plans are generated firstly by average number regions
> per server for a table. Each server will be randomly assigned either
> floor(average) or ceiling(average) regions (if the average is not an integer
> number). But afterwards, the balanceOverall method might generate new plans
> of some regions of the table to balance server loads in whole cluster scope.
> As a result, there are plans to move a same region in one call of balance.
> Currently, branch-2 is using async procedures to implement balancer plans.
> But the concurrency of moving the same regions will cause the balance method
> failed. And all the afterwards plans will not be implement when one plan
> encounters exception.
> We have encountered this problem in our practices, the logs are as follows,
> {color:#205081}2018-09-26,12:12:38,224 INFO
> [master/c4-hadoop-tst-ct15:52900.Chore.1]
> org.apache.hadoop.hbase.master.HMaster: Balancer plans size is 3757, the
> balance interval is 79 ms, and the max number regions in transition is 25
> 2018-09-26,12:12:38,224 INFO [master/c4-hadoop-tst-ct15:52900.Chore.1]
> org.apache.hadoop.hbase.master.HMaster: balance hri=1588230740,
> source=c4-hadoop-tst-st99.bj,52900,1537522783781,
> destination=c4-hadoop-tst-st28.bj,52900,1537520009497
> 2018-09-26,12:12:38,325 INFO [master/c4-hadoop-tst-ct15:52900.Chore.1]
> org.apache.hadoop.hbase.master.HMaster: balance hri=1588230740,
> source=c4-hadoop-tst-st99.bj,52900,1537522783781,
> destination=c4-hadoop-tst-st29.bj,52900,1537522784188
> 2018-09-26,12:12:38,325 INFO [PEWorker-16]
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler:
> pid=119197, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE;
> TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE
> checking lock on 1588230740
> 2018-09-26,12:12:38,325 ERROR [master/c4-hadoop-tst-ct15:52900.Chore.1]
> org.apache.hadoop.hbase.master.balancer.BalancerChore: Failed to balance.
> org.apache.hadoop.hbase.HBaseIOException: rit=OPEN,
> location=c4-hadoop-tst-st99.bj,52900,1537522783781, table=hbase:meta,
> region=1588230740 is currently in transition
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.preTransitCheck(AssignmentManager.java:536)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createMoveRegionProcedure(AssignmentManager.java:592)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.moveAsync(AssignmentManager.java:609)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1707)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1622)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerChore.chore(BalancerChore.java:49)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){color}
> This is a serious problem because it often occurs when new RSs started or old
> RSs failover. And what's more, no effective methods can be used to make the
> balance of the cluster back to normal.
> But the solution of this problem may be simple. We can cache Exceptions
[jira] [Commented] (HBASE-21288) HostingServer in UnassignProcedure is not accurate
[
https://issues.apache.org/jira/browse/HBASE-21288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649620#comment-16649620
]
Allan Yang commented on HBASE-21288:
{quote}
Can this not get the server from the RegionNode?
{quote}
No, we need a MasterProcedureEnv passed in to get the AssignmentManager, which
the toStringClassDetails() doesn't have.
> HostingServer in UnassignProcedure is not accurate
> --
>
> Key: HBASE-21288
> URL: https://issues.apache.org/jira/browse/HBASE-21288
> Project: HBase
> Issue Type: Sub-task
> Components: amv2, Balancer
>Affects Versions: 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-21288.branch-2.0.001.patch,
> HBASE-21288.branch-2.0.002.patch
>
>
> We have a case that a region shows status OPEN on a already dead server in
> meta table(it is hard to trace how this happen), meaning this region is
> actually not online. But balance came and scheduled a MoveReionProcedure for
> this region, which created a mess:
> The balancer 'thought' this region was on the server which has the same
> address(but with different startcode). So it schedules a MRP from this online
> server to another, but the UnassignProcedure dispatch the unassign call to
> the dead server according to regionstate, which then found the server dead
> and schedule a SCP for the dead server. But since the UnassignProcedure's
> hostingServer is not accurate, the SCP can't interrupt it.
> So, in the end, the SCP can't finish since the UnassignProcedure has the
> region' lock, the UnassignProcedure can not finish since no one wake it, thus
> stuck.
> Here is log, notice that the server of the UnassignProcedure is
> 'hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539153278584' but it was
> dispatch to 'hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539076734964'
> {code}
> 2018-10-10 14:34:50,011 INFO [PEWorker-4]
> assignment.RegionTransitionProcedure(252): Dispatch pid=13, ppid=12,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, hasLock=true; UnassignProcedure
> table=hbase:acl, region=267335c85766c62479fb4a5f18a1e95f,
> server=hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539153278584; rit=CLOSING,
> location=hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539076734964
> 2018-10-10 14:34:50,011 WARN [PEWorker-4]
> assignment.RegionTransitionProcedure(230): Remote call failed
> hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539076734964; pid=13, ppid=12,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, hasLock=true; UnassignProcedure
> table=hbase:acl, region=267335c85766c62479fb4a5f18a1e95f,
> server=hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539153278584; rit=CLOSING,
> location=hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539076734964;
> exception=NoServerDispatchException
> org.apache.hadoop.hbase.procedure2.NoServerDispatchException:
> hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539076734964; pid=13, ppid=12,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, hasLock=true; UnassignProcedure
> table=hbase:acl, region=267335c85766c62479fb4a5f18a1e95f,
> server=hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539153278584
> //Then a SCP was scheduled
> 2018-10-10 14:34:50,012 WARN [PEWorker-4] master.ServerManager(635):
> Expiration of hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539076734964 but
> server not online
> 2018-10-10 14:34:50,012 INFO [PEWorker-4] master.ServerManager(615):
> Processing expiration of hb-uf6oyi699w8h700f0-003.hbase.rds.
> ,16020,1539076734964 on hb-uf6oyi699w8h700f0-001.hbase.rds.
> ,16000,1539088156164
> 2018-10-10 14:34:50,017 DEBUG [PEWorker-4]
> procedure2.ProcedureExecutor(1089): Stored pid=14,
> state=RUNNABLE:SERVER_CRASH_START, hasLock=false; ServerCrashProcedure
> server=hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539076734964,
> splitWal=true, meta=false
> //The SCP did not interrupt the UnassignProcedure but schedule new
> AssignProcedure for this region
> 2018-10-10 14:34:50,043 DEBUG [PEWorker-6]
> procedure.ServerCrashProcedure(250): Done splitting WALs pid=14,
> state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, hasLock=true; ServerCrashProcedure
> server=hb-uf6oyi699w8h700f0-003.hbase.rds. ,16020,1539076734964,
> splitWal=true, meta=false
> 2018-10-10 14:34:50,054 INFO [PEWorker-8]
> procedure2.ProcedureExecutor(1691): Initialized subprocedures=[{pid=15,
> ppid=14, state=RUNNABLE:REGION_TRANSITION_QUEUE, hasLock=false;
> AssignProcedure table=hbase:acl, region=267335c85766c62479fb4a5f18a1e95f},
> {pid=16, ppid=14, state=RUNNABLE:REGION_TRANSITION_QUEUE, hasLock=false;
> AssignProcedure table=hbase:req_intercept_rule,
> region=460481706415d776b3742f428a6f579b}, {pid=17, ppid=14,
> state=RUNNABLE:REGION_TRANSITION_QUEUE, hasLock=false; AssignProcedure
> table=hbase:namespace,
[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure
[
https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649617#comment-16649617
]
Ted Yu commented on HBASE-21149:
Patch v2 breaks multiple source files into multiple activations of DistCp - one
source per activation.
TestIncrementalBackupWithBulkLoad passes with v2 against hadoop 3.1.1
> TestIncrementalBackupWithBulkLoad may fail due to file copy failure
> ---
>
> Key: HBASE-21149
> URL: https://issues.apache.org/jira/browse/HBASE-21149
> Project: HBase
> Issue Type: Test
> Components: backuprestore
>Reporter: Ted Yu
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: 21149.v2.txt, HBASE-21149-v1.patch,
> testIncrementalBackupWithBulkLoad-output.txt
>
>
> From
> https://builds.apache.org/job/HBase%20Nightly/job/master/471/testReport/junit/org.apache.hadoop.hbase.backup/TestIncrementalBackupWithBulkLoad/TestIncBackupDeleteTable/
> :
> {code}
> 2018-09-03 11:54:30,526 ERROR [Time-limited test]
> impl.TableBackupClient(235): Unexpected Exception : Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> java.io.IOException: Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.incrementalCopyHFiles(IncrementalTableBackupClient.java:351)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.copyBulkLoadedFiles(IncrementalTableBackupClient.java:219)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.handleBulkLoad(IncrementalTableBackupClient.java:198)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:320)
> at
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:605)
> at
> org.apache.hadoop.hbase.backup.TestIncrementalBackupWithBulkLoad.TestIncBackupDeleteTable(TestIncrementalBackupWithBulkLoad.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> {code}
> However, some part of the test output was lost:
> {code}
> 2018-09-03 11:53:36,793 DEBUG [RS:0;765c9ca5ea28:36357] regions
> ...[truncated 398396 chars]...
> 8)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure
[
https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-21149:
---
Attachment: 21149.v2.txt
> TestIncrementalBackupWithBulkLoad may fail due to file copy failure
> ---
>
> Key: HBASE-21149
> URL: https://issues.apache.org/jira/browse/HBASE-21149
> Project: HBase
> Issue Type: Test
> Components: backuprestore
>Reporter: Ted Yu
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: 21149.v2.txt, HBASE-21149-v1.patch,
> testIncrementalBackupWithBulkLoad-output.txt
>
>
> From
> https://builds.apache.org/job/HBase%20Nightly/job/master/471/testReport/junit/org.apache.hadoop.hbase.backup/TestIncrementalBackupWithBulkLoad/TestIncBackupDeleteTable/
> :
> {code}
> 2018-09-03 11:54:30,526 ERROR [Time-limited test]
> impl.TableBackupClient(235): Unexpected Exception : Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> java.io.IOException: Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.incrementalCopyHFiles(IncrementalTableBackupClient.java:351)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.copyBulkLoadedFiles(IncrementalTableBackupClient.java:219)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.handleBulkLoad(IncrementalTableBackupClient.java:198)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:320)
> at
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:605)
> at
> org.apache.hadoop.hbase.backup.TestIncrementalBackupWithBulkLoad.TestIncBackupDeleteTable(TestIncrementalBackupWithBulkLoad.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> {code}
> However, some part of the test output was lost:
> {code}
> 2018-09-03 11:53:36,793 DEBUG [RS:0;765c9ca5ea28:36357] regions
> ...[truncated 398396 chars]...
> 8)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure
[
https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649606#comment-16649606
]
Ted Yu commented on HBASE-21149:
CopyCommitter#concatFileChunks would throw the following exception when trying
to concatenate the two bulk loaded hfiles:
{code}
2018-10-13 14:09:25,351 WARN [Thread-936] mapred.LocalJobRunner$Job(590):
job_local1795473782_0004
java.io.IOException: Inconsistent sequence file: current chunk file
org.apache.hadoop.tools.CopyListingFileStatus@bb8826ee{hdfs://localhost:42796/user/hbase/test-data/
160aeab5-6bca-9f87-465e-2517a0c43119/data/default/test-1539439707496/96b5a3613d52f4df1ba87a1cef20684c/f/a7599081e835440eb7bf0dd3ef4fd7a5_SeqId_205_
length = 5100 aclEntries = null, xAttrs = null} doesnt match prior entry
org.apache.hadoop.tools.CopyListingFileStatus@243d544d{hdfs://localhost:42796/user/hbase/test-data/160aeab5-6bca-9f87-465e-
2517a0c43119/data/default/test-1539439707496/96b5a3613d52f4df1ba87a1cef20684c/f/394e6d39a9b94b148b9089c4fb967aad_SeqId_205_
length = 5142 aclEntries = null, xAttrs = null}
at
org.apache.hadoop.tools.mapred.CopyCommitter.concatFileChunks(CopyCommitter.java:276)
at
org.apache.hadoop.tools.mapred.CopyCommitter.commitJob(CopyCommitter.java:100)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:567)
{code}
> TestIncrementalBackupWithBulkLoad may fail due to file copy failure
> ---
>
> Key: HBASE-21149
> URL: https://issues.apache.org/jira/browse/HBASE-21149
> Project: HBase
> Issue Type: Test
> Components: backuprestore
>Reporter: Ted Yu
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-21149-v1.patch,
> testIncrementalBackupWithBulkLoad-output.txt
>
>
> From
> https://builds.apache.org/job/HBase%20Nightly/job/master/471/testReport/junit/org.apache.hadoop.hbase.backup/TestIncrementalBackupWithBulkLoad/TestIncBackupDeleteTable/
> :
> {code}
> 2018-09-03 11:54:30,526 ERROR [Time-limited test]
> impl.TableBackupClient(235): Unexpected Exception : Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> java.io.IOException: Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.incrementalCopyHFiles(IncrementalTableBackupClient.java:351)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.copyBulkLoadedFiles(IncrementalTableBackupClient.java:219)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.handleBulkLoad(IncrementalTableBackupClient.java:198)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:320)
> at
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:605)
> at
> org.apache.hadoop.hbase.backup.TestIncrementalBackupWithBulkLoad.TestIncBackupDeleteTable(TestIncrementalBackupWithBulkLoad.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> {code}
> However, some part of the test output was lost:
> {code}
> 2018-09-03 11:53:36,793 DEBUG [RS:0;765c9ca5ea28:36357] regions
> ...[truncated 398396 chars]...
> 8)
> at
>
[jira] [Comment Edited] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure
[
https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642620#comment-16642620
]
Ted Yu edited comment on HBASE-21149 at 10/15/18 12:51 AM:
---
It seems the Warning was due to check during concatenation phase of
CopyCommitter.
The concatenation would happen when input file listing is present.
was (Author: yuzhih...@gmail.com):
It seems the Warning was due to check during concatenation phase of
CopyCommitter.
The concatenation would happen when input file listing is present.
I added debug logging at the beginning of
MapReduceBackupCopyJob$BackupDistCp#createInputFileListing which didn't show up
in test output.
> TestIncrementalBackupWithBulkLoad may fail due to file copy failure
> ---
>
> Key: HBASE-21149
> URL: https://issues.apache.org/jira/browse/HBASE-21149
> Project: HBase
> Issue Type: Test
> Components: backuprestore
>Reporter: Ted Yu
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-21149-v1.patch,
> testIncrementalBackupWithBulkLoad-output.txt
>
>
> From
> https://builds.apache.org/job/HBase%20Nightly/job/master/471/testReport/junit/org.apache.hadoop.hbase.backup/TestIncrementalBackupWithBulkLoad/TestIncBackupDeleteTable/
> :
> {code}
> 2018-09-03 11:54:30,526 ERROR [Time-limited test]
> impl.TableBackupClient(235): Unexpected Exception : Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> java.io.IOException: Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.incrementalCopyHFiles(IncrementalTableBackupClient.java:351)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.copyBulkLoadedFiles(IncrementalTableBackupClient.java:219)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.handleBulkLoad(IncrementalTableBackupClient.java:198)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:320)
> at
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:605)
> at
> org.apache.hadoop.hbase.backup.TestIncrementalBackupWithBulkLoad.TestIncBackupDeleteTable(TestIncrementalBackupWithBulkLoad.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> {code}
> However, some part of the test output was lost:
> {code}
> 2018-09-03 11:53:36,793 DEBUG [RS:0;765c9ca5ea28:36357] regions
> ...[truncated 398396 chars]...
> 8)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21310) Split TestCloneSnapshotFromClient
[ https://issues.apache.org/jira/browse/HBASE-21310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21310: -- Attachment: HBASE-21310-v1.patch > Split TestCloneSnapshotFromClient > - > > Key: HBASE-21310 > URL: https://issues.apache.org/jira/browse/HBASE-21310 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21310-v1.patch, HBASE-21310.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21310) Split TestCloneSnapshotFromClient
[ https://issues.apache.org/jira/browse/HBASE-21310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21310: -- Attachment: HBASE-21310-v1.patch > Split TestCloneSnapshotFromClient > - > > Key: HBASE-21310 > URL: https://issues.apache.org/jira/browse/HBASE-21310 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21310.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21310) Split TestCloneSnapshotFromClient
[ https://issues.apache.org/jira/browse/HBASE-21310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21310: -- Attachment: (was: HBASE-21310-v1.patch) > Split TestCloneSnapshotFromClient > - > > Key: HBASE-21310 > URL: https://issues.apache.org/jira/browse/HBASE-21310 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21310.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21281) Update bouncycastle dependency.
[
https://issues.apache.org/jira/browse/HBASE-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648708#comment-16648708
]
Ted Yu edited comment on HBASE-21281 at 10/14/18 9:18 PM:
--
The following dependency tree output against hadoop3 shows that bouncycastle
dependency was only present for hbase-http module:
{code}
[INFO] org.apache.hbase:hbase-http:jar:3.0.0-SNAPSHOT
...
[INFO] +- org.bouncycastle:bcprov-jdk15on:jar:1.60:test
{code}
Some tests in hbase-server module depend on bouncycastle as well (as the
failing tests showed).
was (Author: yuzhih...@gmail.com):
The following dependency tree output shows that old bouncycastle dependency was
pulled in:
{code}
[INFO] +- org.apache.hadoop:hadoop-minikdc:jar:2.7.7:test
[INFO] | \-
org.apache.directory.server:apacheds-protocol-ldap:jar:2.0.0-M15:test
[INFO] | +- org.apache.directory.api:api-asn1-ber:jar:1.0.0-M20:test
[INFO] | +-
org.apache.directory.api:api-ldap-extras-codec-api:jar:1.0.0-M20:test
[INFO] | +-
org.apache.directory.api:api-ldap-extras-codec:jar:1.0.0-M20:test
[INFO] | +- org.apache.directory.api:api-ldap-extras-sp:jar:1.0.0-M20:test
[INFO] | \- bouncycastle:bcprov-jdk15:jar:140:test
{code}
which we should exclude.
> Update bouncycastle dependency.
> ---
>
> Key: HBASE-21281
> URL: https://issues.apache.org/jira/browse/HBASE-21281
> Project: HBase
> Issue Type: Task
> Components: dependencies, test
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21281.addendum.patch, HBASE-21281.001.branch-2.0.patch
>
>
> Looks like we still depend on bcprov-jdk16 for some x509 certificate
> generation in our tests. Bouncycastle has moved beyond this in 1.47, changing
> the artifact names.
> [http://www.bouncycastle.org/wiki/display/JA1/Porting+from+earlier+BC+releases+to+1.47+and+later]
> There are some API changes too, but it looks like we don't use any of these.
> It seems like we also have vestiges in the POMs from when we were depending
> on a specific BC version that came in from Hadoop. We now have a
> KeyStoreTestUtil class in HBase, which makes me think we can also clean up
> some dependencies.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21281) Update bouncycastle dependency.
[ https://issues.apache.org/jira/browse/HBASE-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649535#comment-16649535 ] Ted Yu commented on HBASE-21281: Updated problem description above. > Update bouncycastle dependency. > --- > > Key: HBASE-21281 > URL: https://issues.apache.org/jira/browse/HBASE-21281 > Project: HBase > Issue Type: Task > Components: dependencies, test >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: 21281.addendum.patch, HBASE-21281.001.branch-2.0.patch > > > Looks like we still depend on bcprov-jdk16 for some x509 certificate > generation in our tests. Bouncycastle has moved beyond this in 1.47, changing > the artifact names. > [http://www.bouncycastle.org/wiki/display/JA1/Porting+from+earlier+BC+releases+to+1.47+and+later] > There are some API changes too, but it looks like we don't use any of these. > It seems like we also have vestiges in the POMs from when we were depending > on a specific BC version that came in from Hadoop. We now have a > KeyStoreTestUtil class in HBase, which makes me think we can also clean up > some dependencies. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21286) Parallelize computeHDFSBlocksDistribution when getting splits of a HBaseSnapshot
[
https://issues.apache.org/jira/browse/HBASE-21286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649534#comment-16649534
]
Hadoop QA commented on HBASE-21286:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.4 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
44s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
6m 48s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}101m
32s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:2cf636a |
| JIRA Issue | HBASE-21286 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943847/HBASE-21286.branch-1.4.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle
[jira] [Commented] (HBASE-21281) Update bouncycastle dependency.
[
https://issues.apache.org/jira/browse/HBASE-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649527#comment-16649527
]
Josh Elser commented on HBASE-21281:
{quote}The following dependency tree output shows that old bouncycastle
dependency was pulled in:
{quote}
This was from the Hadoop2 build, but the test only failed on Hadoop3. That
doesn't really make sense, does it?
> Update bouncycastle dependency.
> ---
>
> Key: HBASE-21281
> URL: https://issues.apache.org/jira/browse/HBASE-21281
> Project: HBase
> Issue Type: Task
> Components: dependencies, test
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21281.addendum.patch, HBASE-21281.001.branch-2.0.patch
>
>
> Looks like we still depend on bcprov-jdk16 for some x509 certificate
> generation in our tests. Bouncycastle has moved beyond this in 1.47, changing
> the artifact names.
> [http://www.bouncycastle.org/wiki/display/JA1/Porting+from+earlier+BC+releases+to+1.47+and+later]
> There are some API changes too, but it looks like we don't use any of these.
> It seems like we also have vestiges in the POMs from when we were depending
> on a specific BC version that came in from Hadoop. We now have a
> KeyStoreTestUtil class in HBase, which makes me think we can also clean up
> some dependencies.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21266) Not running balancer because processing dead regionservers, but empty dead rs list
[ https://issues.apache.org/jira/browse/HBASE-21266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649518#comment-16649518 ] Andrew Purtell commented on HBASE-21266: The 1B row ITBLL with serverKilling policy passed, twice > Not running balancer because processing dead regionservers, but empty dead rs > list > -- > > Key: HBASE-21266 > URL: https://issues.apache.org/jira/browse/HBASE-21266 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.8 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.9 > > Attachments: HBASE-21266-branch-1.patch, HBASE-21266-branch-1.patch, > HBASE-21266-branch-1.patch, HBASE-21266-branch-1.patch, > HBASE-21266-branch-1.patch, HBASE-21266-branch-1.patch, > HBASE-21266-branch-1.patch, HBASE-21266-branch-1.patch, > HBASE-21266-branch-1.patch, HBASE-21266-branch-1.patch, > HBASE-21266-branch-1.patch, HBASE-21266.branch-2.1.001.patch > > > Found during ITBLL testing. AM in master gets into a state where manual > attempts from the shell to run the balancer always return false and this is > printed in the master log: > 2018-10-03 19:17:14,892 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=21,queue=0,port=8100] master.HMaster: > Not running balancer because processing dead regionserver(s): > Note the empty list. > This errant state did not recover without intervention by way of master > restart, but the test environment was chaotic so needs investigation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21286) Parallelize computeHDFSBlocksDistribution when getting splits of a HBaseSnapshot
[ https://issues.apache.org/jira/browse/HBASE-21286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lavinia-Stefania Sirbu updated HBASE-21286: --- Attachment: HBASE-21286.branch-1.4.003.patch Status: Patch Available (was: Open) > Parallelize computeHDFSBlocksDistribution when getting splits of a > HBaseSnapshot > > > Key: HBASE-21286 > URL: https://issues.apache.org/jira/browse/HBASE-21286 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 1.4.0 >Reporter: Lavinia-Stefania Sirbu >Assignee: Lavinia-Stefania Sirbu >Priority: Minor > Attachments: HBASE-21286.branch-1.4.001.patch, > HBASE-21286.branch-1.4.002.patch, HBASE-21286.branch-1.4.003.patch > > > Even if this step is called computeHDFSBlocksDistribution, this is executed > no matter the file system of the snapshot. For example, we have observed an > important slowness when we have a snapshot in s3 (~26k regions, 5column > families, 2 files per column family) the getsplits time is ~40min due to the > calls in s3 for listing the files to get the best locations. > Parallelizing this operation can reduce the overall setup time. The thread > pool should be configurable and a good choice could be > "hbase.snapshot.thread.pool.max" that is also used in RestoreSnapshotHelper. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21286) Parallelize computeHDFSBlocksDistribution when getting splits of a HBaseSnapshot
[ https://issues.apache.org/jira/browse/HBASE-21286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lavinia-Stefania Sirbu updated HBASE-21286: --- Status: Open (was: Patch Available) > Parallelize computeHDFSBlocksDistribution when getting splits of a > HBaseSnapshot > > > Key: HBASE-21286 > URL: https://issues.apache.org/jira/browse/HBASE-21286 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 1.4.0 >Reporter: Lavinia-Stefania Sirbu >Assignee: Lavinia-Stefania Sirbu >Priority: Minor > Attachments: HBASE-21286.branch-1.4.001.patch, > HBASE-21286.branch-1.4.002.patch > > > Even if this step is called computeHDFSBlocksDistribution, this is executed > no matter the file system of the snapshot. For example, we have observed an > important slowness when we have a snapshot in s3 (~26k regions, 5column > families, 2 files per column family) the getsplits time is ~40min due to the > calls in s3 for listing the files to get the best locations. > Parallelizing this operation can reduce the overall setup time. The thread > pool should be configurable and a good choice could be > "hbase.snapshot.thread.pool.max" that is also used in RestoreSnapshotHelper. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-14320) b.a.o Jenkins is obscuring test results on HBase-1.2 and HBase-1.3
[ https://issues.apache.org/jira/browse/HBASE-14320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-14320. --- Resolution: Invalid Not an issue anymore. > b.a.o Jenkins is obscuring test results on HBase-1.2 and HBase-1.3 > -- > > Key: HBASE-14320 > URL: https://issues.apache.org/jira/browse/HBASE-14320 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.2.0, 1.3.0 >Reporter: Dima Spivak >Priority: Major > > Our normal Jenkins setup up on builds.apache.org (e.g. [this > one|https://builds.apache.org/job/HBase-TRUNK/lastCompletedBuild/testReport/]) > makes it easy to get test reports on the latest builds, as well as providing > some [pretty history > views|https://builds.apache.org/job/HBase-TRUNK/lastCompletedBuild/testReport/history/]. > This functionality seems to have been inadvertently broken when we turned > the [1.2|https://builds.apache.org/job/HBase-1.2/] and > [1.3|https://builds.apache.org/job/HBase-1.3/] jobs into matrix jobs in order > to test jdk7 and jdk8 in parallel. Should be an easy fix on the Jenkins side > to sort this out... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-14198) Eclipse project generation is broken in master
[
https://issues.apache.org/jira/browse/HBASE-14198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-14198.
---
Resolution: Invalid
> Eclipse project generation is broken in master
> --
>
> Key: HBASE-14198
> URL: https://issues.apache.org/jira/browse/HBASE-14198
> Project: HBase
> Issue Type: Bug
>Reporter: Vladimir Rodionov
>Priority: Major
>
> After running
> mvn eclipse:eclipse I tried to import projects into Eclipse (Luna) and got
> multiple build errors, similar to:
> {code}
> Cannot nest output folder 'hbase-thrift/target/test-classes/META-INF' inside
> output folder 'hbase-thrift/target/test-classes' hbase-thrift
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-13676) TestRegionServerHostname#testInvalidRegionServerHostnameAbortsServer is failing
[
https://issues.apache.org/jira/browse/HBASE-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-13676.
---
Resolution: Cannot Reproduce
> TestRegionServerHostname#testInvalidRegionServerHostnameAbortsServer is
> failing
> ---
>
> Key: HBASE-13676
> URL: https://issues.apache.org/jira/browse/HBASE-13676
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Andrew Purtell
>Priority: Minor
>
> {noformat}
> Running org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.543 sec
> <<< FAILURE! - in
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> testInvalidRegionServerHostnameAbortsServer(org.apache.hadoop.hbase.regionserver.TestRegionServerHostname)
> Time elapsed: 6.845 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testInvalidRegionServerHostnameAbortsServer(TestRegionServerHostname.java:57)
> Results :
> Failed tests:
> TestRegionServerHostname.testInvalidRegionServerHostnameAbortsServer:57 null
> {noformat}
> The exception message is not what the test expects. We want a string
> containing "Failed resolve of " + invalidHostname. What I have is
> "java.net.BindException: Problem binding to
> hostAddr.invalid/198.105.244.228:0 : Cannot assign requested address."
> This is because my ISP is "helpfully" providing A records for invalid
> hostnames.
> {noformat}
> apurtell@aspire ~ $ dig hostAddr.invalid
> ; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> hostAddr.invalid
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49027
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
> ;; QUESTION SECTION:
> ;hostAddr.invalid.IN A
> ;; ANSWER SECTION:
> hostAddr.invalid. 10 IN A 198.105.244.228
> hostAddr.invalid. 10 IN A 198.105.254.228
> ;; Query time: 27 msec
> ;; SERVER: 127.0.1.1#53(127.0.1.1)
> ;; WHEN: Tue May 12 13:34:21 PDT 2015
> ;; MSG SIZE rcvd: 66
> {noformat}
> The test should be made more general to capture this kind of failure too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-12154) TestLoadAndSwitchEncodeOnDisk is flaky on shared jenkins boxes
[ https://issues.apache.org/jira/browse/HBASE-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-12154. --- Resolution: Cannot Reproduce > TestLoadAndSwitchEncodeOnDisk is flaky on shared jenkins boxes > -- > > Key: HBASE-12154 > URL: https://issues.apache.org/jira/browse/HBASE-12154 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Minor > Original Estimate: 168h > Remaining Estimate: 168h > > It doesn't make sense to run a load test on a shared box where other tests > are being run. We should probably move this to integration tests and make > sure its covered. In cases where the jenkins machines are shared across > several projects which are doing disk IO, I have observed that this test runs > into slow syncs and eventually times out. And that being said, we can't > increase the timeout on this test, since its already 3 minutes. This test > being disk sensitive, might have better coverage on IT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-10977) TestHBaseFsck.testQuarantineMissingHFile fails missing files test
[
https://issues.apache.org/jira/browse/HBASE-10977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-10977.
---
Resolution: Cannot Reproduce
> TestHBaseFsck.testQuarantineMissingHFile fails missing files test
> -
>
> Key: HBASE-10977
> URL: https://issues.apache.org/jira/browse/HBASE-10977
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.15
>Reporter: stack
>Priority: Minor
>
> On our internal rig, ran into this failure:
> {code}
> java.lang.AssertionError: expected:<2> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.doQuarantineTest(TestHBaseFsck.java:1737)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineMissingHFile(TestHBaseFsck.java:1781)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {code}
> This is what is failing:
> assertEquals(hfcc.getMissing().size(), missing);
> The file remove has not run yet or, else, this complaint is related:
> {code}
> 2014-04-12 23:24:57,638 WARN [IPC Server handler 4 on 50919]
> security.UserGroupInformation(1551): PriviledgedActionException as:jenkins
> (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not exist:
> /user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/fam/57a07eaac97e4acd8dc04e08d1950adc
> {code}
> Below is full log. Will come back and add logging
> {code}
> Regression
> org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineMissingHFile
> Failing for the past 1 build (Since Failed#3 )
> Took 10 ms.
> add description
> Error Message
> expected:<2> but was:<1>
> Stacktrace
> java.lang.AssertionError: expected:<2> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.doQuarantineTest(TestHBaseFsck.java:1737)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineMissingHFile(TestHBaseFsck.java:1781)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> Standard Output
> Allow checking/fixes for table: testQuarantineMissingHFile
> Checked 4 hfile for corruption
> HFiles corrupted: 0
> HFiles successfully quarantined: 0
> HFiles failed quarantine:0
> HFiles moved while checking: 2
>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/fam/57a07eaac97e4acd8dc04e08d1950adc
>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351/fam/9dd79f30f29e4cfeaa46f6e20b32e078
> Summary: OK => OK
> Version: 0.96.1.1-cdh5.0.1-SNAPSHOT
> Table 'testQuarantineMissingHFile': region split map
> : [ { meta => null, hdfs =>
>
[jira] [Resolved] (HBASE-11087) Eclipse Import Problem
[ https://issues.apache.org/jira/browse/HBASE-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-11087. --- Resolution: Cannot Reproduce > Eclipse Import Problem > -- > > Key: HBASE-11087 > URL: https://issues.apache.org/jira/browse/HBASE-11087 > Project: HBase > Issue Type: Bug > Environment: Eclipse version is 4.3 Build Id : 20140224-0627 > M2E plugin version : 1.4.0 >Reporter: Talat UYARER >Assignee: Talat UYARER >Priority: Minor > Attachments: HBASE-11087.patch > > > If you try to import eclipse you will get some import errors about maven > plugins. [1] So, in order to avoid the exceptions in Eclipse, looks like one > needs to simply enclose all the plugin tags inside a tag. > I create a patch for this problem. > [1] > http://mail-archives.apache.org/mod_mbox/hbase-dev/201404.mbox/%3CCAEz%2Byv-hMdjpue91TSh%2B1YdGW8oqo5TXPeH09Y4dntvtPro2bQ%40mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-10710) TestLogRolling.testLogRollOnDatanodeDeath fails occasionally in Hadoop QA run
[ https://issues.apache.org/jira/browse/HBASE-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-10710. --- Resolution: Cannot Reproduce > TestLogRolling.testLogRollOnDatanodeDeath fails occasionally in Hadoop QA run > - > > Key: HBASE-10710 > URL: https://issues.apache.org/jira/browse/HBASE-10710 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Honghua Feng >Priority: Major > > This case ever failed in last month for HBASE-10648 / HBASE-10615 / > HBASE-9990 / HBASE-10582 / HBASE-10527 / HBASE-10575 / HBASE-10570 / > HBASE-10532 / HBASE-10537 / HBASE-10534 / HBASE-6642 / HBASE-3909 / > HBASE-10169 and so on and on..., but seems it can't reproduce in local run. > This issue is created for any further tracking. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21310) Split TestCloneSnapshotFromClient
[
https://issues.apache.org/jira/browse/HBASE-21310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649424#comment-16649424
]
Hadoop QA commented on HBASE-21310:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 17 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
12s{color} | {color:red} hbase-server: The patch generated 1 new + 8 unchanged
- 1 fixed = 9 total (was 9) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}114m 38s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}156m 37s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.client.TestMobCloneSnapshotFromClientAfterSplittingRegion |
| | hadoop.hbase.client.TestMobCloneSnapshotFromClientNormal |
| | hadoop.hbase.client.TestMobCloneSnapshotFromClientError |
| | hadoop.hbase.TestCheckTestClasses |
| | hadoop.hbase.client.TestMobCloneSnapshotFromClientCloneLinksAfterDelete |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21310 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943840/HBASE-21310.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 67b4e216a696 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / dde336f6ef |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
|
[jira] [Resolved] (HBASE-9725) lost brackets for createMethodTimeMetrics?
[ https://issues.apache.org/jira/browse/HBASE-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-9725. -- Resolution: Invalid createMethodTimeMetrics not in codebase currently. > lost brackets for createMethodTimeMetrics? > -- > > Key: HBASE-9725 > URL: https://issues.apache.org/jira/browse/HBASE-9725 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Liang Xie >Assignee: Liang Xie >Priority: Major > Attachments: HBASE-9725-0.94.txt > > > to me, seems lost brackets for createMethodTimeMetrics -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-9712) TestSplitLogManager still fails on occasion
[
https://issues.apache.org/jira/browse/HBASE-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-9712.
--
Resolution: Cannot Reproduce
> TestSplitLogManager still fails on occasion
> ---
>
> Key: HBASE-9712
> URL: https://issues.apache.org/jira/browse/HBASE-9712
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
> Attachments:
> org.apache.hadoop.hbase.master.TestSplitLogManager-output (1).txt
>
>
> Opening this issue to keep account of failures. It failed for me locally
> just now.
> Failed tests:
> testTaskResigned(org.apache.hadoop.hbase.master.TestSplitLogManager):
> version1=2, version=2
> {code}
> durruti:hbase stack$ more
> hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestSplitLogManager.txt
> ---
> Test set: org.apache.hadoop.hbase.master.TestSplitLogManager
> ---
> Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 86.697 sec
> <<< FAILURE!
> testTaskResigned(org.apache.hadoop.hbase.master.TestSplitLogManager) Time
> elapsed: 0.004 sec <<< FAILURE!
> java.lang.AssertionError: version1=2, version=2
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestSplitLogManager.testTaskResigned(TestSplitLogManager.java:387)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> Let me attach the log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-9014) TestHLog.testAppendClose fails
[
https://issues.apache.org/jira/browse/HBASE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-9014.
--
Resolution: Cannot Reproduce
> TestHLog.testAppendClose fails
> --
>
> Key: HBASE-9014
> URL: https://issues.apache.org/jira/browse/HBASE-9014
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
>
> http://54.241.6.143/job/HBase-0.95/665/org.apache.hbase$hbase-server/testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLog/testAppendClose/
> {code}
> Error Message
> Problem binding to localhost/127.0.0.1:37036 : Address already in use
> Stacktrace
> java.net.BindException: Problem binding to localhost/127.0.0.1:37036 :
> Address already in use
> at org.apache.hadoop.ipc.Server.bind(Server.java:228)
> at org.apache.hadoop.ipc.Server$Listener.(Server.java:302)
> at org.apache.hadoop.ipc.Server.(Server.java:1488)
> at org.apache.hadoop.ipc.RPC$Server.(RPC.java:560)
> at org.apache.hadoop.ipc.RPC.getServer(RPC.java:521)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:302)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:536)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:278)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSClusterForTestHLog(HBaseTestingUtility.java:525)
> ...
> {code}
> This testAppendClose stops hdfs and starts it again. It looks problematic.
> Has waits of 7 seconds for the hdfs cluster to go down but in this test it
> seems like it needs even more time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-8922) TestAdmin.testForceSplitMultiFamily fails in hadoopqa
[ https://issues.apache.org/jira/browse/HBASE-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-8922. -- Resolution: Cannot Reproduce > TestAdmin.testForceSplitMultiFamily fails in hadoopqa > - > > Key: HBASE-8922 > URL: https://issues.apache.org/jira/browse/HBASE-8922 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 8922.txt > > > Started today as far as I can tell. Doesn't always happen. Looking at it, > we are splitting region not where we are supposed to -- at the 1/4 mark > rather than 1/2 mark which then throws off all calculations. Let me commit > some debug to help elicit where we are going wrong. Here is a sample: > https://builds.apache.org/job/PreCommit-HBASE-Build/6294//testReport/org.apache.hadoop.hbase.client/TestAdmin/testForceSplitMultiFamily/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-8999) TestCompactionState.testMinorCompaction goes zombie
[ https://issues.apache.org/jira/browse/HBASE-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-8999. -- Resolution: Cannot Reproduce > TestCompactionState.testMinorCompaction goes zombie > --- > > Key: HBASE-8999 > URL: https://issues.apache.org/jira/browse/HBASE-8999 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Priority: Major > > [~sershe] Is this your test boss? Mind taking a looksee. Here is the fail: > https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/624/consoleFull > Go to the end for the stack trace. > Otherwise I'll just disable for now. Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-8107) Allow non-trunk patch to be tested by Hadoop QA
[
https://issues.apache.org/jira/browse/HBASE-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-8107.
--
Resolution: Fixed
This is not an issue anymore.
> Allow non-trunk patch to be tested by Hadoop QA
> ---
>
> Key: HBASE-8107
> URL: https://issues.apache.org/jira/browse/HBASE-8107
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Major
>
> If the contributor clicks 'Submit Patch' button for the backport patch, he /
> she would get:
> -1 patch. The patch command could not apply the patch.
> I tried patch testing in 0.94 branch locally with the following command:
> {code}
> cd 94hbase
> ~/trunk/dev-support/test-patch.sh --jenkins --basedir=`pwd` HBASE-8085
> {code}
> I used test-patch.sh from trunk because it is up-to-date.
> It seemed to work mostly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-7988) Confusing log messages in assignment
[
https://issues.apache.org/jira/browse/HBASE-7988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-7988.
--
Resolution: Duplicate
Log messages were rewritten in HBASE-8696.
> Confusing log messages in assignment
>
>
> Key: HBASE-7988
> URL: https://issues.apache.org/jira/browse/HBASE-7988
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.98.0
>Reporter: Nicolas Liochon
>Priority: Minor
>
> This appears in 7840, where we kill one server with 10 regions out of two
> servers.
> I don't know why we explicitly test against 1 server here.
> {code}
> int servers = bulkPlan.size();
> if (servers == 1 || (regions < bulkAssignThresholdRegions
> && servers < bulkAssignThresholdServers)) {
> {code}
> At the end, the logs says once we're using bulk, once we're not:
> 2013-03-04 15:31:19,339 INFO
> org.apache.hadoop.hbase.master.AssignmentManager: Not use bulk assigning
> since we are assigning only 10 region(s) to 1 server(s)
> 2013-03-04 15:31:19,339 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 region(s)
> to box1,60020,1362407279158
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21312) ReplicationSource should log the incorrect peerClusterId at Error level instead of Info
[
https://issues.apache.org/jira/browse/HBASE-21312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649416#comment-16649416
]
Hadoop QA commented on HBASE-21312:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
14s{color} | {color:red} hbase-server: The patch generated 8 new + 10 unchanged
- 0 fixed = 18 total (was 10) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
38s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 53s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 4s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.regionserver.TestReplicationSource |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21312 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943839/HBASE-21312-master-001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fb30159138f5 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / dde336f6ef |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14691/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
[jira] [Resolved] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-7896. -- Resolution: Won't Fix The rename_table.rb script is no longer part of HBase. > make rename_table working in 92/94 > -- > > Key: HBASE-7896 > URL: https://issues.apache.org/jira/browse/HBASE-7896 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.92.2, 0.94.5 >Reporter: Tianying Chang >Assignee: Tianying Chang >Priority: Major > Attachments: rename_table.rb, rename_table.rb > > > The rename_table function is very useful for our customers. However, > rename_table.rb does not work for 92/94. It has several bugs. It will be > useful to fix them so that users can solve their problems. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-11829) TestHCM.testClusterStatus fails with timeout
[ https://issues.apache.org/jira/browse/HBASE-11829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-11829. --- Resolution: Cannot Reproduce Assignee: Peter Somogyi > TestHCM.testClusterStatus fails with timeout > > > Key: HBASE-11829 > URL: https://issues.apache.org/jira/browse/HBASE-11829 > Project: HBase > Issue Type: Bug > Components: test > Environment: Ubuntu 14.04 64bit > java version "1.7.0_65" > Java(TM) SE Runtime Environment (build 1.7.0_65-b17) > Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) >Reporter: Sergey Soldatov >Assignee: Peter Somogyi >Priority: Minor > > Test fails with an exception: > java.lang.Exception: Unexpected exception, > expected > but was > at > org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:28) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > Note: the failure usually reproduces on the machines where openvpn is > installed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21312) ReplicationSource should log the incorrect peerClusterId at Error level instead of Info
[
https://issues.apache.org/jira/browse/HBASE-21312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649390#comment-16649390
]
Ted Yu commented on HBASE-21312:
{code}
407 tearDownAfterClass();
{code}
Why is the method called at the end of the test ?
{code}
@AfterClass
public static void tearDownAfterClass() throws Exception {
{code}
> ReplicationSource should log the incorrect peerClusterId at Error level
> instead of Info
> ---
>
> Key: HBASE-21312
> URL: https://issues.apache.org/jira/browse/HBASE-21312
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.2, 1.4.7
>Reporter: Lingeshwaran Radhakrishnan
>Priority: Minor
> Attachments: HBASE-21312-master-001.patch
>
>
> Normally, Replication needs peer cluster ID to be different from the source.
> However, if target carries the same cluster ID as source, then during the
> ReplicationSource initialization process, following is reported in the
> RegionServer logs before terminating the ReplicationSource thread.
> {code:java}
> 2018-10-08 10:19:09,155 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Closing
> source 1 because: ClusterId 67318852-e2b7-47a7-a303-f1722cb61a06 is
> replicating to itself: peerClusterId 67318852-e2b7-47a7-a303-f1722cb61a06
> which is not allowed by
> ReplicationEndpoint:org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint{code}
> Here is the related code snippet which does this in the ReplicationSource
> class
> {code:java}
> // In rare case, zookeeper setting may be messed up. That leads to the
> incorrect
> // peerClusterId value, which is the same as the source clusterId
> if (clusterId.equals(peerClusterId) &&
> !replicationEndpoint.canReplicateToSameCluster()) {
> this.terminate("ClusterId " + clusterId + " is replicating to itself:
> peerClusterId "
> + peerClusterId + " which is not allowed by ReplicationEndpoint:"
> + replicationEndpoint.getClass().getName(), null, false);
> this.manager.removeSource(this);
> return;
> }{code}
> It would be good to have this logged at ERROR level instead of INFO for the
> following reasons
> 1. Under normal situation/case, the Peer Cluster ID would be different
> 2. It would help to easily troubleshoot issues that arises due to matching
> replication Peer cluster ID
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21309) Increase the waiting timeout for TestProcedurePriority
[
https://issues.apache.org/jira/browse/HBASE-21309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649389#comment-16649389
]
Hudson commented on HBASE-21309:
Results for branch master
[build #545 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/545/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/545//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/545//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/545//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Increase the waiting timeout for TestProcedurePriority
> --
>
> Key: HBASE-21309
> URL: https://issues.apache.org/jira/browse/HBASE-21309
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21309.patch
>
>
> On loaded jenkins nodes it will be easily timed out, so let's increase the
> timeout value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21310) Split TestCloneSnapshotFromClient
[ https://issues.apache.org/jira/browse/HBASE-21310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21310: -- Assignee: Duo Zhang Fix Version/s: 2.2.0 3.0.0 Status: Patch Available (was: Open) > Split TestCloneSnapshotFromClient > - > > Key: HBASE-21310 > URL: https://issues.apache.org/jira/browse/HBASE-21310 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21310.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21310) Split TestCloneSnapshotFromClient
[ https://issues.apache.org/jira/browse/HBASE-21310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649362#comment-16649362 ] Duo Zhang commented on HBASE-21310: --- Just lots of copy paste... > Split TestCloneSnapshotFromClient > - > > Key: HBASE-21310 > URL: https://issues.apache.org/jira/browse/HBASE-21310 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21310.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21310) Split TestCloneSnapshotFromClient
[ https://issues.apache.org/jira/browse/HBASE-21310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21310: -- Attachment: HBASE-21310.patch > Split TestCloneSnapshotFromClient > - > > Key: HBASE-21310 > URL: https://issues.apache.org/jira/browse/HBASE-21310 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21310.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649359#comment-16649359
]
Duo Zhang commented on HBASE-21278:
---
OK, good, all green.
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278-v1.patch, HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649358#comment-16649358
]
Hadoop QA commented on HBASE-21278:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 12 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} hbase-procedure: The patch generated 0 new + 19
unchanged - 1 fixed = 19 total (was 20) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} The patch passed checkstyle in hbase-server {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 54s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
53s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}115m
28s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21278 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943835/HBASE-21278-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ae7091733738 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Updated] (HBASE-21312) ReplicationSource should log the incorrect peerClusterId at Error level instead of Info
[
https://issues.apache.org/jira/browse/HBASE-21312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lingeshwaran Radhakrishnan updated HBASE-21312:
---
Attachment: (was: HBASE-21312-master-001.patch)
> ReplicationSource should log the incorrect peerClusterId at Error level
> instead of Info
> ---
>
> Key: HBASE-21312
> URL: https://issues.apache.org/jira/browse/HBASE-21312
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.2, 1.4.7
>Reporter: Lingeshwaran Radhakrishnan
>Priority: Minor
> Attachments: HBASE-21312-master-001.patch
>
>
> Normally, Replication needs peer cluster ID to be different from the source.
> However, if target carries the same cluster ID as source, then during the
> ReplicationSource initialization process, following is reported in the
> RegionServer logs before terminating the ReplicationSource thread.
> {code:java}
> 2018-10-08 10:19:09,155 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Closing
> source 1 because: ClusterId 67318852-e2b7-47a7-a303-f1722cb61a06 is
> replicating to itself: peerClusterId 67318852-e2b7-47a7-a303-f1722cb61a06
> which is not allowed by
> ReplicationEndpoint:org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint{code}
> Here is the related code snippet which does this in the ReplicationSource
> class
> {code:java}
> // In rare case, zookeeper setting may be messed up. That leads to the
> incorrect
> // peerClusterId value, which is the same as the source clusterId
> if (clusterId.equals(peerClusterId) &&
> !replicationEndpoint.canReplicateToSameCluster()) {
> this.terminate("ClusterId " + clusterId + " is replicating to itself:
> peerClusterId "
> + peerClusterId + " which is not allowed by ReplicationEndpoint:"
> + replicationEndpoint.getClass().getName(), null, false);
> this.manager.removeSource(this);
> return;
> }{code}
> It would be good to have this logged at ERROR level instead of INFO for the
> following reasons
> 1. Under normal situation/case, the Peer Cluster ID would be different
> 2. It would help to easily troubleshoot issues that arises due to matching
> replication Peer cluster ID
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21312) ReplicationSource should log the incorrect peerClusterId at Error level instead of Info
[
https://issues.apache.org/jira/browse/HBASE-21312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lingeshwaran Radhakrishnan updated HBASE-21312:
---
Attachment: HBASE-21312-master-001.patch
Status: Patch Available (was: Open)
> ReplicationSource should log the incorrect peerClusterId at Error level
> instead of Info
> ---
>
> Key: HBASE-21312
> URL: https://issues.apache.org/jira/browse/HBASE-21312
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 1.4.7, 2.0.2
>Reporter: Lingeshwaran Radhakrishnan
>Priority: Minor
> Attachments: HBASE-21312-master-001.patch
>
>
> Normally, Replication needs peer cluster ID to be different from the source.
> However, if target carries the same cluster ID as source, then during the
> ReplicationSource initialization process, following is reported in the
> RegionServer logs before terminating the ReplicationSource thread.
> {code:java}
> 2018-10-08 10:19:09,155 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Closing
> source 1 because: ClusterId 67318852-e2b7-47a7-a303-f1722cb61a06 is
> replicating to itself: peerClusterId 67318852-e2b7-47a7-a303-f1722cb61a06
> which is not allowed by
> ReplicationEndpoint:org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint{code}
> Here is the related code snippet which does this in the ReplicationSource
> class
> {code:java}
> // In rare case, zookeeper setting may be messed up. That leads to the
> incorrect
> // peerClusterId value, which is the same as the source clusterId
> if (clusterId.equals(peerClusterId) &&
> !replicationEndpoint.canReplicateToSameCluster()) {
> this.terminate("ClusterId " + clusterId + " is replicating to itself:
> peerClusterId "
> + peerClusterId + " which is not allowed by ReplicationEndpoint:"
> + replicationEndpoint.getClass().getName(), null, false);
> this.manager.removeSource(this);
> return;
> }{code}
> It would be good to have this logged at ERROR level instead of INFO for the
> following reasons
> 1. Under normal situation/case, the Peer Cluster ID would be different
> 2. It would help to easily troubleshoot issues that arises due to matching
> replication Peer cluster ID
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21312) ReplicationSource should log the incorrect peerClusterId at Error level instead of Info
[
https://issues.apache.org/jira/browse/HBASE-21312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649356#comment-16649356
]
Lingeshwaran Radhakrishnan commented on HBASE-21312:
I am interested to work on this. Attached an initial version of the patch. Let
me know if there is anything to improve the patch.
> ReplicationSource should log the incorrect peerClusterId at Error level
> instead of Info
> ---
>
> Key: HBASE-21312
> URL: https://issues.apache.org/jira/browse/HBASE-21312
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.2, 1.4.7
>Reporter: Lingeshwaran Radhakrishnan
>Priority: Minor
> Attachments: HBASE-21312-master-001.patch
>
>
> Normally, Replication needs peer cluster ID to be different from the source.
> However, if target carries the same cluster ID as source, then during the
> ReplicationSource initialization process, following is reported in the
> RegionServer logs before terminating the ReplicationSource thread.
> {code:java}
> 2018-10-08 10:19:09,155 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Closing
> source 1 because: ClusterId 67318852-e2b7-47a7-a303-f1722cb61a06 is
> replicating to itself: peerClusterId 67318852-e2b7-47a7-a303-f1722cb61a06
> which is not allowed by
> ReplicationEndpoint:org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint{code}
> Here is the related code snippet which does this in the ReplicationSource
> class
> {code:java}
> // In rare case, zookeeper setting may be messed up. That leads to the
> incorrect
> // peerClusterId value, which is the same as the source clusterId
> if (clusterId.equals(peerClusterId) &&
> !replicationEndpoint.canReplicateToSameCluster()) {
> this.terminate("ClusterId " + clusterId + " is replicating to itself:
> peerClusterId "
> + peerClusterId + " which is not allowed by ReplicationEndpoint:"
> + replicationEndpoint.getClass().getName(), null, false);
> this.manager.removeSource(this);
> return;
> }{code}
> It would be good to have this logged at ERROR level instead of INFO for the
> following reasons
> 1. Under normal situation/case, the Peer Cluster ID would be different
> 2. It would help to easily troubleshoot issues that arises due to matching
> replication Peer cluster ID
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21312) ReplicationSource should log the incorrect peerClusterId at Error level instead of Info
[
https://issues.apache.org/jira/browse/HBASE-21312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lingeshwaran Radhakrishnan updated HBASE-21312:
---
Attachment: HBASE-21312-master-001.patch
> ReplicationSource should log the incorrect peerClusterId at Error level
> instead of Info
> ---
>
> Key: HBASE-21312
> URL: https://issues.apache.org/jira/browse/HBASE-21312
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.2, 1.4.7
>Reporter: Lingeshwaran Radhakrishnan
>Priority: Minor
> Attachments: HBASE-21312-master-001.patch
>
>
> Normally, Replication needs peer cluster ID to be different from the source.
> However, if target carries the same cluster ID as source, then during the
> ReplicationSource initialization process, following is reported in the
> RegionServer logs before terminating the ReplicationSource thread.
> {code:java}
> 2018-10-08 10:19:09,155 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Closing
> source 1 because: ClusterId 67318852-e2b7-47a7-a303-f1722cb61a06 is
> replicating to itself: peerClusterId 67318852-e2b7-47a7-a303-f1722cb61a06
> which is not allowed by
> ReplicationEndpoint:org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint{code}
> Here is the related code snippet which does this in the ReplicationSource
> class
> {code:java}
> // In rare case, zookeeper setting may be messed up. That leads to the
> incorrect
> // peerClusterId value, which is the same as the source clusterId
> if (clusterId.equals(peerClusterId) &&
> !replicationEndpoint.canReplicateToSameCluster()) {
> this.terminate("ClusterId " + clusterId + " is replicating to itself:
> peerClusterId "
> + peerClusterId + " which is not allowed by ReplicationEndpoint:"
> + replicationEndpoint.getClass().getName(), null, false);
> this.manager.removeSource(this);
> return;
> }{code}
> It would be good to have this logged at ERROR level instead of INFO for the
> following reasons
> 1. Under normal situation/case, the Peer Cluster ID would be different
> 2. It would help to easily troubleshoot issues that arises due to matching
> replication Peer cluster ID
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[ https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649342#comment-16649342 ] Ted Yu commented on HBASE-13468: Option A seems to be better. > hbase.zookeeper.quorum supports ipv6 address > > > Key: HBASE-13468 > URL: https://issues.apache.org/jira/browse/HBASE-13468 > Project: HBase > Issue Type: Bug >Reporter: Mingtao Zhang >Assignee: maoling >Priority: Major > Attachments: HBASE-13468.master.001.patch, > HBASE-13468.master.002.patch > > > I put ipv6 address in hbase.zookeeper.quorum, by the time this string went to > zookeeper code, the address is messed up, i.e. only '[1234' left. > I started using pseudo mode with embedded zk = true. > I downloaded 1.0.0, not sure which affected version should be here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21312) ReplicationSource should log the incorrect peerClusterId at Error level instead of Info
Lingeshwaran Radhakrishnan created HBASE-21312:
--
Summary: ReplicationSource should log the incorrect peerClusterId
at Error level instead of Info
Key: HBASE-21312
URL: https://issues.apache.org/jira/browse/HBASE-21312
Project: HBase
Issue Type: Improvement
Components: Replication
Affects Versions: 1.4.7, 2.0.2
Reporter: Lingeshwaran Radhakrishnan
Normally, Replication needs peer cluster ID to be different from the source.
However, if target carries the same cluster ID as source, then during the
ReplicationSource initialization process, following is reported in the
RegionServer logs before terminating the ReplicationSource thread.
{code:java}
2018-10-08 10:19:09,155 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Closing
source 1 because: ClusterId 67318852-e2b7-47a7-a303-f1722cb61a06 is replicating
to itself: peerClusterId 67318852-e2b7-47a7-a303-f1722cb61a06 which is not
allowed by
ReplicationEndpoint:org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint{code}
Here is the related code snippet which does this in the ReplicationSource class
{code:java}
// In rare case, zookeeper setting may be messed up. That leads to the incorrect
// peerClusterId value, which is the same as the source clusterId
if (clusterId.equals(peerClusterId) &&
!replicationEndpoint.canReplicateToSameCluster()) {
this.terminate("ClusterId " + clusterId + " is replicating to itself:
peerClusterId "
+ peerClusterId + " which is not allowed by ReplicationEndpoint:"
+ replicationEndpoint.getClass().getName(), null, false);
this.manager.removeSource(this);
return;
}{code}
It would be good to have this logged at ERROR level instead of INFO for the
following reasons
1. Under normal situation/case, the Peer Cluster ID would be different
2. It would help to easily troubleshoot issues that arises due to matching
replication Peer cluster ID
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21278:
--
Attachment: HBASE-21278-v1.patch
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278-v1.patch, HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21278:
--
Component/s: proc-v2
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21278:
--
Priority: Critical (was: Major)
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[ https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649293#comment-16649293 ] maoling commented on HBASE-13468: - [~te...@apache.org] *can you perform more valitity check for IPv6?e.g. there should be 8 groups.* IMHO,we don't need to have strict validations with IP which will be very trivial.I will give a strict validation if you really want to.two option: A: use ready-made util directly. e.g InetAddressValidator of apache commons or something else in the Guava. B: implement IPUtil.java in the hbase-common module for hbase which can validate IPv4/IPv6 which one would you like? BTW:IPv6 like this [2001:db8::1] which don't have 8 groups > hbase.zookeeper.quorum supports ipv6 address > > > Key: HBASE-13468 > URL: https://issues.apache.org/jira/browse/HBASE-13468 > Project: HBase > Issue Type: Bug >Reporter: Mingtao Zhang >Assignee: maoling >Priority: Major > Attachments: HBASE-13468.master.001.patch, > HBASE-13468.master.002.patch > > > I put ipv6 address in hbase.zookeeper.quorum, by the time this string went to > zookeeper code, the address is messed up, i.e. only '[1234' left. > I started using pseudo mode with embedded zk = true. > I downloaded 1.0.0, not sure which affected version should be here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
