[jira] [Updated] (HBASE-20047) AuthenticationTokenIdentifier should provide a toString
[
https://issues.apache.org/jira/browse/HBASE-20047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maoling updated HBASE-20047:
Attachment: HBASE-20047.master.v0.patch
> AuthenticationTokenIdentifier should provide a toString
> ---
>
> Key: HBASE-20047
> URL: https://issues.apache.org/jira/browse/HBASE-20047
> Project: HBase
> Issue Type: Improvement
> Components: Usability
>Reporter: Sean Busbey
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20047.master.v0.patch
>
>
> It'd be easier to debug things like MapReduce and Spark jobs if our
> AuthenticationTokenIdentifier provided a toString method.
> For comparison, here's an example of a MapReduce job that has both an HDFS
> delegation token and our delegation token:
> {code:java}
> 18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HBASE_AUTH_TOKEN,
> Service: 92a63bd8-9e00-4c04-ab61-da8e606068e1, Ident:
> (org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier@17)
> 18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN,
> Service: 172.31.118.118:8020, Ident: (token for some_user:
> HDFS_DELEGATION_TOKEN owner=some_u...@example.com, renewer=yarn, realUser=,
> issueDate=1519274405003, maxDate=1519879205003, sequenceNumber=23,
> masterKeyId=9)
> {code}
> Stuff in TokenIdentifier is supposed to be public, so we should be fine to
> dump everything, similar to Hadoop's AbstractDelegationTokenIdentifier.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20047) AuthenticationTokenIdentifier should provide a toString
[
https://issues.apache.org/jira/browse/HBASE-20047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maoling updated HBASE-20047:
Status: Patch Available (was: Open)
> AuthenticationTokenIdentifier should provide a toString
> ---
>
> Key: HBASE-20047
> URL: https://issues.apache.org/jira/browse/HBASE-20047
> Project: HBase
> Issue Type: Improvement
> Components: Usability
>Reporter: Sean Busbey
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20047.master.v0.patch
>
>
> It'd be easier to debug things like MapReduce and Spark jobs if our
> AuthenticationTokenIdentifier provided a toString method.
> For comparison, here's an example of a MapReduce job that has both an HDFS
> delegation token and our delegation token:
> {code:java}
> 18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HBASE_AUTH_TOKEN,
> Service: 92a63bd8-9e00-4c04-ab61-da8e606068e1, Ident:
> (org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier@17)
> 18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN,
> Service: 172.31.118.118:8020, Ident: (token for some_user:
> HDFS_DELEGATION_TOKEN owner=some_u...@example.com, renewer=yarn, realUser=,
> issueDate=1519274405003, maxDate=1519879205003, sequenceNumber=23,
> masterKeyId=9)
> {code}
> Stuff in TokenIdentifier is supposed to be public, so we should be fine to
> dump everything, similar to Hadoop's AbstractDelegationTokenIdentifier.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20065) Revisit the timestamp usage in MetaTableAccessor
[ https://issues.apache.org/jira/browse/HBASE-20065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375367#comment-16375367 ] Duo Zhang commented on HBASE-20065: --- Must provide a timestamp when constructing a Put or Delete, and except one of the updateRegionLocation, do not pass timestamp for methods which are used to update meta. Make methods private if possible. > Revisit the timestamp usage in MetaTableAccessor > > > Key: HBASE-20065 > URL: https://issues.apache.org/jira/browse/HBASE-20065 > Project: HBase > Issue Type: Improvement >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20065.patch > > > It is totally a mess and makes me confusing when reimplementing the serial > replication feature. Let me do a clean up first. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20065) Revisit the timestamp usage in MetaTableAccessor
[ https://issues.apache.org/jira/browse/HBASE-20065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20065: -- Attachment: HBASE-20065.patch > Revisit the timestamp usage in MetaTableAccessor > > > Key: HBASE-20065 > URL: https://issues.apache.org/jira/browse/HBASE-20065 > Project: HBase > Issue Type: Improvement >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20065.patch > > > It is totally a mess and makes me confusing when reimplementing the serial > replication feature. Let me do a clean up first. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20065) Revisit the timestamp usage in MetaTableAccessor
[ https://issues.apache.org/jira/browse/HBASE-20065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20065: -- Status: Patch Available (was: Open) > Revisit the timestamp usage in MetaTableAccessor > > > Key: HBASE-20065 > URL: https://issues.apache.org/jira/browse/HBASE-20065 > Project: HBase > Issue Type: Improvement >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20065.patch > > > It is totally a mess and makes me confusing when reimplementing the serial > replication feature. Let me do a clean up first. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19761) Fix Checkstyle errors in hbase-zookeeper
[
https://issues.apache.org/jira/browse/HBASE-19761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375357#comment-16375357
]
Hadoop QA commented on HBASE-19761:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
28s{color} | {color:red} hbase-zookeeper in master has 3 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
28s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} hbase-zookeeper: The patch generated 0 new + 0
unchanged - 13 fixed = 0 total (was 13) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
13s{color} | {color:red} patch has 20 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
54s{color} | {color:red} The patch causes 20 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
25s{color} | {color:red} The patch causes 20 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
2s{color} | {color:red} The patch causes 20 errors with Hadoop v3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
52s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19761 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911713/HBASE-19761-master-v0.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile findbugs hbaseanti checkstyle |
| uname | Linux 5a5386f7762f 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bbbfeae25e |
| maven | version: Apache Maven
[jira] [Commented] (HBASE-20057) update .gitignore file to ignores *.patch file
[
https://issues.apache.org/jira/browse/HBASE-20057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375353#comment-16375353
]
Hadoop QA commented on HBASE-20057:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
53s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 4m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20057 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911740/HBASE-20057-master-v0.patch
|
| Optional Tests | asflicense |
| uname | Linux 6a5e21c7c0bf 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bbbfeae25e |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Max. process+thread count | 43 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11653/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> update .gitignore file to ignores *.patch file
> ---
>
> Key: HBASE-20057
> URL: https://issues.apache.org/jira/browse/HBASE-20057
> Project: HBase
> Issue Type: Improvement
> Components: conf
>Reporter: maoling
>Assignee: maoling
>Priority: Trivial
> Attachments: HBASE-20057-master-v0.patch
>
>
> When git format-patch a patch, a generated *.patch file will be commited with
> real code changes easily.
> Update .gitignore file which ignores *.patch will improve the git workflow
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19728) Add lock to filesCompacting in all place.
[ https://issues.apache.org/jira/browse/HBASE-19728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-19728: Fix Version/s: 1.2.7 1.3.2 > Add lock to filesCompacting in all place. > - > > Key: HBASE-19728 > URL: https://issues.apache.org/jira/browse/HBASE-19728 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: binlijin >Assignee: binlijin >Priority: Major > Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.2.7, 1.4.2 > > Attachments: HBASE-19728.branch-1.001.patch, > HBASE-19728.master.001.patch, HBASE-19728.master.002.patch, > HBASE-19728.master.002.patch > > > We find regionserver abort with the following exception: > 2017-05-09 17:40:06,369 FATAL > [regionserver/hadoop0349.et2.tbsite.net/11.251.152.199:16020-shortCompactions-1493026663275] > regionserver.HRegionServer: ABORTING region server > hadoop0349.et2.tbsite.net,16020,1493026637177: > Thread[regionserver/hadoop0349.et2.tbsite.net/11.251.152.199:16020-shortCompactions-1493026663275,5,main] > throw uncaught exception > java.lang.ArrayIndexOutOfBoundsException > at java.lang.System.arraycopy(Native Method) > at java.util.ArrayList.batchRemove(ArrayList.java:726) > at java.util.ArrayList.removeAll(ArrayList.java:690) > at > org.apache.hadoop.hbase.regionserver.HStore.finishCompactionRequest(HStore.java:1666) > at > org.apache.hadoop.hbase.regionserver.HStore.cancelRequestedCompaction(HStore.java:1656) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:504) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) > at java.lang.Thread.run(Thread.java:834) > 2017-05-08 21:15:31,979 FATAL > [regionserver/hadoop1191.et2.tbsite.net/11.251.159.40:16020-longCompactions-1494249331978] > regionserver.HRegionServer: ABORTING region server > hadoop1191.et2.tbsite.net,16020,1493196567798: > Thread[regionserver/hadoop1191.et2.tbsite.net/11.251.159.40:16020-longCompactions-1494249331978,5,main] > throw uncaught exception > java.lang.IllegalArgumentException > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:76) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.getCurrentEligibleFiles(RatioBasedCompactionPolicy.java:64) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.preSelectCompactionForCoprocessor(RatioBasedCompactionPolicy.java:72) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.preSelect(DefaultStoreEngine.java:117) > at > org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1542) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.selectCompaction(CompactSplitThread.java:362) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.access$200(CompactSplitThread.java:58) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:491) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) > at java.lang.Thread.run(Thread.java:834) > HStore#finishCompactionRequest do not require any HStore#lock's lock so > HStore.replaceStoreFiles need to synchronized on filesCompacting. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-19728) Add lock to filesCompacting in all place.
[ https://issues.apache.org/jira/browse/HBASE-19728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-19728: - reopening to pull back into branch-1.3 and branch-1.2. > Add lock to filesCompacting in all place. > - > > Key: HBASE-19728 > URL: https://issues.apache.org/jira/browse/HBASE-19728 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: binlijin >Assignee: binlijin >Priority: Major > Fix For: 2.0.0, 3.0.0, 1.5.0, 1.4.2 > > Attachments: HBASE-19728.branch-1.001.patch, > HBASE-19728.master.001.patch, HBASE-19728.master.002.patch, > HBASE-19728.master.002.patch > > > We find regionserver abort with the following exception: > 2017-05-09 17:40:06,369 FATAL > [regionserver/hadoop0349.et2.tbsite.net/11.251.152.199:16020-shortCompactions-1493026663275] > regionserver.HRegionServer: ABORTING region server > hadoop0349.et2.tbsite.net,16020,1493026637177: > Thread[regionserver/hadoop0349.et2.tbsite.net/11.251.152.199:16020-shortCompactions-1493026663275,5,main] > throw uncaught exception > java.lang.ArrayIndexOutOfBoundsException > at java.lang.System.arraycopy(Native Method) > at java.util.ArrayList.batchRemove(ArrayList.java:726) > at java.util.ArrayList.removeAll(ArrayList.java:690) > at > org.apache.hadoop.hbase.regionserver.HStore.finishCompactionRequest(HStore.java:1666) > at > org.apache.hadoop.hbase.regionserver.HStore.cancelRequestedCompaction(HStore.java:1656) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:504) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) > at java.lang.Thread.run(Thread.java:834) > 2017-05-08 21:15:31,979 FATAL > [regionserver/hadoop1191.et2.tbsite.net/11.251.159.40:16020-longCompactions-1494249331978] > regionserver.HRegionServer: ABORTING region server > hadoop1191.et2.tbsite.net,16020,1493196567798: > Thread[regionserver/hadoop1191.et2.tbsite.net/11.251.159.40:16020-longCompactions-1494249331978,5,main] > throw uncaught exception > java.lang.IllegalArgumentException > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:76) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.getCurrentEligibleFiles(RatioBasedCompactionPolicy.java:64) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.preSelectCompactionForCoprocessor(RatioBasedCompactionPolicy.java:72) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.preSelect(DefaultStoreEngine.java:117) > at > org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1542) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.selectCompaction(CompactSplitThread.java:362) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.access$200(CompactSplitThread.java:58) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:491) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) > at java.lang.Thread.run(Thread.java:834) > HStore#finishCompactionRequest do not require any HStore#lock's lock so > HStore.replaceStoreFiles need to synchronized on filesCompacting. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19835) Make explicit casting of atleast one operand to final type
[ https://issues.apache.org/jira/browse/HBASE-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375343#comment-16375343 ] Hudson commented on HBASE-19835: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4639 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4639/]) Revert "HBASE-19835 Use explicit casting to avoid side effects" (busbey: rev 216d2d464833c683b07c5d9fb0568f31819261a9) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Append.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/PrivateCellUtil.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanInfo.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionPipeline.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MultiVersionConcurrencyControl.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java > Make explicit casting of atleast one operand to final type > -- > > Key: HBASE-19835 > URL: https://issues.apache.org/jira/browse/HBASE-19835 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 3.0.0 >Reporter: Aman Poonia >Assignee: Aman Poonia >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19835.master.01.patch, HBASE-19835.master.02.patch > > > We have used > _long = int + int_ > at many places mostly wherever ClassSize.java variables are used for > calculation. > Need to cast explicitly at-least one operand to final type(i.e. type the > result is intended to be casted). > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20057) update .gitignore file to ignores *.patch file
[ https://issues.apache.org/jira/browse/HBASE-20057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling updated HBASE-20057: Status: Patch Available (was: Open) > update .gitignore file to ignores *.patch file > --- > > Key: HBASE-20057 > URL: https://issues.apache.org/jira/browse/HBASE-20057 > Project: HBase > Issue Type: Improvement > Components: conf >Reporter: maoling >Assignee: maoling >Priority: Trivial > Attachments: HBASE-20057-master-v0.patch > > > When git format-patch a patch, a generated *.patch file will be commited with > real code changes easily. > Update .gitignore file which ignores *.patch will improve the git workflow -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20019) Document the ColumnValueFilter
[ https://issues.apache.org/jira/browse/HBASE-20019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20019: -- Attachment: HBASE-20019.master.001.patch > Document the ColumnValueFilter > -- > > Key: HBASE-20019 > URL: https://issues.apache.org/jira/browse/HBASE-20019 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Reid Chan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-20019.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20019) Document the ColumnValueFilter
[ https://issues.apache.org/jira/browse/HBASE-20019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20019: -- Status: Patch Available (was: Open) v1 for review. > Document the ColumnValueFilter > -- > > Key: HBASE-20019 > URL: https://issues.apache.org/jira/browse/HBASE-20019 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Reid Chan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-20019.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20058) improper quoting in presplitting command docs
[
https://issues.apache.org/jira/browse/HBASE-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maoling updated HBASE-20058:
Status: Patch Available (was: Open)
[^HBASE-20058-master-v0.patch]
> improper quoting in presplitting command docs
> -
>
> Key: HBASE-20058
> URL: https://issues.apache.org/jira/browse/HBASE-20058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Mike Drob
>Assignee: maoling
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20058-master-v0.patch
>
>
> http://hbase.apache.org/book.html#tricks.pre-split
> {code}
> hbase>create 't1','f',SPLITS => ['10','20',30']
> {code}
> Missing a quote before the 30./
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19761) Fix Checkstyle errors in hbase-zookeeper
[ https://issues.apache.org/jira/browse/HBASE-19761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling updated HBASE-19761: Status: Patch Available (was: Reopened) [^HBASE-19761-master-v0.patch] > Fix Checkstyle errors in hbase-zookeeper > > > Key: HBASE-19761 > URL: https://issues.apache.org/jira/browse/HBASE-19761 > Project: HBase > Issue Type: Sub-task >Reporter: Jan Hentschel >Assignee: maoling >Priority: Minor > Attachments: HBASE-19761-master-v0.patch > > > Fix the remaining Checkstyle errors in the *hbase-zookeeper* module and > enable Checkstyle to fail on violations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-19761) Fix Checkstyle errors in hbase-zookeeper
[ https://issues.apache.org/jira/browse/HBASE-19761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling reopened HBASE-19761: - > Fix Checkstyle errors in hbase-zookeeper > > > Key: HBASE-19761 > URL: https://issues.apache.org/jira/browse/HBASE-19761 > Project: HBase > Issue Type: Sub-task >Reporter: Jan Hentschel >Assignee: maoling >Priority: Minor > Attachments: HBASE-19761-master-v0.patch > > > Fix the remaining Checkstyle errors in the *hbase-zookeeper* module and > enable Checkstyle to fail on violations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19761) Fix Checkstyle errors in hbase-zookeeper
[ https://issues.apache.org/jira/browse/HBASE-19761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling resolved HBASE-19761. - Resolution: Feedback Received > Fix Checkstyle errors in hbase-zookeeper > > > Key: HBASE-19761 > URL: https://issues.apache.org/jira/browse/HBASE-19761 > Project: HBase > Issue Type: Sub-task >Reporter: Jan Hentschel >Assignee: maoling >Priority: Minor > Attachments: HBASE-19761-master-v0.patch > > > Fix the remaining Checkstyle errors in the *hbase-zookeeper* module and > enable Checkstyle to fail on violations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-19761) Fix Checkstyle errors in hbase-zookeeper
[ https://issues.apache.org/jira/browse/HBASE-19761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-19761 started by maoling. --- > Fix Checkstyle errors in hbase-zookeeper > > > Key: HBASE-19761 > URL: https://issues.apache.org/jira/browse/HBASE-19761 > Project: HBase > Issue Type: Sub-task >Reporter: Jan Hentschel >Assignee: maoling >Priority: Minor > Attachments: HBASE-19761-master-v0.patch > > > Fix the remaining Checkstyle errors in the *hbase-zookeeper* module and > enable Checkstyle to fail on violations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work stopped] (HBASE-19761) Fix Checkstyle errors in hbase-zookeeper
[ https://issues.apache.org/jira/browse/HBASE-19761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-19761 stopped by maoling. --- > Fix Checkstyle errors in hbase-zookeeper > > > Key: HBASE-19761 > URL: https://issues.apache.org/jira/browse/HBASE-19761 > Project: HBase > Issue Type: Sub-task >Reporter: Jan Hentschel >Assignee: maoling >Priority: Minor > Attachments: HBASE-19761-master-v0.patch > > > Fix the remaining Checkstyle errors in the *hbase-zookeeper* module and > enable Checkstyle to fail on violations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20061) HStore synchronized member variable filesCompacting should be private
[ https://issues.apache.org/jira/browse/HBASE-20061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20061: Status: Patch Available (was: In Progress) > HStore synchronized member variable filesCompacting should be private > - > > Key: HBASE-20061 > URL: https://issues.apache.org/jira/browse/HBASE-20061 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0-alpha-1, 1.4.0, 1.3.0, 1.2.0, 3.0.0, 1.5.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.2.7, 1.4.3 > > Attachments: HBASE-20061.0.patch > > > follow up to HBASE-19728 (which is in an RC for 1.4.2 currently, so no > addendums): > the filesCompacting member is currently package private, which allows folks > outside of HStore to modify it without ensuring we have hte needed locking in > place. it looks like we don't actually use it anywhere outside of HStore, so > we should just make it private. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20061) HStore synchronized member variable filesCompacting should be private
[ https://issues.apache.org/jira/browse/HBASE-20061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20061: Attachment: HBASE-20061.0.patch > HStore synchronized member variable filesCompacting should be private > - > > Key: HBASE-20061 > URL: https://issues.apache.org/jira/browse/HBASE-20061 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 1.2.0, 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0-alpha-1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.2.7, 1.4.3 > > Attachments: HBASE-20061.0.patch > > > follow up to HBASE-19728 (which is in an RC for 1.4.2 currently, so no > addendums): > the filesCompacting member is currently package private, which allows folks > outside of HStore to modify it without ensuring we have hte needed locking in > place. it looks like we don't actually use it anywhere outside of HStore, so > we should just make it private. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20043:
--
Attachment: HBASE-20043.branch-2.006.patch
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch, HBASE-20043.branch-2.003.patch,
> HBASE-20043.branch-2.004.patch, HBASE-20043.branch-2.005.patch,
> HBASE-20043.branch-2.006.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19835) Make explicit casting of atleast one operand to final type
[ https://issues.apache.org/jira/browse/HBASE-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375308#comment-16375308 ] Sean Busbey commented on HBASE-19835: - commit has now been reverted from branch-2 as well > Make explicit casting of atleast one operand to final type > -- > > Key: HBASE-19835 > URL: https://issues.apache.org/jira/browse/HBASE-19835 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 3.0.0 >Reporter: Aman Poonia >Assignee: Aman Poonia >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19835.master.01.patch, HBASE-19835.master.02.patch > > > We have used > _long = int + int_ > at many places mostly wherever ClassSize.java variables are used for > calculation. > Need to cast explicitly at-least one operand to final type(i.e. type the > result is intended to be casted). > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20047) AuthenticationTokenIdentifier should provide a toString
[
https://issues.apache.org/jira/browse/HBASE-20047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maoling updated HBASE-20047:
Description:
It'd be easier to debug things like MapReduce and Spark jobs if our
AuthenticationTokenIdentifier provided a toString method.
For comparison, here's an example of a MapReduce job that has both an HDFS
delegation token and our delegation token:
{code:java}
18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HBASE_AUTH_TOKEN, Service:
92a63bd8-9e00-4c04-ab61-da8e606068e1, Ident:
(org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier@17)
18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN,
Service: 172.31.118.118:8020, Ident: (token for some_user:
HDFS_DELEGATION_TOKEN owner=some_u...@example.com, renewer=yarn, realUser=,
issueDate=1519274405003, maxDate=1519879205003, sequenceNumber=23,
masterKeyId=9)
{code}
Stuff in TokenIdentifier is supposed to be public, so we should be fine to dump
everything, similar to Hadoop's AbstractDelegationTokenIdentifier.
was:
It'd be easier to debug things like MapReduce and Spark jobs if our
AuthenticationTokenIdentifier provided a toString method.
For comparison, here's an example of a MapReduce job that has both an HDFS
delegation token and our delegation token:
{code}
18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HBASE_AUTH_TOKEN, Service:
92a63bd8-9e00-4c04-ab61-da8e606068e1, Ident:
(org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier@17)
18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN,
Service: 172.31.118.118:8020, Ident: (token for some_user:
HDFS_DELEGATION_TOKEN owner=some_u...@example.com, renewer=yarn, realUser=,
issueDate=1519274405003, maxDate=1519879205003, sequenceNumber=23,
masterKeyId=9)
{code}
Stuff in TokenIdentifier is supposed to be public, so we should be fine to dump
everything, similar to Hadoop's AbstractDelegationTokenIdentifier.
> AuthenticationTokenIdentifier should provide a toString
> ---
>
> Key: HBASE-20047
> URL: https://issues.apache.org/jira/browse/HBASE-20047
> Project: HBase
> Issue Type: Improvement
> Components: Usability
>Reporter: Sean Busbey
>Priority: Minor
> Labels: beginner
>
> It'd be easier to debug things like MapReduce and Spark jobs if our
> AuthenticationTokenIdentifier provided a toString method.
> For comparison, here's an example of a MapReduce job that has both an HDFS
> delegation token and our delegation token:
> {code:java}
> 18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HBASE_AUTH_TOKEN,
> Service: 92a63bd8-9e00-4c04-ab61-da8e606068e1, Ident:
> (org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier@17)
> 18/02/21 20:40:06 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN,
> Service: 172.31.118.118:8020, Ident: (token for some_user:
> HDFS_DELEGATION_TOKEN owner=some_u...@example.com, renewer=yarn, realUser=,
> issueDate=1519274405003, maxDate=1519879205003, sequenceNumber=23,
> masterKeyId=9)
> {code}
> Stuff in TokenIdentifier is supposed to be public, so we should be fine to
> dump everything, similar to Hadoop's AbstractDelegationTokenIdentifier.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375296#comment-16375296
]
Hadoop QA commented on HBASE-20043:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
1s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
57s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
4s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 12m
1s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
47s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
59s{color} | {color:red} hbase-server: The patch generated 1 new + 596
unchanged - 4 fixed = 597 total (was 600) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} hbase-it: The patch generated 1 new + 8 unchanged - 0
fixed = 9 total (was 8) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
55s{color} | {color:red} root: The patch generated 2 new + 610 unchanged - 4
fixed = 612 total (was 614) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
25s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 56s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}207m 4s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
54s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}266m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | hadoop.hbase.thrift.TestThriftHttpServer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-20043 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911834/HBASE-20043.branch-2.005.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
[jira] [Commented] (HBASE-18864) NullPointerException thrown when adding rows to a table from peer cluster, table with replication factor other than 0 or 1
[ https://issues.apache.org/jira/browse/HBASE-18864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375292#comment-16375292 ] Sakthi commented on HBASE-18864: Hey, [~stack]. Please share your thoughts on this. I am looking into writing test cases. The thought process while setting scope as '1' was that when a user puts a non-zero value, then probably the user wanted the replication to be a part? Hence the '1'. But seeing it the other way, shutting down the replication by setting the scope to '0' would be a safer option as it would help in cases where '5' was a mere typo or such. To tackle the problem of flooding log files when a user puts in a large number of writes with bad scope, should the log messages be put to "LOG.debug" instead, or checking for redundant log message before putting in another "LOG.warn" would be a better option? Thanks. > NullPointerException thrown when adding rows to a table from peer cluster, > table with replication factor other than 0 or 1 > -- > > Key: HBASE-18864 > URL: https://issues.apache.org/jira/browse/HBASE-18864 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.3.0 >Reporter: smita >Assignee: Sakthi >Priority: Major > Labels: beginner > Attachments: hbase-18864.branch-1.2.001.patch > > > Scenario: > = > add_peer > create a table > alter table with REPLICATION_SCOPE => '5' > enable table replication > login to peer cluster and try putting data to the table -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20058) improper quoting in presplitting command docs
[
https://issues.apache.org/jira/browse/HBASE-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maoling reassigned HBASE-20058:
---
Assignee: maoling
> improper quoting in presplitting command docs
> -
>
> Key: HBASE-20058
> URL: https://issues.apache.org/jira/browse/HBASE-20058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Mike Drob
>Assignee: maoling
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20058-master-v0.patch
>
>
> http://hbase.apache.org/book.html#tricks.pre-split
> {code}
> hbase>create 't1','f',SPLITS => ['10','20',30']
> {code}
> Missing a quote before the 30./
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20058) improper quoting in presplitting command docs
[
https://issues.apache.org/jira/browse/HBASE-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maoling updated HBASE-20058:
Attachment: HBASE-20058-master-v0.patch
> improper quoting in presplitting command docs
> -
>
> Key: HBASE-20058
> URL: https://issues.apache.org/jira/browse/HBASE-20058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Mike Drob
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20058-master-v0.patch
>
>
> http://hbase.apache.org/book.html#tricks.pre-split
> {code}
> hbase>create 't1','f',SPLITS => ['10','20',30']
> {code}
> Missing a quote before the 30./
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375290#comment-16375290
]
Chia-Ping Tsai commented on HBASE-19863:
+1
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: HBASE-19863-branch-2.patch, HBASE-19863-branch1.patch,
> HBASE-19863-test.patch, HBASE-19863.v2-branch-2.patch,
> HBASE-19863.v3-branch-2.patch, HBASE-19863.v4-branch-2.patch,
> HBASE-19863.v4-master.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The query matcher will then just skip this fake
> // key/value and the store scanner will progress to the next column. This
> // is obviously not a "real real" seek, but unlike the fake KV earlier in
> // this method, we want this to be propagated to ScanQueryMatcher.
> {noformat}
>
> For store B we would set it to fake 10:0/C3/createFirstOnRowColTS()/Maximum
> to skip C3 entirely.
> After that we start searching for qualifier C5 using seekOrSkipToNextColumn
> which run first trySkipToNextColumn:
> {noformat}
> protected boolean trySkipToNextColumn(Cell cell) throws IOException {
> Cell nextCell = null;
> do {
> Cell
[jira] [Commented] (HBASE-20001) cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
[
https://issues.apache.org/jira/browse/HBASE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375285#comment-16375285
]
Chia-Ping Tsai commented on HBASE-20001:
{quote} We rollback during split and merge failures in zk assignment. So, we
online the parents and cleanup the failed splits/merges. This is demonstrated
by the unit tests - testMergeIsRolledBackOnMERGEFailure and
testSplitIsRolledBackOnSPLITFailure. I also explained this in one of the
comments above, lemme know what you think.
{quote}
Got it. Thanks for the explanation. Both of tests look good to me. +1 if the
check of passed server is necessary.
> cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
> -
>
> Key: HBASE-20001
> URL: https://issues.apache.org/jira/browse/HBASE-20001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.1.7
>Reporter: Francis Liu
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.5.0, 1.2.7, 1.4.3
>
> Attachments: HBASE-20001.branch-1.4.001.patch,
> HBASE-20001.branch-1.4.002.patch, HBASE-20001.branch-1.4.003.patch,
> HBASE-20001.branch-1.4.004.patch, HBASE-20001.branch-1.4.005.patch
>
>
> In RegionStates.cleanIfNoMetaEntry()
> {{if (MetaTableAccessor.getRegion(server.getConnection(),
> hri.getEncodedNameAsBytes()) == null) {}}
> {{regionOffline(hri);}}
> {{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
> }
> But api expects regionname
> {{public static Pair getRegion(Connection
> connection, byte [] regionName)}}
> So we might end up cleaning good regions.
>
> ADDENDUM:
> The scenario mentioned occurs when zkless assignment is used. With zk-based
> assignment without the patch what could occur is the daughter regions are
> offlined and have no hdfs directory but have entries in meta. The daughter
> meta entries will prolly be picked up by the client causing NSREs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20055) Remove declaration of un-thrown exceptions and unused setRegionStateBackToOpen() from MergeTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375281#comment-16375281 ] Umesh Agashe commented on HBASE-20055: -- The test is clearly failing because of the changes and its to do with InterruptedException. Looking... > Remove declaration of un-thrown exceptions and unused > setRegionStateBackToOpen() from MergeTableRegionsProcedure > > > Key: HBASE-20055 > URL: https://issues.apache.org/jira/browse/HBASE-20055 > Project: HBase > Issue Type: Improvement > Components: amv2 >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: hbase-20055.master.001.patch, > hbase-20055.master.001.patch > > > A few methods declare exceptions in throws statement that are not thrown and > method setRegionStateBackToOpen() is not used in MergeTableRegionsProcedure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375278#comment-16375278
]
Ted Yu commented on HBASE-19863:
[~chia7712] [~ram_krish]:
Mind taking another look ?
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: HBASE-19863-branch-2.patch, HBASE-19863-branch1.patch,
> HBASE-19863-test.patch, HBASE-19863.v2-branch-2.patch,
> HBASE-19863.v3-branch-2.patch, HBASE-19863.v4-branch-2.patch,
> HBASE-19863.v4-master.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The query matcher will then just skip this fake
> // key/value and the store scanner will progress to the next column. This
> // is obviously not a "real real" seek, but unlike the fake KV earlier in
> // this method, we want this to be propagated to ScanQueryMatcher.
> {noformat}
>
> For store B we would set it to fake 10:0/C3/createFirstOnRowColTS()/Maximum
> to skip C3 entirely.
> After that we start searching for qualifier C5 using seekOrSkipToNextColumn
> which run first trySkipToNextColumn:
> {noformat}
> protected boolean trySkipToNextColumn(Cell cell) throws IOException {
> Cell nextCell =
[jira] [Updated] (HBASE-19157) IntegrationTestBackupRestore should warn about missing config
[
https://issues.apache.org/jira/browse/HBASE-19157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19157:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Artem.
> IntegrationTestBackupRestore should warn about missing config
> -
>
> Key: HBASE-19157
> URL: https://issues.apache.org/jira/browse/HBASE-19157
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: backup, integration-test
> Fix For: 3.0.0
>
> Attachments: HBASE-19157.v01.patch, HBASE-19157.v02.patch,
> HBASE-19157.v03.patch, HBASE-19157.v05.patch
>
>
> I was running IntegrationTestBackupRestore on a newly created cluster running
> hbase-2.
> It failed with:
> {code}
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException):
> org.apache.hadoop.hbase.DoNotRetryIOException: The procedure is not
> registered: rolllog-proc
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.execProcedure(MasterRpcServices.java:817)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:134)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:325)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:305)
> {code}
> {{hbase backup}} command warns about missing config up-front.
> IntegrationTestBackupRestore should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19997) [rolling upgrade] 1.x => 2.x
[ https://issues.apache.org/jira/browse/HBASE-19997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375272#comment-16375272 ] stack commented on HBASE-19997: --- >From description, item #1 maybe ok --- hbase1 can read hbase2 WALs HBASE-19166 >-- and item #2 should be ok too. Trying a rolling upgrade. Started a 0.98 cluster. Loaded it w/ data. Stopped 0.98 master. Started branch-2 master. It comes up, the extant 0.98 regionservers register. Then we get stuck on figuring hbase:meta. RecoverMetaProcedure seems to skip reading zk location of hbase:meta and tries to assign but fails w/ the below 2018-02-23 16:49:36,252 WARN [RSProcedureDispatcher-pool3-t2] procedure.RSProcedureDispatcher: Failed dispatch to server=ve0542.halxg.cloudera.com,60020,1519276337941 try=0 java.io.IOException: java.io.IOException at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2247) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:3991) at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:20408) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) ... 4 more at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:93) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:83) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:345) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:322) at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$OpenRegionRemoteCall.sendRequest(RSProcedureDispatcher.java:389) at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$OpenRegionRemoteCall.call(RSProcedureDispatcher.java:371) at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$OpenRegionRemoteCall.call(RSProcedureDispatcher.java:356) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Need to figure why it can't get server from hbase:meta > [rolling upgrade] 1.x => 2.x > > > Key: HBASE-19997 > URL: https://issues.apache.org/jira/browse/HBASE-19997 > Project: HBase > Issue Type: Umbrella >Reporter: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > > An umbrella issue of issues needed so folks can do a rolling upgrade from > hbase-1.x to hbase-2.x. > (Recent) Notables: > * hbase-1.x can't read hbase-2.x WALs -- hbase-1.x doesn't know the > AsyncProtobufLogWriter class used writing the WAL -- see > https://issues.apache.org/jira/browse/HBASE-19166?focusedCommentId=16362897=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16362897 > for exception. > ** Might be ok... means WAL split fails on an hbase1 RS... must wait till an > hbase-2.x RS picks up the WAL for it to be split. > * hbase-1 can't open regions from tables created by hbase-2; it can't find > the Table descriptor. See > https://issues.apache.org/jira/browse/HBASE-19116?focusedCommentId=16363276=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16363276 > ** This might be ok if the tables we are doing rolling upgrade over were > written with hbase-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20054: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the review, Ben > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.branch-2.v2.txt, 20054.v1.txt, 20054.v2.txt, > 20054.v3.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the new test to master branch. > After discussion, the retry from #1 above is going to be dropped - we abort > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19157) IntegrationTestBackupRestore should warn about missing config
[
https://issues.apache.org/jira/browse/HBASE-19157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-19157:
-
Attachment: HBASE-19157.v05.patch
> IntegrationTestBackupRestore should warn about missing config
> -
>
> Key: HBASE-19157
> URL: https://issues.apache.org/jira/browse/HBASE-19157
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: backup, integration-test
> Attachments: HBASE-19157.v01.patch, HBASE-19157.v02.patch,
> HBASE-19157.v03.patch, HBASE-19157.v05.patch
>
>
> I was running IntegrationTestBackupRestore on a newly created cluster running
> hbase-2.
> It failed with:
> {code}
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException):
> org.apache.hadoop.hbase.DoNotRetryIOException: The procedure is not
> registered: rolllog-proc
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.execProcedure(MasterRpcServices.java:817)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:134)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:325)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:305)
> {code}
> {{hbase backup}} command warns about missing config up-front.
> IntegrationTestBackupRestore should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[
https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375268#comment-16375268
]
Hadoop QA commented on HBASE-20054:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
47s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
44s{color} | {color:red} hbase-server in branch-2 has 22 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
30s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}113m
51s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}151m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-20054 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911847/20054.branch-2.v2.txt
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8111e6ff2c82 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 4ddfdaffdc |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
|
[jira] [Updated] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20054: --- Attachment: 20054.v3.txt > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.branch-2.v2.txt, 20054.v1.txt, 20054.v2.txt, > 20054.v3.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the new test to master branch. > After discussion, the retry from #1 above is going to be dropped - we abort > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20065) Revisit the timestamp usage in MetaTableAccessor
Duo Zhang created HBASE-20065: - Summary: Revisit the timestamp usage in MetaTableAccessor Key: HBASE-20065 URL: https://issues.apache.org/jira/browse/HBASE-20065 Project: HBase Issue Type: Improvement Reporter: Duo Zhang Assignee: Duo Zhang It is totally a mess and makes me confusing when reimplementing the serial replication feature. Let me do a clean up first. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[
https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375264#comment-16375264
]
Hadoop QA commented on HBASE-20054:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
47s{color} | {color:red} hbase-server in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
56s{color} | {color:red} hbase-server: The patch generated 3 new + 1 unchanged
- 0 fixed = 4 total (was 1) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}106m
28s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}144m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20054 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911845/20054.v2.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b6c90ce6cc0a 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ad5cd50dfc |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| findbugs |
[jira] [Updated] (HBASE-20001) cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
[
https://issues.apache.org/jira/browse/HBASE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-20001:
Description:
In RegionStates.cleanIfNoMetaEntry()
{{if (MetaTableAccessor.getRegion(server.getConnection(),
hri.getEncodedNameAsBytes()) == null) {}}
{{regionOffline(hri);}}
{{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
}
But api expects regionname
{{public static Pair getRegion(Connection connection,
byte [] regionName)}}
So we might end up cleaning good regions.
ADDENDUM:
The scenario mentioned occurs when zkless assignment is used. With zk-based
assignment without the patch what could occur is the daughter regions are
offlined and have no hdfs directory but have entries in meta. The daughter meta
entries will prolly be picked up by the client causing NSREs.
was:
In RegionStates.cleanIfNoMetaEntry()
{{if (MetaTableAccessor.getRegion(server.getConnection(),
hri.getEncodedNameAsBytes()) == null) {}}
{{regionOffline(hri);}}
{{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
}
But api expects regionname
{{public static Pair getRegion(Connection connection,
byte [] regionName)}}
So we might end up cleaning good regions.
ADDENDUM:
The outcome
> cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
> -
>
> Key: HBASE-20001
> URL: https://issues.apache.org/jira/browse/HBASE-20001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.1.7
>Reporter: Francis Liu
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.5.0, 1.2.7, 1.4.3
>
> Attachments: HBASE-20001.branch-1.4.001.patch,
> HBASE-20001.branch-1.4.002.patch, HBASE-20001.branch-1.4.003.patch,
> HBASE-20001.branch-1.4.004.patch, HBASE-20001.branch-1.4.005.patch
>
>
> In RegionStates.cleanIfNoMetaEntry()
> {{if (MetaTableAccessor.getRegion(server.getConnection(),
> hri.getEncodedNameAsBytes()) == null) {}}
> {{regionOffline(hri);}}
> {{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
> }
> But api expects regionname
> {{public static Pair getRegion(Connection
> connection, byte [] regionName)}}
> So we might end up cleaning good regions.
>
> ADDENDUM:
> The scenario mentioned occurs when zkless assignment is used. With zk-based
> assignment without the patch what could occur is the daughter regions are
> offlined and have no hdfs directory but have entries in meta. The daughter
> meta entries will prolly be picked up by the client causing NSREs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19400) Add missing security checks in MasterRpcServices
[
https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375246#comment-16375246
]
Hadoop QA commented on HBASE-19400:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
10s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
45s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
23s{color} | {color:red} hbase-server: The patch generated 1 new + 299
unchanged - 2 fixed = 300 total (was 301) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} hbase-rsgroup: The patch generated 1 new + 1 unchanged
- 0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
37s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
30s{color} | {color:red} The patch causes 44 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
23s{color} | {color:red} The patch causes 44 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}116m 20s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} |
[jira] [Commented] (HBASE-19987) update error-prone to 2.2.0
[ https://issues.apache.org/jira/browse/HBASE-19987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375239#comment-16375239 ] stack commented on HBASE-19987: --- +1 for branch-2. > update error-prone to 2.2.0 > --- > > Key: HBASE-19987 > URL: https://issues.apache.org/jira/browse/HBASE-19987 > Project: HBase > Issue Type: Bug >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-19987.patch > > > keep ourselves healthy and up to date -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20052) TestRegionOpen#testNonExistentRegionReplica fails due to NPE
[
https://issues.apache.org/jira/browse/HBASE-20052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375238#comment-16375238
]
Ted Yu commented on HBASE-20052:
HRegionFileSystem#createRegionOnFileSystem() is used in several tests, not just
in hbase2+ :
{code}
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifestV1.java:
HRegionFileSystem snapshotRegionFs =
HRegionFileSystem.createRegionOnFileSystem(conf,
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionFileSystem.java:
HRegionFileSystem regionFs =
HRegionFileSystem.createRegionOnFileSystem(conf, fs
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionFileSystem.java:
HRegionFileSystem regionFs =
HRegionFileSystem.createRegionOnFileSystem(conf, fs
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionFileSystem.java:
// HRegionFileSystem.createRegionOnFileSystem(conf, new
MockFileSystemForCreate(
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionFileSystem.java:
HRegionFileSystem regionFs =
HRegionFileSystem.createRegionOnFileSystem(conf, fs
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java:
HRegionFileSystem regionFs =
HRegionFileSystem.createRegionOnFileSystem(conf, fs,
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java:
HRegionFileSystem regionFs = HRegionFileSystem.createRegionOnFileSystem(
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java:
HRegionFileSystem regionFs = HRegionFileSystem.createRegionOnFileSystem(
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java:
HRegionFileSystem regionFs = HRegionFileSystem.createRegionOnFileSystem(
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java:
HRegionFileSystem regionFs = HRegionFileSystem.createRegionOnFileSystem(
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java:
HRegionFileSystem cloneRegionFs =
HRegionFileSystem.createRegionOnFileSystem(
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java:
HRegionFileSystem rfs =
HRegionFileSystem.createRegionOnFileSystem(conf, fs, tabl
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java:
rfs = HRegionFileSystem.createRegionOnFileSystem(conf, fs, tableDir,
hri);
{code}
The above is from branch-1.
If return type for the method is to be changed, another JIRA should be opened.
> TestRegionOpen#testNonExistentRegionReplica fails due to NPE
>
>
> Key: HBASE-20052
> URL: https://issues.apache.org/jira/browse/HBASE-20052
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.3
>
> Attachments: 20052.v1.txt, 20052.v2.txt
>
>
> After HBASE-19391 was integrated, the following test failure can be observed:
> {code}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.TestRegionOpen.testNonExistentRegionReplica(TestRegionOpen.java:122)
> {code}
> This was due null being returned from
> HRegionFileSystem#createRegionOnFileSystem().
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-20052) TestRegionOpen#testNonExistentRegionReplica fails due to NPE
[
https://issues.apache.org/jira/browse/HBASE-20052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reopened HBASE-20052:
---
[~yuzhih...@gmail.com] See @appy comment. Reopening for attention.
> TestRegionOpen#testNonExistentRegionReplica fails due to NPE
>
>
> Key: HBASE-20052
> URL: https://issues.apache.org/jira/browse/HBASE-20052
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.3
>
> Attachments: 20052.v1.txt, 20052.v2.txt
>
>
> After HBASE-19391 was integrated, the following test failure can be observed:
> {code}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.TestRegionOpen.testNonExistentRegionReplica(TestRegionOpen.java:122)
> {code}
> This was due null being returned from
> HRegionFileSystem#createRegionOnFileSystem().
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20060) Add details of off heap memstore into book.
[ https://issues.apache.org/jira/browse/HBASE-20060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375229#comment-16375229 ] stack commented on HBASE-20060: --- This will help w/ adoption of offheaping. Just needs to be how to turn it on and what to look for when it is on... what to monitor. > Add details of off heap memstore into book. > --- > > Key: HBASE-20060 > URL: https://issues.apache.org/jira/browse/HBASE-20060 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20060) Add details of off heap memstore into book.
[ https://issues.apache.org/jira/browse/HBASE-20060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20060: -- Priority: Critical (was: Major) > Add details of off heap memstore into book. > --- > > Key: HBASE-20060 > URL: https://issues.apache.org/jira/browse/HBASE-20060 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20001) cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
[
https://issues.apache.org/jira/browse/HBASE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-20001:
Description:
In RegionStates.cleanIfNoMetaEntry()
{{if (MetaTableAccessor.getRegion(server.getConnection(),
hri.getEncodedNameAsBytes()) == null) {}}
{{regionOffline(hri);}}
{{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
}
But api expects regionname
{{public static Pair getRegion(Connection connection,
byte [] regionName)}}
So we might end up cleaning good regions.
ADDENDUM:
The outcome
was:
In RegionStates.cleanIfNoMetaEntry()
{{if (MetaTableAccessor.getRegion(server.getConnection(),
hri.getEncodedNameAsBytes()) == null) {}}
{{regionOffline(hri);}}
{{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
}
But api expects regionname
{{public static Pair getRegion(Connection connection,
byte [] regionName)}}
So we might end up cleaning good regions.
> cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
> -
>
> Key: HBASE-20001
> URL: https://issues.apache.org/jira/browse/HBASE-20001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.1.7
>Reporter: Francis Liu
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.5.0, 1.2.7, 1.4.3
>
> Attachments: HBASE-20001.branch-1.4.001.patch,
> HBASE-20001.branch-1.4.002.patch, HBASE-20001.branch-1.4.003.patch,
> HBASE-20001.branch-1.4.004.patch, HBASE-20001.branch-1.4.005.patch
>
>
> In RegionStates.cleanIfNoMetaEntry()
> {{if (MetaTableAccessor.getRegion(server.getConnection(),
> hri.getEncodedNameAsBytes()) == null) {}}
> {{regionOffline(hri);}}
> {{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
> }
> But api expects regionname
> {{public static Pair getRegion(Connection
> connection, byte [] regionName)}}
> So we might end up cleaning good regions.
>
> ADDENDUM:
> The outcome
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-19835) Make explicit casting of atleast one operand to final type
[ https://issues.apache.org/jira/browse/HBASE-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-19835: - reverted from master because the rebase for v02 reintroduced the concurrency bug fixed by HBASE-19728. Looks like it also undid some findbugs fixes done in HBASE-19811. Please only do the change intended in v1 without undoing changes that happened on master between that patch and current. (revert from branch-2 later tonight when I finish working through tests on it.) > Make explicit casting of atleast one operand to final type > -- > > Key: HBASE-19835 > URL: https://issues.apache.org/jira/browse/HBASE-19835 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 3.0.0 >Reporter: Aman Poonia >Assignee: Aman Poonia >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19835.master.01.patch, HBASE-19835.master.02.patch > > > We have used > _long = int + int_ > at many places mostly wherever ClassSize.java variables are used for > calculation. > Need to cast explicitly at-least one operand to final type(i.e. type the > result is intended to be casted). > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375186#comment-16375186
]
Hadoop QA commented on HBASE-19863:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
21s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} hbase-server: The patch generated 0 new + 283
unchanged - 6 fixed = 283 total (was 289) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 35s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}113m
14s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}144m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19863 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911819/HBASE-19863.v4-branch-2.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 87ae32597196 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 4ddfdaffdc |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11641/testReport/ |
| Max. process+thread count | 4851 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11641/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
[jira] [Updated] (HBASE-19553) Old replica regions should be cleared from AM memory after primary region split or merge
[ https://issues.apache.org/jira/browse/HBASE-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19553: --- Fix Version/s: 1.4.3 > Old replica regions should be cleared from AM memory after primary region > split or merge > > > Key: HBASE-19553 > URL: https://issues.apache.org/jira/browse/HBASE-19553 > Project: HBase > Issue Type: Bug > Components: master >Reporter: huaxiang sun >Assignee: Pankaj Kumar >Priority: Minor > Fix For: 1.5.0, 1.4.3 > > Attachments: HBASE-19553-branch-1-v2.patch, > HBASE-19553-branch-1-v3.patch, HBASE-19553-branch-1-v4.patch, > HBASE-19553-branch-1-v4.patch, HBASE-19553-branch-1.patch > > > Similar to HBASE-18025, the replica parent's info is not removed from master. > Actually I think it can be removed after replica region is split or merged, I > will check the logic and apply one patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19974) Decommissioned servers cannot be removed by remove_servers_rsgroup methods
[ https://issues.apache.org/jira/browse/HBASE-19974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375183#comment-16375183 ] Xiaolin Ha commented on HBASE-19974: [~hadoopqa] timeout is chance, please run once more. > Decommissioned servers cannot be removed by remove_servers_rsgroup methods > -- > > Key: HBASE-19974 > URL: https://issues.apache.org/jira/browse/HBASE-19974 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0-beta-2 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: HBASE-19974.branch-2.001.patch, > HBASE-19974.branch-2.002.patch, HBASE-19974.branch-2.003.patch, > HBASE-19974.branch-2.004.patch > > > When remove servers from a rsgroup, it will check the server is not online or > dead. But when we decommision a server, the server will be both in online > list and drainning list. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19391) Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory
[ https://issues.apache.org/jira/browse/HBASE-19391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19391: --- Fix Version/s: 1.4.3 > Calling HRegion#initializeRegionInternals from a region replica can still > re-create a region directory > -- > > Key: HBASE-19391 > URL: https://issues.apache.org/jira/browse/HBASE-19391 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-4, 1.4.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez >Priority: Major > Fix For: 1.5.0, 2.0.0-beta-2, 1.4.3 > > Attachments: > 0001-HBASE-19391-Calling-HRegion-initializeRegionInternal.patch, > HBASE-19391.master.v0.patch > > > This is a follow up from HBASE-18024. There stills a chance that attempting > to open a region that is not the default region replica can still create a > GC'd region directory by the CatalogJanitor causing inconsistencies with hbck. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19364) Truncate_preserve fails with table when replica region > 1
[ https://issues.apache.org/jira/browse/HBASE-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19364: --- Fix Version/s: 1.4.3 > Truncate_preserve fails with table when replica region > 1 > -- > > Key: HBASE-19364 > URL: https://issues.apache.org/jira/browse/HBASE-19364 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Major > Fix For: 1.5.0, 1.4.3 > > Attachments: HBASE-19364-branch-1-v2.patch, > HBASE-19364-branch-1-v3.patch, HBASE-19364-branch-1.patch, > HBASE-19364-branch-1.patch > > > Root cause is same as HBASE-17319, here we need to exclude secondary regions > while reading meta. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20052) TestRegionOpen#testNonExistentRegionReplica fails due to NPE
[
https://issues.apache.org/jira/browse/HBASE-20052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20052:
---
Fix Version/s: 1.4.3
> TestRegionOpen#testNonExistentRegionReplica fails due to NPE
>
>
> Key: HBASE-20052
> URL: https://issues.apache.org/jira/browse/HBASE-20052
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.3
>
> Attachments: 20052.v1.txt, 20052.v2.txt
>
>
> After HBASE-19391 was integrated, the following test failure can be observed:
> {code}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.TestRegionOpen.testNonExistentRegionReplica(TestRegionOpen.java:122)
> {code}
> This was due null being returned from
> HRegionFileSystem#createRegionOnFileSystem().
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20052) TestRegionOpen#testNonExistentRegionReplica fails due to NPE
[
https://issues.apache.org/jira/browse/HBASE-20052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375180#comment-16375180
]
Appy commented on HBASE-20052:
--
Don't have any context here, so my question might not be right, but here goes..
bq. HRegionFileSystem#createRegionOnFileSystem() should always return the
instance of HRegionFileSystem (even if .regioninfo file is not created)
Why so?
I don't see it's return value being used anywhere except here:
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifestV1.java#L80,
and i think that snapshot may have nothing to do with replicas? If so,
please revert the change in prod code and rather fix the test.
> TestRegionOpen#testNonExistentRegionReplica fails due to NPE
>
>
> Key: HBASE-20052
> URL: https://issues.apache.org/jira/browse/HBASE-20052
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.3
>
> Attachments: 20052.v1.txt, 20052.v2.txt
>
>
> After HBASE-19391 was integrated, the following test failure can be observed:
> {code}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.TestRegionOpen.testNonExistentRegionReplica(TestRegionOpen.java:122)
> {code}
> This was due null being returned from
> HRegionFileSystem#createRegionOnFileSystem().
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19157) IntegrationTestBackupRestore should warn about missing config
[
https://issues.apache.org/jira/browse/HBASE-19157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375170#comment-16375170
]
Hadoop QA commented on HBASE-19157:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} hbase-it: The patch generated 0 new + 2 unchanged -
1 fixed = 2 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 27s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
43s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19157 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911843/HBASE-19157.v03.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux cc5d9c501567 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ad5cd50dfc |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11644/testReport/ |
| Max. process+thread count | 397 (vs. ulimit of 1) |
| modules | C: hbase-it U: hbase-it |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11644/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375166#comment-16375166 ] Ben Lau commented on HBASE-20054: - Patches look good to me +1. > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.branch-2.v2.txt, 20054.v1.txt, 20054.v2.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the new test to master branch. > After discussion, the retry from #1 above is going to be dropped - we abort > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20054: --- Attachment: 20054.branch-2.v2.txt > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.branch-2.v2.txt, 20054.v1.txt, 20054.v2.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the new test to master branch. > After discussion, the retry from #1 above is going to be dropped - we abort > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375161#comment-16375161 ] Ben Lau commented on HBASE-20054: - Sounds good [~yuzhih...@gmail.com]. > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.v1.txt, 20054.v2.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the new test to master branch. > After discussion, the retry from #1 above is going to be dropped - we abort > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20054: --- Description: Some changes from HBASE-18282 were missing in master branch: * The retry in ReplicationSourceManager#adoptAbandonedQueues * new tests in TestLogsCleaner This issue is to forward port the new test to master branch. After discussion, the retry from #1 above is going to be dropped - we abort directly. was: Some changes from HBASE-18282 were missing in master branch: * The retry in ReplicationSourceManager#adoptAbandonedQueues * new tests in TestLogsCleaner This issue is to forward port the above to master branch > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.v1.txt, 20054.v2.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the new test to master branch. > After discussion, the retry from #1 above is going to be dropped - we abort > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375157#comment-16375157 ] Ted Yu commented on HBASE-20054: w.r.t. the retry in ReplicationSourceManager#adoptAbandonedQueues, I will put up patch for branch-2 which removes the indefinite retry. > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.v1.txt, 20054.v2.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the above to master branch -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20054: --- Attachment: 20054.v2.txt > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.v1.txt, 20054.v2.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the above to master branch -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20062) findbugs is not running on precommit checks
[
https://issues.apache.org/jira/browse/HBASE-20062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob reassigned HBASE-20062:
-
Assignee: Mike Drob
> findbugs is not running on precommit checks
> ---
>
> Key: HBASE-20062
> URL: https://issues.apache.org/jira/browse/HBASE-20062
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Assignee: Mike Drob
>Priority: Blocker
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.3
>
>
> as of build #11638 ~3 hours ago precommit isn't using findbugs:
> {quote}
> 0 findbugs0m 0s Findbugs executables are not available.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20062) findbugs is not running on precommit checks
[
https://issues.apache.org/jira/browse/HBASE-20062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375156#comment-16375156
]
Mike Drob commented on HBASE-20062:
---
k, updated the config, precommit builds after 11644 should be happy, will leave
this open to monitor
> findbugs is not running on precommit checks
> ---
>
> Key: HBASE-20062
> URL: https://issues.apache.org/jira/browse/HBASE-20062
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Assignee: Mike Drob
>Priority: Blocker
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.3
>
>
> as of build #11638 ~3 hours ago precommit isn't using findbugs:
> {quote}
> 0 findbugs0m 0s Findbugs executables are not available.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20045) When running compaction, cache recent blocks.
[
https://issues.apache.org/jira/browse/HBASE-20045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375155#comment-16375155
]
Vladimir Rodionov commented on HBASE-20045:
---
{quote}
This is to garantie a better 99.999% latency
{quote}
No it will not (at least, in most applications). 99+%% latencies are
*guaranteed* by GC activity. Especially if one runs GCG1 collector (where GC
frequency is much higher than in CMS).
> When running compaction, cache recent blocks.
> -
>
> Key: HBASE-20045
> URL: https://issues.apache.org/jira/browse/HBASE-20045
> Project: HBase
> Issue Type: New Feature
> Components: BlockCache, Compaction
>Affects Versions: 2.0.0-beta-1
>Reporter: Jean-Marc Spaggiari
>Priority: Major
>
> HBase already allows to cache blocks on flush. This is very useful for
> usecases where most queries are against recent data. However, as soon as
> their is a compaction, those blocks are evicted. It will be interesting to
> have a table level parameter to say "When compacting, cache blocks less than
> 24 hours old". That way, when running compaction, all blocks where some data
> are less than 24h hold, will be automatically cached.
>
> Very useful for table design where there is TS in the key but a long history
> (Like a year of sensor data).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19157) IntegrationTestBackupRestore should warn about missing config
[
https://issues.apache.org/jira/browse/HBASE-19157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-19157:
-
Attachment: HBASE-19157.v03.patch
> IntegrationTestBackupRestore should warn about missing config
> -
>
> Key: HBASE-19157
> URL: https://issues.apache.org/jira/browse/HBASE-19157
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: backup, integration-test
> Attachments: HBASE-19157.v01.patch, HBASE-19157.v02.patch,
> HBASE-19157.v03.patch
>
>
> I was running IntegrationTestBackupRestore on a newly created cluster running
> hbase-2.
> It failed with:
> {code}
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException):
> org.apache.hadoop.hbase.DoNotRetryIOException: The procedure is not
> registered: rolllog-proc
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.execProcedure(MasterRpcServices.java:817)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:134)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:325)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:305)
> {code}
> {{hbase backup}} command warns about missing config up-front.
> IntegrationTestBackupRestore should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375150#comment-16375150 ] Ben Lau commented on HBASE-20054: - [~yuzhih...@gmail.com] I guess it doesn't hurt to forward port the test improvement although the bug doesn't exist in master. For the other change though (queue adoption), I wonder. If master branch decided to abort the server if there is a replication error instead of doing a retry, maybe that is something we should go with instead. In other words, maybe we should backport the server abortion to branch-2, instead forward-porting the RPC retry to master. I also thought about this more and I wonder if indefinitely retrying is a good idea anyway. Maybe it would be better to have a limited # of retries and then abort. But it might be simpler to just abort immediately like in master-branch. Thoughts? > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.v1.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the above to master branch -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20062) findbugs is not running on precommit checks
[
https://issues.apache.org/jira/browse/HBASE-20062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375149#comment-16375149
]
Sean Busbey commented on HBASE-20062:
-
I believe just on the fly for precommit. dunno what the default is.
> findbugs is not running on precommit checks
> ---
>
> Key: HBASE-20062
> URL: https://issues.apache.org/jira/browse/HBASE-20062
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Priority: Blocker
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.3
>
>
> as of build #11638 ~3 hours ago precommit isn't using findbugs:
> {quote}
> 0 findbugs0m 0s Findbugs executables are not available.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375142#comment-16375142
]
Mike Drob commented on HBASE-20043:
---
Looks good overall, QA probably has something to say about checkstyle somewhere
(smile)
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch, HBASE-20043.branch-2.003.patch,
> HBASE-20043.branch-2.004.patch, HBASE-20043.branch-2.005.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20036) TestAvoidCellReferencesIntoShippedBlocks timed out
[ https://issues.apache.org/jira/browse/HBASE-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375139#comment-16375139 ] stack commented on HBASE-20036: --- It failed one of the nightlies in hadoop3 run https://builds.apache.org/job/HBase%20Nightly/job/branch-2/371/artifact/output-jdk8-hadoop3/patch-unit-root.txt Any luck [~ram_krish] > TestAvoidCellReferencesIntoShippedBlocks timed out > -- > > Key: HBASE-20036 > URL: https://issues.apache.org/jira/browse/HBASE-20036 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > > Looks like it is stuck can't flush; bad math? > See the dashboard where it hung here: > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/2428/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAvoidCellReferencesIntoShippedBlocks-output.txt > ... the provocation could be this? > 2018-02-21 04:18:44,973 DEBUG [Thread-178] bucket.BucketCache(629): This > block eefaa9a7b10e437b9fc2b55a67d63191_4356 is still referred by 1 readers. > Can not be freed now. Hence will mark this for evicting at a later point > Exception in thread "Thread-178" java.lang.AssertionError: old blocks should > still be found expected:<6> but was:<5> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > ... Then we get stuck doing this: > 2018-02-21 04:23:34,661 DEBUG [master/asf903:0.Chore.1] master.HMaster(1524): > Skipping normalization for table: testHBase16372InCompactionWritePath, as > it's either system table or doesn't have auto normalization turned on > 2018-02-21 04:23:35,695 INFO [regionserver/asf903:0.Chore.1] > regionserver.HRegionServer$PeriodicMemStoreFlusher(1752): > MemstoreFlusherChore requesting flush of hbase:meta,,1.1588230740 because > info has an old edit so flush to free WALs after random delay 286820ms > 2018-02-21 04:23:36,009 DEBUG [ReadOnlyZKClient-localhost:61855@0x227ea3ed] > zookeeper.ReadOnlyZKClient(316): 0x227ea3ed to localhost:61855 inactive for > 6ms; closing (Will reconnect when new requests) > It also failed a recent nightly for same reason: > https://builds.apache.org/job/HBase%20Nightly/job/branch-2/355/ > Any chance you'd take a look [~ram_krish]? You best at this stuff? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20055) Remove declaration of un-thrown exceptions and unused setRegionStateBackToOpen() from MergeTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375124#comment-16375124
]
Hadoop QA commented on HBASE-20055:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} hbase-server: The patch generated 0 new + 72
unchanged - 2 fixed = 72 total (was 74) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
10s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
9s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
12s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 8s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}143m 4s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20055 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911799/hbase-20055.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 1ec15e8c8a23 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ad5cd50dfc |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d;
[jira] [Updated] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20043:
--
Attachment: HBASE-20043.branch-2.005.patch
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch, HBASE-20043.branch-2.003.patch,
> HBASE-20043.branch-2.004.patch, HBASE-20043.branch-2.005.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20043:
--
Attachment: HBASE-20043.branch-2.004.patch
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch, HBASE-20043.branch-2.003.patch,
> HBASE-20043.branch-2.004.patch, HBASE-20043.branch-2.005.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20043:
--
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch, HBASE-20043.branch-2.003.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375121#comment-16375121
]
stack commented on HBASE-20043:
---
.002 addresses [~mdrob] RB comments.
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch, HBASE-20043.branch-2.003.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20043:
--
Attachment: HBASE-20043.branch-2.003.patch
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch, HBASE-20043.branch-2.003.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19400) Add missing security checks in MasterRpcServices
[ https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anonymous updated HBASE-19400: -- Status: Patch Available (was: Reopened) > Add missing security checks in MasterRpcServices > > > Key: HBASE-19400 > URL: https://issues.apache.org/jira/browse/HBASE-19400 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19400.branch-1.001.patch, > HBASE-19400.master.001.patch, HBASE-19400.master.002.patch, > HBASE-19400.master.003.patch, HBASE-19400.master.004.patch, > HBASE-19400.master.004.patch, HBASE-19400.master.005.patch, > HBASE-19400.master.006.patch, HBASE-19400.master.007.patch, > HBASE-19400.master.007.patch > > > The following RPC methods in MasterRpcServices do not have ACL check for > ADMIN rights. > - normalize > - setNormalizerRunning > - runCatalogScan > - enableCatalogJanitor > - runCleanerChore > - setCleanerChoreRunning > - execMasterService > - execProcedure > - execProcedureWithRet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375109#comment-16375109
]
Ted Yu commented on HBASE-19863:
Looks good overall.
{code}
+ @BeforeClass public static void setUpBeforeClass() throws Exception {
{code}
Tradition is to put the annotation on separate line (a few other places).
{code}
+Table ht = TEST_UTIL
{code}
Please close the table at the end of the test.
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: HBASE-19863-branch-2.patch, HBASE-19863-branch1.patch,
> HBASE-19863-test.patch, HBASE-19863.v2-branch-2.patch,
> HBASE-19863.v3-branch-2.patch, HBASE-19863.v4-branch-2.patch,
> HBASE-19863.v4-master.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The query matcher will then just skip this fake
> // key/value and the store scanner will progress to the next column. This
> // is obviously not a "real real" seek, but unlike the fake KV earlier in
> // this method, we want this to be propagated to ScanQueryMatcher.
> {noformat}
>
> For store B we would set it to fake 10:0/C3/createFirstOnRowColTS()/Maximum
> to skip C3 entirely.
> After that we
[jira] [Commented] (HBASE-20062) findbugs is not running on precommit checks
[
https://issues.apache.org/jira/browse/HBASE-20062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375108#comment-16375108
]
Mike Drob commented on HBASE-20062:
---
need to change to {{\-\-findbugs-home=/usr}} in our precommit job - can i just
do that live, or is that generated from a jenkinsfile somewhere?
What's the default value for findbugs home anyway? I tried looking through
yetus source but must have missed it
> findbugs is not running on precommit checks
> ---
>
> Key: HBASE-20062
> URL: https://issues.apache.org/jira/browse/HBASE-20062
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Priority: Blocker
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.3
>
>
> as of build #11638 ~3 hours ago precommit isn't using findbugs:
> {quote}
> 0 findbugs0m 0s Findbugs executables are not available.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20064) Disable MOB threads that are running whether you MOB or not
stack created HBASE-20064: - Summary: Disable MOB threads that are running whether you MOB or not Key: HBASE-20064 URL: https://issues.apache.org/jira/browse/HBASE-20064 Project: HBase Issue Type: Bug Reporter: stack Fix For: 2.0.0 Master starts up some cleaner and compacting threads even though no MOB. Disable them and have users explicitly enable it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375093#comment-16375093
]
stack commented on HBASE-20043:
---
[~mdrob] Yeah, sorry. The log for this test is unreadable. While trying to
figure it, was doing edits as I went
Two main things are pom.xml change and the change to random region move Action.
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375093#comment-16375093
]
stack edited comment on HBASE-20043 at 2/23/18 11:10 PM:
-
[~mdrob] Yeah, sorry. The log for this test is unreadable. While trying to
figure it, was doing edits as I went
Two main things are pom.xml change and the change to random region move Action.
was (Author: stack):
[~mdrob] Yeah, sorry. The log for this test is unreadable. While trying to
figure it, was doing edits as I went
Two main things are pom.xml change and the change to random region move Action.
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375091#comment-16375091
]
Mike Drob edited comment on HBASE-20043 at 2/23/18 11:08 PM:
-
There's a lot going on in this patch... lots of logging cleanup, lots of
updating HRI to RegionInfo, lots of refactoring things into methods... hard to
review all at once. Put it on RB?
Edit: I see it's already there, nevermind.
was (Author: mdrob):
There's a lot going on in this patch... lots of logging cleanup, lots of
updating HRI to RegionInfo, lots of refactoring things into methods... hard to
review all at once. Put it on RB?
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20043:
--
Attachment: HBASE-20043.branch-2.002.patch
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375091#comment-16375091
]
Mike Drob commented on HBASE-20043:
---
There's a lot going on in this patch... lots of logging cleanup, lots of
updating HRI to RegionInfo, lots of refactoring things into methods... hard to
review all at once. Put it on RB?
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch,
> HBASE-20043.branch-2.002.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20043:
--
Attachment: HBASE-20043.branch-2.001.patch
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20043.branch-2.001.patch
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated HBASE-19863:
Attachment: HBASE-19863.v4-branch-2.patch
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: HBASE-19863-branch-2.patch, HBASE-19863-branch1.patch,
> HBASE-19863-test.patch, HBASE-19863.v2-branch-2.patch,
> HBASE-19863.v3-branch-2.patch, HBASE-19863.v4-branch-2.patch,
> HBASE-19863.v4-master.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The query matcher will then just skip this fake
> // key/value and the store scanner will progress to the next column. This
> // is obviously not a "real real" seek, but unlike the fake KV earlier in
> // this method, we want this to be propagated to ScanQueryMatcher.
> {noformat}
>
> For store B we would set it to fake 10:0/C3/createFirstOnRowColTS()/Maximum
> to skip C3 entirely.
> After that we start searching for qualifier C5 using seekOrSkipToNextColumn
> which run first trySkipToNextColumn:
> {noformat}
> protected boolean trySkipToNextColumn(Cell cell) throws IOException {
> Cell nextCell = null;
> do {
> Cell
[jira] [Updated] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated HBASE-19863:
Attachment: (was: HBASE-19863.v4-branch-2.patch)
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: HBASE-19863-branch-2.patch, HBASE-19863-branch1.patch,
> HBASE-19863-test.patch, HBASE-19863.v2-branch-2.patch,
> HBASE-19863.v3-branch-2.patch, HBASE-19863.v4-branch-2.patch,
> HBASE-19863.v4-master.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The query matcher will then just skip this fake
> // key/value and the store scanner will progress to the next column. This
> // is obviously not a "real real" seek, but unlike the fake KV earlier in
> // this method, we want this to be propagated to ScanQueryMatcher.
> {noformat}
>
> For store B we would set it to fake 10:0/C3/createFirstOnRowColTS()/Maximum
> to skip C3 entirely.
> After that we start searching for qualifier C5 using seekOrSkipToNextColumn
> which run first trySkipToNextColumn:
> {noformat}
> protected boolean trySkipToNextColumn(Cell cell) throws IOException {
> Cell nextCell = null;
> do {
>
[jira] [Commented] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375057#comment-16375057
]
Sergey Soldatov commented on HBASE-19863:
-
[~yuzhih...@gmail.com] Ough. Didn't notice that. Reattached.
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: HBASE-19863-branch-2.patch, HBASE-19863-branch1.patch,
> HBASE-19863-test.patch, HBASE-19863.v2-branch-2.patch,
> HBASE-19863.v3-branch-2.patch, HBASE-19863.v4-branch-2.patch,
> HBASE-19863.v4-master.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The query matcher will then just skip this fake
> // key/value and the store scanner will progress to the next column. This
> // is obviously not a "real real" seek, but unlike the fake KV earlier in
> // this method, we want this to be propagated to ScanQueryMatcher.
> {noformat}
>
> For store B we would set it to fake 10:0/C3/createFirstOnRowColTS()/Maximum
> to skip C3 entirely.
> After that we start searching for qualifier C5 using seekOrSkipToNextColumn
> which run first trySkipToNextColumn:
> {noformat}
> protected boolean trySkipToNextColumn(Cell cell) throws IOException {
[jira] [Commented] (HBASE-20062) findbugs is not running on precommit checks
[
https://issues.apache.org/jira/browse/HBASE-20062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375036#comment-16375036
]
Mike Drob commented on HBASE-20062:
---
Yetus output:
{noformat}
00:00:24.321 executable '/opt/findbugs/bin/findbugs' for 'findbugs' does not
exist.
00:00:24.321 executable '/opt/findbugs/bin/computeBugHistory' for
'computeBugHistory' does not exist.
00:00:24.321 executable '/opt/findbugs/bin/convertXmlToText' for
'convertXmlToText' does not exist.
00:00:24.321 executable '/opt/findbugs/bin/filterBugs' for 'filterBugs' does
not exist.
00:00:24.321 executable '/opt/findbugs/bin/setBugDatabaseInfo' for
'setBugDatabaseInfo' does not exist.
{noformat}
poking around in the created docker image:
{noformat}
mdrob@9602b392cccf:/root$ which findbugs
/usr/bin/findbugs
{noformat}
> findbugs is not running on precommit checks
> ---
>
> Key: HBASE-20062
> URL: https://issues.apache.org/jira/browse/HBASE-20062
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Priority: Blocker
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.3
>
>
> as of build #11638 ~3 hours ago precommit isn't using findbugs:
> {quote}
> 0 findbugs0m 0s Findbugs executables are not available.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19364) Truncate_preserve fails with table when replica region > 1
[ https://issues.apache.org/jira/browse/HBASE-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375035#comment-16375035 ] huaxiang sun commented on HBASE-19364: -- I pushed the patch to branch-1. For branch-2 and master, it needs another patch for HBASE-17319 , keep it open until patch is pushed for branch-2 and master. > Truncate_preserve fails with table when replica region > 1 > -- > > Key: HBASE-19364 > URL: https://issues.apache.org/jira/browse/HBASE-19364 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-19364-branch-1-v2.patch, > HBASE-19364-branch-1-v3.patch, HBASE-19364-branch-1.patch, > HBASE-19364-branch-1.patch > > > Root cause is same as HBASE-17319, here we need to exclude secondary regions > while reading meta. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18864) NullPointerException thrown when adding rows to a table from peer cluster, table with replication factor other than 0 or 1
[
https://issues.apache.org/jira/browse/HBASE-18864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375033#comment-16375033
]
Hadoop QA commented on HBASE-18864:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
58s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
46s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
16s{color} | {color:red} hbase-server: The patch generated 1 new + 14 unchanged
- 1 fixed = 15 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
28s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 21s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 78m 48s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}111m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestWALLockup |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:e77c578 |
| JIRA Issue |
[jira] [Created] (HBASE-20063) Port HBASE-19799 (Add web UI to rsgroup) to branch-1
Andrew Purtell created HBASE-20063: -- Summary: Port HBASE-19799 (Add web UI to rsgroup) to branch-1 Key: HBASE-20063 URL: https://issues.apache.org/jira/browse/HBASE-20063 Project: HBase Issue Type: Task Components: rsgroup, UI Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.5.0, 1.4.3 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19364) Truncate_preserve fails with table when replica region > 1
[ https://issues.apache.org/jira/browse/HBASE-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375029#comment-16375029 ] Ted Yu commented on HBASE-19364: I ran TestHBaseFsck with patch which passed. Should be good to go. > Truncate_preserve fails with table when replica region > 1 > -- > > Key: HBASE-19364 > URL: https://issues.apache.org/jira/browse/HBASE-19364 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-19364-branch-1-v2.patch, > HBASE-19364-branch-1-v3.patch, HBASE-19364-branch-1.patch, > HBASE-19364-branch-1.patch > > > Root cause is same as HBASE-17319, here we need to exclude secondary regions > while reading meta. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19987) update error-prone to 2.2.0
[ https://issues.apache.org/jira/browse/HBASE-19987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375028#comment-16375028 ] Andrew Purtell commented on HBASE-19987: lgtm Planning to apply this to master and branch-2 I assume. Also branch-1? Fixes will need to be a bit different > update error-prone to 2.2.0 > --- > > Key: HBASE-19987 > URL: https://issues.apache.org/jira/browse/HBASE-19987 > Project: HBase > Issue Type: Bug >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-19987.patch > > > keep ourselves healthy and up to date -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375027#comment-16375027
]
Ted Yu commented on HBASE-19863:
The error was on branch-2 v3, not v4.
You can re-attach v4 patch one more time.
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: HBASE-19863-branch-2.patch, HBASE-19863-branch1.patch,
> HBASE-19863-test.patch, HBASE-19863.v2-branch-2.patch,
> HBASE-19863.v3-branch-2.patch, HBASE-19863.v4-branch-2.patch,
> HBASE-19863.v4-master.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The query matcher will then just skip this fake
> // key/value and the store scanner will progress to the next column. This
> // is obviously not a "real real" seek, but unlike the fake KV earlier in
> // this method, we want this to be propagated to ScanQueryMatcher.
> {noformat}
>
> For store B we would set it to fake 10:0/C3/createFirstOnRowColTS()/Maximum
> to skip C3 entirely.
> After that we start searching for qualifier C5 using seekOrSkipToNextColumn
> which run first trySkipToNextColumn:
> {noformat}
> protected boolean trySkipToNextColumn(Cell cell) throws IOException
[jira] [Commented] (HBASE-20062) findbugs is not running on precommit checks
[
https://issues.apache.org/jira/browse/HBASE-20062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375026#comment-16375026
]
Sean Busbey commented on HBASE-20062:
-
don't know how long it's been out. I'm in the middle of reverting a patch that
introduced a bunch of findbugs failures. I'll pick this up after that presuming
no one beats me to it.
> findbugs is not running on precommit checks
> ---
>
> Key: HBASE-20062
> URL: https://issues.apache.org/jira/browse/HBASE-20062
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Priority: Blocker
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.3
>
>
> as of build #11638 ~3 hours ago precommit isn't using findbugs:
> {quote}
> 0 findbugs0m 0s Findbugs executables are not available.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20062) findbugs is not running on precommit checks
Sean Busbey created HBASE-20062:
---
Summary: findbugs is not running on precommit checks
Key: HBASE-20062
URL: https://issues.apache.org/jira/browse/HBASE-20062
Project: HBase
Issue Type: Bug
Components: community, test
Reporter: Sean Busbey
Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.3
as of build #11638 ~3 hours ago precommit isn't using findbugs:
{quote}
0 findbugs0m 0s Findbugs executables are not available.
{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20043) ITBLL fails against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375020#comment-16375020
]
Mike Drob commented on HBASE-20043:
---
bq. This 'Action' was stale, from hbase1 AM days
glad our tests are finding things! even if it's only problems with the tests...
do we need a review of the other actions, or we think test framework will shake
bad ones loose?
bq. Other interesting stuff in here is we were using a hadoop2 netty but it is
changed in hadoop3 which was causing job to fail.
we need to put the version property in a profile?
bq. Also at one point during this test run we had 1k threads up in the one JVM
(we need to pare back how many we spin up by default have executors run w/
keepalive and die-off if no work...).
any idea which ones? follow up jiras?
> ITBLL fails against hadoop3
> ---
>
> Key: HBASE-20043
> URL: https://issues.apache.org/jira/browse/HBASE-20043
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Mike Drob
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
>
> This has been failing for a while, I haven't tried to bisec but it was before
> my changes for HBASE-19991 at least.
> {code}
> mvn clean verify -pl hbase-it -Dhadoop.profile=3.0
> -Dit.test=IntegrationTestBigLinkedList -Dtest=none -am
> {code}
> {code}
> 2018-02-21 16:43:13,265 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.RpcServer(464): Unexpected throwable object
> java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2018-02-21 16:43:13,276 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=60450]
> ipc.CallRunner(141): callId: 49 service: MasterService methodName: MoveRegion
> size: 106 connection: 192.168.1.134:60743 deadline: 1519253053263
> java.io.IOException:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.AssertionError:
> hri=IntegrationTestBigLinkedList,\x8E8\xE3\x8E8\xE3\x8E5,1519252895022.236bbedde32e4549691c108a1a7005a8.,
> source=, destination=mdrob-mbp.hsd1.tx.comcast.net,60456,1519252856027
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1691)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1348)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> {code}
> The assertion that it trips is:
> {code}
> // Now we can do the move
> RegionPlan rp = new RegionPlan(hri, regionState.getServerName(), dest);
> assert rp.getDestination() != null: rp.toString() + " " + dest;
> assert rp.getSource() != null: rp.toString();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
