[jira] [Commented] (HBASE-18587) Fix Flaky TestFileIOEngine
[
https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124774#comment-16124774
]
Hadoop QA commented on HBASE-18587:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 31s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}124m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure |
| | org.apache.hadoop.hbase.client.TestScanWithoutFetchingData |
| | org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay |
| | org.apache.hadoop.hbase.master.procedure.TestModifyTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestDeleteTableProcedure |
| | org.apache.hadoop.hbase.regionserver.TestRowTooBig |
| | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| | org.apache.hadoop.hbase.regionserver.wal.TestAsyncWALReplay |
| | org.apache.hadoop.hbase.mapreduce.TestWALPlayer |
| | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.mapreduce.TestHRegionPartitioner |
| | org.apache.hadoop.hbase.mapreduce.TestTableInputFormat |
| | org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
| | org.apache.hadoop.hbase.client.TestFromClientSide |
| | org.apache.hadoop.hbase.client.TestMultipleTimestamps |
| | org.apache.hadoop.hbase.client.TestAsyncTableScanAll |
| | org.apache.hadoop.hbase.client.TestAsyncReplicationAdminApi |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| | org.apache.hadoop.hbase.trace.TestHTraceHooks |
| |
[jira] [Commented] (HBASE-18271) Shade netty
[
https://issues.apache.org/jira/browse/HBASE-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124772#comment-16124772
]
Hadoop QA commented on HBASE-18271:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
4s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
1s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
4s{color} | {color:red} The patch generated 5 new + 493 unchanged - 5 fixed =
498 total (was 498) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
9s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
24s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hbase-prefix-tree in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
10s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
50s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}145m 35s{color}
| {color:red} root in the patch failed. {color}
[jira] [Updated] (HBASE-18587) Fix Flaky TestFileIOEngine
[ https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18587: -- Status: Patch Available (was: Open) > Fix Flaky TestFileIOEngine > -- > > Key: HBASE-18587 > URL: https://issues.apache.org/jira/browse/HBASE-18587 > Project: HBase > Issue Type: Bug > Components: BucketCache, test >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18587.branch-1.001.patch, > HBASE-18587.master.001.patch > > > As a part of HBASE-18533, the Jenkins report said that TestFileIOEngine > failed. I investigated and noticed that there are two cases where this test > is flaky: > len = 0 and offset 0; it will fail for > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java#L198 > will pass in -1 for offset which is invalid. > len = 0 and i = 13; it will try to specify an offset of the totalCapacity > which is not allowed. > > This patch fixes this test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18587) Fix Flaky TestFileIOEngine
[ https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18587: -- Attachment: HBASE-18587.master.001.patch > Fix Flaky TestFileIOEngine > -- > > Key: HBASE-18587 > URL: https://issues.apache.org/jira/browse/HBASE-18587 > Project: HBase > Issue Type: Bug > Components: BucketCache, test >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18587.branch-1.001.patch, > HBASE-18587.master.001.patch > > > As a part of HBASE-18533, the Jenkins report said that TestFileIOEngine > failed. I investigated and noticed that there are two cases where this test > is flaky: > len = 0 and offset 0; it will fail for > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java#L198 > will pass in -1 for offset which is invalid. > len = 0 and i = 13; it will try to specify an offset of the totalCapacity > which is not allowed. > > This patch fixes this test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18587) Fix Flaky TestFileIOEngine
[ https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18587: -- Attachment: HBASE-18587.branch-1.001.patch > Fix Flaky TestFileIOEngine > -- > > Key: HBASE-18587 > URL: https://issues.apache.org/jira/browse/HBASE-18587 > Project: HBase > Issue Type: Bug > Components: BucketCache, test >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18587.branch-1.001.patch > > > As a part of HBASE-18533, the Jenkins report said that TestFileIOEngine > failed. I investigated and noticed that there are two cases where this test > is flaky: > len = 0 and offset 0; it will fail for > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java#L198 > will pass in -1 for offset which is invalid. > len = 0 and i = 13; it will try to specify an offset of the totalCapacity > which is not allowed. > > This patch fixes this test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18587) Fix Flaky TestFileIOEngine
Zach York created HBASE-18587: - Summary: Fix Flaky TestFileIOEngine Key: HBASE-18587 URL: https://issues.apache.org/jira/browse/HBASE-18587 Project: HBase Issue Type: Bug Components: BucketCache, test Reporter: Zach York Assignee: Zach York As a part of HBASE-18533, the Jenkins report said that TestFileIOEngine failed. I investigated and noticed that there are two cases where this test is flaky: len = 0 and offset 0; it will fail for https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java#L198 will pass in -1 for offset which is invalid. len = 0 and i = 13; it will try to specify an offset of the totalCapacity which is not allowed. This patch fixes this test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18586) Multiple column families - scan performance
Partha Sarathy created HBASE-18586: -- Summary: Multiple column families - scan performance Key: HBASE-18586 URL: https://issues.apache.org/jira/browse/HBASE-18586 Project: HBase Issue Type: Bug Components: scan Reporter: Partha Sarathy I have 2 HBase tables - one with a single column family, and other has 4 column families. Both tables are keyed by same rowkey, and the column families all have a single column qualifier each, with a json string as value (each json payload is about 10-20K in size). All column families use fast-diff encoding and gzip compression. After loading about 60MM rows to each table, a scan test on (any) single column family in the 2nd table takes 4x the time to scan the single column family from the 1st table. In both cases, the scanner is bounded by a start and stop key to scan 1MM rows. Performance did not change much even after running a major compaction on both tables. Though HBase doc and other tech forums recommend not using more than 1 column family per table, nothing I have read so far suggests scan performance will linearly degrade based on number of column families. Has anyone else experienced this, and is there a simple explanation for this? To note, the reason second table has 4 column families is even though I only scan one column family at a time now, there are requirements to scan multiple column families from that table given a set of rowkeys. Thanks for any insight into the performance question. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15511) ClusterStatus should be able to return responses by scope
[ https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124750#comment-16124750 ] Ajay Jadhav commented on HBASE-15511: - +1 lgtm > ClusterStatus should be able to return responses by scope > - > > Key: HBASE-15511 > URL: https://issues.apache.org/jira/browse/HBASE-15511 > Project: HBase > Issue Type: Improvement >Reporter: Esteban Gutierrez >Assignee: Reid Chan > Fix For: 2.0.0 > > Attachments: HBASE-15511.master.001.patch, > HBASE-15511.master.002.patch, HBASE-15511.master.003.patch, > HBASE-15511.master.004.patch, HBASE-15511.master.005.patch, > HBASE-15511.master.006.patch, HBASE-15511.master.007.patch, > HBASE-15511.master.008.patch, HBASE-15511.master.009.patch, > HBASE-15511.master.010.patch, HBASE-15511.master.011.patch, > HBASE-15511.master.012.patch, HBASE-15511.master.013.patch, > HBASE-15511.master.014.patch > > > The current ClusterStatus response returns too much information about the > load per region and replication cluster wide. Sometimes that response can be > quite large (10s or 100s of MBs) and methods like getServerSize() or > getRegionsCount() don't really need the full response. One possibility is to > provide a scope (or filter) for the ClusterStatus requests to limit the > response back to the client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18585) Update release instructions involving gpg-agent
Nick Dimiduk created HBASE-18585:
Summary: Update release instructions involving gpg-agent
Key: HBASE-18585
URL: https://issues.apache.org/jira/browse/HBASE-18585
Project: HBase
Issue Type: Task
Components: build, documentation
Reporter: Nick Dimiduk
Priority: Minor
Could be what I ran into is hyper-local, but making a note here none the less.
For the build of 1.1.12RC0, I stumbled into MGPG-59. Presumably this is because
of an graded version of {{gpg-agent}}. The release build takes some time, such
that even if you purse the advice of invoking gpg before starting the build,
the default config expires before the build finished on my machine. I haven't
tested this out yet, but a solution appears to look something like
# configure {{gpg-agent}} with a longer value for {{default-cache-ttl}} in
{{$HOME/.gnupg/gpg-agent.conf}}.
# invoke {{gpg}} such that the agent will cache the passphrase. MGPG-59
suggests signing some random file.
# run {{./dev-support/make_release.sh}} as normal.
My context:
{noformat}
diocles:~ ndimiduk$ sw_vers
ProductName:Mac OS X
ProductVersion: 10.11.6
BuildVersion: 15G1611
diocles:~ ndimiduk$ gpg-agent --version
gpg-agent (GnuPG) 2.1.23
libgcrypt 1.8.0
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18584) [hbasecon] Assume domain hbasecon.com
stack created HBASE-18584: - Summary: [hbasecon] Assume domain hbasecon.com Key: HBASE-18584 URL: https://issues.apache.org/jira/browse/HBASE-18584 Project: HBase Issue Type: Task Reporter: stack Cloudera currently hosts and owns hbasecon.com. They would like to transfer ownership and hosting if wanted to hbase PMC. This issue is about assuming the domain. Currently hbasecon.com is pointing at the eventbrite used by hbase asia. Eventbrite won't let you link into their site after the event has happened; you get redirected to the home page. I've asked to have hbasecon.com point back to our archive pages at hbase.apache.org/www.hbasecon.com/. Should be done in next day or so. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18583) [hbasecon] Update the hbasecon archive
[ https://issues.apache.org/jira/browse/HBASE-18583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18583: -- Tags: beginner Labels: beginner (was: ) > [hbasecon] Update the hbasecon archive > -- > > Key: HBASE-18583 > URL: https://issues.apache.org/jira/browse/HBASE-18583 > Project: HBase > Issue Type: Task >Reporter: stack > Labels: beginner > > The HBase PMC now hosts the hbasecon archive. It is here: > http://hbase.apache.org/www.hbasecon.com/ > The homepage needs an edit to remove the 'hosted by cloudera' but also to > fill in hbasecon east 2016, hbasecon west 2017, and the recent hbasecon asia > 2017. > The slideshare for hbasecon east needs to have video links added. Ditto for > hbasecon west and hbasecon asia. > TODO: a blog post on apache hbase site that points at the new updated > archive. There are also some great pictures from hbasecon asia at least that > could do w/ uploading. > Links: > https://www.youtube.com/channel/UCy25rIFxWRBokFg-2Cm83BQ/videos?sort=dd_id=0=0 > https://www.slideshare.net/search/slideshow?searchfrom=header=hbaseconeast2016 > https://www.slideshare.net/search/slideshow?searchfrom=header=hbasecon2017=any=all=en= > Look for Achieving MultiTenancy.mp4 > https://www.slideshare.net/search/slideshow?searchfrom=header=hbaseconasia2017=any=all=en= > Marking this as a beginner task since it mostly admin. I keep meaning to just > do it but never get to it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18583) [hbasecon] Update the hbasecon archive
stack created HBASE-18583: - Summary: [hbasecon] Update the hbasecon archive Key: HBASE-18583 URL: https://issues.apache.org/jira/browse/HBASE-18583 Project: HBase Issue Type: Task Reporter: stack The HBase PMC now hosts the hbasecon archive. It is here: http://hbase.apache.org/www.hbasecon.com/ The homepage needs an edit to remove the 'hosted by cloudera' but also to fill in hbasecon east 2016, hbasecon west 2017, and the recent hbasecon asia 2017. The slideshare for hbasecon east needs to have video links added. Ditto for hbasecon west and hbasecon asia. TODO: a blog post on apache hbase site that points at the new updated archive. There are also some great pictures from hbasecon asia at least that could do w/ uploading. Links: https://www.youtube.com/channel/UCy25rIFxWRBokFg-2Cm83BQ/videos?sort=dd_id=0=0 https://www.slideshare.net/search/slideshow?searchfrom=header=hbaseconeast2016 https://www.slideshare.net/search/slideshow?searchfrom=header=hbasecon2017=any=all=en= Look for Achieving MultiTenancy.mp4 https://www.slideshare.net/search/slideshow?searchfrom=header=hbaseconasia2017=any=all=en= Marking this as a beginner task since it mostly admin. I keep meaning to just do it but never get to it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18471) Deleted qualifier re-appearing after multiple puts.
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124705#comment-16124705
]
Hadoop QA commented on HBASE-18471:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 16s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
14s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}114m
32s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18471 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12881625/HBASE-18471.v0.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux eeac63df9221 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 173dce7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8058/testReport/ |
| modules | C: hbase-common hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8058/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Deleted qualifier re-appearing after multiple
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124697#comment-16124697 ] Hudson commented on HBASE-18020: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK7 #1900 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1900/]) Revert "HBASE-18020 Update API Compliance Checker to Incorporate (ndimiduk: rev 952c1ebab43354acf27f89b7c060ed75fea1a879) * (delete) dev-support/checkcompatibility.py * (add) dev-support/check_compatibility.sh > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18271) Shade netty
[ https://issues.apache.org/jira/browse/HBASE-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18271: -- Attachment: HBASE-18271.master.005.patch > Shade netty > --- > > Key: HBASE-18271 > URL: https://issues.apache.org/jira/browse/HBASE-18271 > Project: HBase > Issue Type: Sub-task > Components: rpc >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18271.master.001.patch, > HBASE-18271.master.002.patch, HBASE-18271.master.003.patch, > HBASE-18271.master.004.patch, HBASE-18271.master.005.patch, > HBASE-18271-Shade-netty-Purge-mention-of-netty-all.003.patch > > > Our new prefatory project, hbase-thirdparty, includes a relocated netty > 4.1.12. This issue is about changing references in hbase to make use of this > shaded netty. This way we will take ourselves out of the clashing library > saga and change netty as we see fit. > One kink is the inclusion inside the netty-all jar of an .so. We need to make > sure it gets loaded and that on linux we are doing native epoll; all should > be in place to do this but need to add a system property to bin/hbase for > netty to pick up. TODO as part of this issue. See tail of upgrade guava > sibling issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18582) Correct the docs for Mutation#setCellVisibility
[
https://issues.apache.org/jira/browse/HBASE-18582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18582:
---
Component/s: documentation
> Correct the docs for Mutation#setCellVisibility
> ---
>
> Key: HBASE-18582
> URL: https://issues.apache.org/jira/browse/HBASE-18582
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Chia-Ping Tsai
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
>
> {code}
> /**
>* Sets the visibility expression associated with cells in this Mutation.
>* It is illegal to set CellVisibility on Delete
> mutation.
>* @param expression
>*/
> {code}
> HBASE-10885 enable Delete to be specified with Cell Visibility, so we should
> remove the comment " It is illegal to set..."
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18582) Correct the docs for Mutation#setCellVisibility
[
https://issues.apache.org/jira/browse/HBASE-18582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18582:
---
Fix Version/s: 1.2.7
1.5.0
1.3.2
1.4.0
2.0.0
> Correct the docs for Mutation#setCellVisibility
> ---
>
> Key: HBASE-18582
> URL: https://issues.apache.org/jira/browse/HBASE-18582
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Chia-Ping Tsai
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
>
> {code}
> /**
>* Sets the visibility expression associated with cells in this Mutation.
>* It is illegal to set CellVisibility on Delete
> mutation.
>* @param expression
>*/
> {code}
> HBASE-10885 enable Delete to be specified with Cell Visibility, so we should
> remove the comment " It is illegal to set..."
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18582) Correct the docs for Mutation#setCellVisibility
Chia-Ping Tsai created HBASE-18582:
--
Summary: Correct the docs for Mutation#setCellVisibility
Key: HBASE-18582
URL: https://issues.apache.org/jira/browse/HBASE-18582
Project: HBase
Issue Type: Task
Reporter: Chia-Ping Tsai
Priority: Minor
{code}
/**
* Sets the visibility expression associated with cells in this Mutation.
* It is illegal to set CellVisibility on Delete
mutation.
* @param expression
*/
{code}
HBASE-10885 enable Delete to be specified with Cell Visibility, so we should
remove the comment " It is illegal to set..."
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18582) Correct the docs for Mutation#setCellVisibility
[
https://issues.apache.org/jira/browse/HBASE-18582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18582:
---
Labels: beginner (was: )
> Correct the docs for Mutation#setCellVisibility
> ---
>
> Key: HBASE-18582
> URL: https://issues.apache.org/jira/browse/HBASE-18582
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Chia-Ping Tsai
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
>
> {code}
> /**
>* Sets the visibility expression associated with cells in this Mutation.
>* It is illegal to set CellVisibility on Delete
> mutation.
>* @param expression
>*/
> {code}
> HBASE-10885 enable Delete to be specified with Cell Visibility, so we should
> remove the comment " It is illegal to set..."
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18572) Delete#addColumn can't remove the cell which has no visibility label
[
https://issues.apache.org/jira/browse/HBASE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18572:
---
Status: Open (was: Patch Available)
> Delete#addColumn can't remove the cell which has no visibility label
>
>
> Key: HBASE-18572
> URL: https://issues.apache.org/jira/browse/HBASE-18572
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Attachments: HBASE-18572.ut.patch
>
>
> {code:title=DefaultVisibilityLabelServiceImpl.java}
> @Override
> public boolean matchVisibility(List putVisTags, Byte putTagsFormat,
> List deleteVisTags,
> Byte deleteTagsFormat) throws IOException {
> if ((deleteTagsFormat != null && deleteTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)
> && (putTagsFormat == null || putTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)) {
> if (putVisTags.isEmpty()) {
> // Early out if there are no tags in the cell
> return false;
> }
> if (putTagsFormat == null) {
> return matchUnSortedVisibilityTags(putVisTags, deleteVisTags);
> } else {
> return matchOrdinalSortedVisibilityTags(putVisTags, deleteVisTags);
> }
> }
> throw new IOException("Unexpected tag format passed for comparison,
> deleteTagsFormat : "
> + deleteTagsFormat + ", putTagsFormat : " + putTagsFormat);
> }
> {code}
> If the putVisTags is empty, it will return the false directly. Hence, the
> Delete#addColumn can't "see" the cell as well as deleting it. The
> Delete#addColumns, by contrast, doesn't invoke the
> cp#prePrepareTimeStampForDeleteVersion so it can remove the cell.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124689#comment-16124689 ] Hudson commented on HBASE-18020: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #1984 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1984/]) Revert "HBASE-18020 Update API Compliance Checker to Incorporate (ndimiduk: rev 952c1ebab43354acf27f89b7c060ed75fea1a879) * (delete) dev-support/checkcompatibility.py * (add) dev-support/check_compatibility.sh > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-15042) refactor so that site materials are in the Standard Maven Place
[
https://issues.apache.org/jira/browse/HBASE-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124681#comment-16124681
]
Andrew Purtell edited comment on HBASE-15042 at 8/12/17 7:08 PM:
-
I eventually had to stop copying back doc updates from master branch for 0.98
releases. I forget what broke them exactly. I think a skinning change that
relied on alpha-this or beta-that maven subcomponents. We had to build with
JDK6 for 0.98 and the maven binaries were incompatible at the bytecode level.
If we do LTS with 1.1 on JDK7 sooner or later that can happen again.
Unless there is a change in doc relevant for 1.1 I don't think it necessary to
further update them on branch.
We probably should update how to contribute sections in our book (for patches
and doc) to call out that changes to be applied to branches which also have doc
updates need the doc updates ported back. (As opposed to relying on naive copy
of master docs back to branch for update.)
was (Author: apurtell):
I eventually had to stop copying back doc updates from master branch for 0.98
releases. I forget what broke them exactly. I think a skinning change that
relied on alpha-this or beta-that maven subcomponents. Unless there is a change
in doc relevant for 1.1 I don't think it necessary to further update them on
branch.
We probably should update how to contribute sections in our book (for patches
and doc) to call out that changes to be applied to branches which also have doc
updates need the doc updates ported back. (As opposed to relying on naive copy
of master docs back to branch for update.)
> refactor so that site materials are in the Standard Maven Place
> ---
>
> Key: HBASE-15042
> URL: https://issues.apache.org/jira/browse/HBASE-15042
> Project: HBase
> Issue Type: Task
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15042.master.001.patch,
> HBASE-15042.master.002.patch, HBASE-15042.master.002.patch,
> HBASE-15042.master.003.patch
>
>
> for some reason we currently have our site materials in {{src/main/site}}
> rather than the maven prescribed {{src/site}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-15042) refactor so that site materials are in the Standard Maven Place
[
https://issues.apache.org/jira/browse/HBASE-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124681#comment-16124681
]
Andrew Purtell edited comment on HBASE-15042 at 8/12/17 7:06 PM:
-
I eventually had to stop copying back doc updates from master branch for 0.98
releases. I forget what broke them exactly. I think a skinning change that
relied on alpha-this or beta-that maven subcomponents. Unless there is a change
in doc relevant for 1.1 I don't think it necessary to further update them on
branch.
We probably should update how to contribute sections in our book (for patches
and doc) to call out that changes to be applied to branches which also have doc
updates need the doc updates ported back. (As opposed to relying on naive copy
of master docs back to branch for update.)
was (Author: apurtell):
I eventually had to stop copying back doc updates from master branch for 0.98
releases. Unless there is a change in doc relevant for 1.1 I don't think it
necessary to further update them on branch.
We probably should update how to contribute sections in our book (for patches
and doc) to call out that changes to be applied to branches which also have doc
updates need the doc updates ported back. (As opposed to relying on naive copy
of master docs back to branch for update.)
> refactor so that site materials are in the Standard Maven Place
> ---
>
> Key: HBASE-15042
> URL: https://issues.apache.org/jira/browse/HBASE-15042
> Project: HBase
> Issue Type: Task
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15042.master.001.patch,
> HBASE-15042.master.002.patch, HBASE-15042.master.002.patch,
> HBASE-15042.master.003.patch
>
>
> for some reason we currently have our site materials in {{src/main/site}}
> rather than the maven prescribed {{src/site}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-15042) refactor so that site materials are in the Standard Maven Place
[
https://issues.apache.org/jira/browse/HBASE-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124681#comment-16124681
]
Andrew Purtell commented on HBASE-15042:
I eventually had to stop copying back doc updates from master branch for 0.98
releases. Unless there is a change in doc relevant for 1.1 I don't think it
necessary to further update them on branch.
We probably should update how to contribute sections in our book (for patches
and doc) to call out that changes to be applied to branches which also have doc
updates need the doc updates ported back. (As opposed to relying on naive copy
of master docs back to branch for update.)
> refactor so that site materials are in the Standard Maven Place
> ---
>
> Key: HBASE-15042
> URL: https://issues.apache.org/jira/browse/HBASE-15042
> Project: HBase
> Issue Type: Task
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15042.master.001.patch,
> HBASE-15042.master.002.patch, HBASE-15042.master.002.patch,
> HBASE-15042.master.003.patch
>
>
> for some reason we currently have our site materials in {{src/main/site}}
> rather than the maven prescribed {{src/site}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Reopened] (HBASE-15042) refactor so that site materials are in the Standard Maven Place
[
https://issues.apache.org/jira/browse/HBASE-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-15042:
--
> refactor so that site materials are in the Standard Maven Place
> ---
>
> Key: HBASE-15042
> URL: https://issues.apache.org/jira/browse/HBASE-15042
> Project: HBase
> Issue Type: Task
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15042.master.001.patch,
> HBASE-15042.master.002.patch, HBASE-15042.master.002.patch,
> HBASE-15042.master.003.patch
>
>
> for some reason we currently have our site materials in {{src/main/site}}
> rather than the maven prescribed {{src/site}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-15042) refactor so that site materials are in the Standard Maven Place
[
https://issues.apache.org/jira/browse/HBASE-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124661#comment-16124661
]
Nick Dimiduk commented on HBASE-15042:
--
[~Jan Hentschel], [~busbey] My approach for 1.1 releases has been pretty naive,
works like this.
{noformat}
## bring back docs from master
git checkout apache-rw/master -- src/main/asciidoc
git add .
mvn clean install -DskipTests -Dcheckstyle.skip
mvn clean site -DskipTests -Dcheckstyle.skip -Dmaven.javadoc.skip=true
open target/site/index.html # verify html, pdf it all looks good
{noformat}
The patch for master does not apply cleanly on branch-1.1, mostly due to
conflicts in the top-level pom.xml. Re-opening while we decide on a course of
action for release braches. FYI [~stack], [~misty], [~mantonov], [~apurtell]
> refactor so that site materials are in the Standard Maven Place
> ---
>
> Key: HBASE-15042
> URL: https://issues.apache.org/jira/browse/HBASE-15042
> Project: HBase
> Issue Type: Task
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15042.master.001.patch,
> HBASE-15042.master.002.patch, HBASE-15042.master.002.patch,
> HBASE-15042.master.003.patch
>
>
> for some reason we currently have our site materials in {{src/main/site}}
> rather than the maven prescribed {{src/site}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18471) Deleted qualifier re-appearing after multiple puts.
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18471:
---
Status: Patch Available (was: Open)
> Deleted qualifier re-appearing after multiple puts.
> ---
>
> Key: HBASE-18471
> URL: https://issues.apache.org/jira/browse/HBASE-18471
> Project: HBase
> Issue Type: Bug
> Components: Deletes, hbase, scan
>Affects Versions: 2.0.0-alpha-1, 1.3.1, 1.3.0, 3.0.0
>Reporter: Thomas Martens
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18471.v0.patch, HBaseDmlTest.java
>
>
> The qualifier of a deleted row (with keep deleted cells true) re-appears
> after re-inserting the same row multiple times (with different timestamp)
> with an empty qualifier.
> Scenario:
> # Put row with family and qualifier (timestamp 1).
> # Delete entire row (timestamp 2).
> # Put same row again with family without qualifier (timestamp 3).
> A scan (latest version) returns the row with family without qualifier,
> version 3 (which is correct).
> # Put the same row again with family without qualifier (timestamp 4).
> A scan (latest version) returns multiple rows:
> * the row with family without qualifier, version 4 (which is correct).
> * the row with family with qualifier, version 1 (which is wrong).
> There is a test scenario attached.
> output:
> 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml
> 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml
> 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml
> 13:42:58,592 [main] client.HBaseAdmin - Created test_dml
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with
> timestamp: '1'
> Scan printout =>
> Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier',
> Value: 'myValue'
> Delete row: 'myRow'
> Scan printout =>
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '3'
> Scan printout =>
> Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '4'
> Scan printout =>
> Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier:
> 'myQualifier', Value: 'myValue'{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18471) Deleted qualifier re-appearing after multiple puts.
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18471:
---
Attachment: HBASE-18471.v0.patch
> Deleted qualifier re-appearing after multiple puts.
> ---
>
> Key: HBASE-18471
> URL: https://issues.apache.org/jira/browse/HBASE-18471
> Project: HBase
> Issue Type: Bug
> Components: Deletes, hbase, scan
>Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1
>Reporter: Thomas Martens
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18471.v0.patch, HBaseDmlTest.java
>
>
> The qualifier of a deleted row (with keep deleted cells true) re-appears
> after re-inserting the same row multiple times (with different timestamp)
> with an empty qualifier.
> Scenario:
> # Put row with family and qualifier (timestamp 1).
> # Delete entire row (timestamp 2).
> # Put same row again with family without qualifier (timestamp 3).
> A scan (latest version) returns the row with family without qualifier,
> version 3 (which is correct).
> # Put the same row again with family without qualifier (timestamp 4).
> A scan (latest version) returns multiple rows:
> * the row with family without qualifier, version 4 (which is correct).
> * the row with family with qualifier, version 1 (which is wrong).
> There is a test scenario attached.
> output:
> 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml
> 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml
> 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml
> 13:42:58,592 [main] client.HBaseAdmin - Created test_dml
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with
> timestamp: '1'
> Scan printout =>
> Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier',
> Value: 'myValue'
> Delete row: 'myRow'
> Scan printout =>
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '3'
> Scan printout =>
> Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '4'
> Scan printout =>
> Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier:
> 'myQualifier', Value: 'myValue'{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-15042) refactor so that site materials are in the Standard Maven Place
[
https://issues.apache.org/jira/browse/HBASE-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124659#comment-16124659
]
Nick Dimiduk commented on HBASE-15042:
--
The changes here are breaking the release process I've been using for
branch-1.1, which was to simply copy in the latest doc changes from master.
Because of pom.xml differences, I think this will need to be backported
manually to all active branches.
> refactor so that site materials are in the Standard Maven Place
> ---
>
> Key: HBASE-15042
> URL: https://issues.apache.org/jira/browse/HBASE-15042
> Project: HBase
> Issue Type: Task
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15042.master.001.patch,
> HBASE-15042.master.002.patch, HBASE-15042.master.002.patch,
> HBASE-15042.master.003.patch
>
>
> for some reason we currently have our site materials in {{src/main/site}}
> rather than the maven prescribed {{src/site}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-18020: - Fix Version/s: (was: 1.1.12) > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124649#comment-16124649 ] Nick Dimiduk commented on HBASE-18020: -- Reverted from branch-1.1 for now. > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.12 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[
https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124638#comment-16124638
]
Nick Dimiduk commented on HBASE-18020:
--
I get some slightly different stacktrace running the version on master.
{noformat}
diocles:dev-support ndimiduk$ ./checkcompatibility.py rel/1.1.11 branch-1.1
INFO:root:Source revision: rel/1.1.11
INFO:root:Destination revision: branch-1.1
INFO:root:Java ACC is already downloaded.
INFO:root:Removing scratch dir /Users/ndimiduk/repos/hbase/target/compat-check
INFO:root:Creating empty scratch dir
/Users/ndimiduk/repos/hbase/target/compat-check
INFO:root:Checking out b67a13e6b6285563da51d926645ba3832acfd2bf in
/Users/ndimiduk/repos/hbase/target/compat-check/src
INFO:root:Checking out ebc1d2bf2c52bbaf5e2bc81afab6298558978239 in
/Users/ndimiduk/repos/hbase/target/compat-check/dst
INFO:root:Building in /Users/ndimiduk/repos/hbase/target/compat-check/src
Traceback (most recent call last):
File "./checkcompatibility.py", line 514, in
main()
File "./checkcompatibility.py", line 492, in main
build_tree(src_dir, args.verbose)
File "./checkcompatibility.py", line 138, in build_tree
subprocess.check_call(mvn_cmd, cwd=java_path)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 535, in check_call
retcode = call(*popenargs, **kwargs)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 522, in call
return Popen(*popenargs, **kwargs).wait()
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 710, in __init__
errread, errwrite)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 1335, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
{noformat}
> Update API Compliance Checker to Incorporate Improvements Done in Hadoop
>
>
> Key: HBASE-18020
> URL: https://issues.apache.org/jira/browse/HBASE-18020
> Project: HBase
> Issue Type: Improvement
> Components: API, community
>Reporter: Alex Leblang
>Assignee: Alex Leblang
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.12
>
> Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch,
> HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch,
> HBASE-18020.branch-1.2.004.patch
>
>
> Recently the Hadoop community has made a number of improvements in their api
> compliance checker based on feedback from the hbase and kudu community. We
> should adopt these changes ourselves.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18577) shaded client includes several non-relocated third party dependencies
[ https://issues.apache.org/jira/browse/HBASE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-18577: - Fix Version/s: (was: 1.1.12) 1.1.13 > shaded client includes several non-relocated third party dependencies > - > > Key: HBASE-18577 > URL: https://issues.apache.org/jira/browse/HBASE-18577 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0, 1.1.2, 1.3.0, 2.0.0-alpha-1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.13 > > Attachments: HBASE-18577.WIP.0.patch, HBASE-18577.WIP.-1.patch > > > we have some unexpected unrelocated third party dependencies in our shaded > artifacts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17513) Thrift Server 1 uses different QOP settings than RPC and Thrift Server 2 and can easily be misconfigured so there is no encryption when the operator expects it.
[
https://issues.apache.org/jira/browse/HBASE-17513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-17513:
-
Fix Version/s: (was: 1.1.12)
1.1.13
> Thrift Server 1 uses different QOP settings than RPC and Thrift Server 2 and
> can easily be misconfigured so there is no encryption when the operator
> expects it.
>
>
> Key: HBASE-17513
> URL: https://issues.apache.org/jira/browse/HBASE-17513
> Project: HBase
> Issue Type: Bug
> Components: documentation, security, Thrift, Usability
>Affects Versions: 2.0.0, 1.2.0, 1.3.0, 0.98.15, 1.0.3, 1.1.3
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.2.7, 1.1.13
>
>
> As of HBASE-14400 the setting {{hbase.thrift.security.qop}} was unified to
> behave the same as the general HBase RPC protection. However, this only
> happened for the Thrift2 server. The Thrift server found in the thrift
> package (aka Thrift Server 1) still hard codes the old configs of 'auth',
> 'auth-int', and 'auth-conf'.
> Additionally, these Quality of Protection (qop) settings are used only by the
> SASL transport. If a user configures the HBase Thrift Server to make use of
> the HTTP transport (to enable doAs proxying e.g. for Hue) then a QOP setting
> of 'privacy' or 'auth-conf' won't get them encryption as expected.
> We should
> 1) update {{hbase-thrift/src/main/.../thrift/ThriftServerRunner}} to rely on
> {{SaslUtil}} to use the same 'authentication', 'integrity', 'privacy' configs
> in a backward compatible way
> 2) also have ThriftServerRunner warn when both {{hbase.thrift.security.qop}}
> and {{hbase.regionserver.thrift.http}} are set, since the latter will cause
> the former to be ignored. (users should be directed to
> {{hbase.thrift.ssl.enabled}} and related configs to ensure their transport is
> encrypted when using the HTTP transport.)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-14223) Meta WALs are not cleared if meta region was closed and RS aborts
[
https://issues.apache.org/jira/browse/HBASE-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-14223:
-
Fix Version/s: (was: 1.1.12)
1.1.13
> Meta WALs are not cleared if meta region was closed and RS aborts
> -
>
> Key: HBASE-14223
> URL: https://issues.apache.org/jira/browse/HBASE-14223
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13
>
> Attachments: HBASE-14223logs, hbase-14223_v0.patch,
> hbase-14223_v1-branch-1.patch, hbase-14223_v2-branch-1.patch,
> hbase-14223_v3-branch-1.patch, hbase-14223_v3-branch-1.patch,
> hbase-14223_v3-master.patch
>
>
> When an RS opens meta, and later closes it, the WAL(FSHlog) is not closed.
> The last WAL file just sits there in the RS WAL directory. If RS stops
> gracefully, the WAL file for meta is deleted. Otherwise if RS aborts, WAL for
> meta is not cleaned. It is also not split (which is correct) since master
> determines that the RS no longer hosts meta at the time of RS abort.
> From a cluster after running ITBLL with CM, I see a lot of {{-splitting}}
> directories left uncleaned:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs
> Found 31 items
> drwxr-xr-x - hbase hadoop 0 2015-06-05 01:14
> /apps/hbase/data/WALs/hregion-58203265
> drwxr-xr-x - hbase hadoop 0 2015-06-05 07:54
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433489308745-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 09:28
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433494382959-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 10:01
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433498252205-splitting
> ...
> {code}
> The directories contain WALs from meta:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting
> Found 2 items
> -rw-r--r-- 3 hbase hadoop 201608 2015-06-05 03:15
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> -rw-r--r-- 3 hbase hadoop 44420 2015-06-05 04:36
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> The RS hosted the meta region for some time:
> {code}
> 2015-06-05 03:14:28,692 INFO [PostOpenDeployTasks:1588230740]
> zookeeper.MetaTableLocator: Setting hbase:meta region location in ZooKeeper
> as os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285
> ...
> 2015-06-05 03:15:17,302 INFO
> [RS_CLOSE_META-os-enis-dal-test-jun-4-5:16020-0] regionserver.HRegion: Closed
> hbase:meta,,1.1588230740
> {code}
> In between, a WAL is created:
> {code}
> 2015-06-05 03:15:11,707 INFO
> [RS_OPEN_META-os-enis-dal-test-jun-4-5:16020-0-MetaLogRoller] wal.FSHLog:
> Rolled WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> with entries=385, filesize=196.88 KB; new WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> When CM killed the region server later master did not see these WAL files:
> {code}
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:46,075
> INFO [MASTER_SERVER_OPERATIONS-os-enis-dal-test-jun-4-3:16000-0]
> master.SplitLogManager: started splitting 2 logs in
> [hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting]
> for [os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285]
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:47,300
> INFO [main-EventThread] wal.WALSplitter: Archived processed log
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> to
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/oldWALs/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:50,497
> INFO [main-EventThread]
[jira] [Updated] (HBASE-14391) Empty regionserver WAL will never be deleted although the coresponding regionserver has been stale
[
https://issues.apache.org/jira/browse/HBASE-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-14391:
-
Fix Version/s: (was: 1.1.12)
1.1.13
> Empty regionserver WAL will never be deleted although the coresponding
> regionserver has been stale
> --
>
> Key: HBASE-14391
> URL: https://issues.apache.org/jira/browse/HBASE-14391
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.0.2
>Reporter: Qianxi Zhang
>Assignee: Qianxi Zhang
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13
>
> Attachments: HBASE-14391-master-v3.patch,
> HBASE_14391_master_v4.patch, HBASE_14391_trunk_v1.patch,
> HBASE_14391_trunk_v2.patch, WALs-leftover-dir.txt
>
>
> When I restarted the hbase cluster in which there was few data, I found there
> are two directories for one host with different timestamp which indicates
> that the old regionserver wal directory is not deleted.
> FHLog#989
> {code}
> @Override
> public void close() throws IOException {
> shutdown();
> final FileStatus[] files = getFiles();
> if (null != files && 0 != files.length) {
> for (FileStatus file : files) {
> Path p = getWALArchivePath(this.fullPathArchiveDir, file.getPath());
> // Tell our listeners that a log is going to be archived.
> if (!this.listeners.isEmpty()) {
> for (WALActionsListener i : this.listeners) {
> i.preLogArchive(file.getPath(), p);
> }
> }
> if (!FSUtils.renameAndSetModifyTime(fs, file.getPath(), p)) {
> throw new IOException("Unable to rename " + file.getPath() + " to "
> + p);
> }
> // Tell our listeners that a log was archived.
> if (!this.listeners.isEmpty()) {
> for (WALActionsListener i : this.listeners) {
> i.postLogArchive(file.getPath(), p);
> }
> }
> }
> LOG.debug("Moved " + files.length + " WAL file(s) to " +
> FSUtils.getPath(this.fullPathArchiveDir));
> }
> LOG.info("Closed WAL: " + toString());
> }
> {code}
> When regionserver is stopped, the hlog will be archived, so wal/regionserver
> is empty in hdfs.
> MasterFileSystem#252
> {code}
> if (curLogFiles == null || curLogFiles.length == 0) {
> // Empty log folder. No recovery needed
> continue;
> }
> {code}
> The regionserver directory will be not splitted, it makes sense. But it will
> be not deleted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-15580) Tag coprocessor limitedprivate scope to StoreFile.Reader
[
https://issues.apache.org/jira/browse/HBASE-15580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-15580:
-
Fix Version/s: (was: 1.1.12)
1.1.13
> Tag coprocessor limitedprivate scope to StoreFile.Reader
>
>
> Key: HBASE-15580
> URL: https://issues.apache.org/jira/browse/HBASE-15580
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13
>
> Attachments: HBASE-15580_branch-1.0.patch, HBASE-15580.patch
>
>
> For phoenix local indexing we need to have custom storefile reader
> constructor(IndexHalfStoreFileReader) to distinguish from other storefile
> readers. So wanted to mark StoreFile.Reader scope as
> InterfaceAudience.LimitedPrivate("Coprocessor")
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-14610) IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client
[ https://issues.apache.org/jira/browse/HBASE-14610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-14610: - Fix Version/s: (was: 1.1.12) 1.1.13 > IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client > -- > > Key: HBASE-14610 > URL: https://issues.apache.org/jira/browse/HBASE-14610 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: Enis Soztutar > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13 > > Attachments: output > > > HBASE-14535 introduces an IT to simulate a running cluster with RPC servers > and RPC clients doing requests against the servers. > It passes with the sync client, but fails with async client. Probably we need > to take a look. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17885) Backport HBASE-15871 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-17885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-17885: - Fix Version/s: (was: 1.1.12) 1.1.13 > Backport HBASE-15871 to branch-1 > > > Key: HBASE-17885 > URL: https://issues.apache.org/jira/browse/HBASE-17885 > Project: HBase > Issue Type: Bug > Components: Scanners >Affects Versions: 1.3.1, 1.2.5, 1.1.8 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.3.2, 1.2.7, 1.1.13 > > > Will try to rebase the branch-1 patch at the earliest. Hope the fix versions > are correct. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124625#comment-16124625
]
Chia-Ping Tsai commented on HBASE-18554:
LGTM.
[~anoop.hbase] Any suggestions?
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch,
> HBASE-18554.master.002.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-18554:
--
Attachment: HBASE-18554.master.002.patch
When row doesn't match, throw IllegalArgumentException().
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch,
> HBASE-18554.master.002.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-18554:
--
Status: Open (was: Patch Available)
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18471) Deleted qualifier re-appearing after multiple puts.
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18471:
---
Fix Version/s: 1.2.7
1.5.0
1.3.2
1.4.0
2.0.0
> Deleted qualifier re-appearing after multiple puts.
> ---
>
> Key: HBASE-18471
> URL: https://issues.apache.org/jira/browse/HBASE-18471
> Project: HBase
> Issue Type: Bug
> Components: Deletes, hbase, scan
>Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1
>Reporter: Thomas Martens
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBaseDmlTest.java
>
>
> The qualifier of a deleted row (with keep deleted cells true) re-appears
> after re-inserting the same row multiple times (with different timestamp)
> with an empty qualifier.
> Scenario:
> # Put row with family and qualifier (timestamp 1).
> # Delete entire row (timestamp 2).
> # Put same row again with family without qualifier (timestamp 3).
> A scan (latest version) returns the row with family without qualifier,
> version 3 (which is correct).
> # Put the same row again with family without qualifier (timestamp 4).
> A scan (latest version) returns multiple rows:
> * the row with family without qualifier, version 4 (which is correct).
> * the row with family with qualifier, version 1 (which is wrong).
> There is a test scenario attached.
> output:
> 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml
> 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml
> 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml
> 13:42:58,592 [main] client.HBaseAdmin - Created test_dml
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with
> timestamp: '1'
> Scan printout =>
> Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier',
> Value: 'myValue'
> Delete row: 'myRow'
> Scan printout =>
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '3'
> Scan printout =>
> Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '4'
> Scan printout =>
> Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier:
> 'myQualifier', Value: 'myValue'{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18528) DON'T allow user to modify the passed table/column descriptor
[ https://issues.apache.org/jira/browse/HBASE-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124620#comment-16124620 ] Chia-Ping Tsai commented on HBASE-18528: [~anoop.hbase] Could you take a look? Thanks > DON'T allow user to modify the passed table/column descriptor > - > > Key: HBASE-18528 > URL: https://issues.apache.org/jira/browse/HBASE-18528 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, master >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-18528.v0.patch, HBASE-18528.v1.patch, > HBASE-18528.v2.patch > > > We are replacing the HTableDescriptor by TableDescriptor from code base. The > TableDescriptor is designed to be a read-only object so user can't modifiy it > through MasterObserver. HBASE-18502 change many methods of MasterObserver to > use TableDescriptor but some deprecated methods still accept the > HTableDescriptor. User may be confused by why some methods can't modify the > table descriptor. > In short, Should we allow user to modify the passed table descriptor? > # if yes, we should introduce a mechanism that user can return a modified > table descripror > # if no, we should pass ImmutableHTableDescriptor to user. Or we just remove > all methods accepting the HTableDescriptor > Ditto for HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15511) ClusterStatus should be able to return responses by scope
[ https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124596#comment-16124596 ] Chia-Ping Tsai commented on HBASE-15511: [~ajayjadhav] Any more comment? > ClusterStatus should be able to return responses by scope > - > > Key: HBASE-15511 > URL: https://issues.apache.org/jira/browse/HBASE-15511 > Project: HBase > Issue Type: Improvement >Reporter: Esteban Gutierrez >Assignee: Reid Chan > Fix For: 2.0.0 > > Attachments: HBASE-15511.master.001.patch, > HBASE-15511.master.002.patch, HBASE-15511.master.003.patch, > HBASE-15511.master.004.patch, HBASE-15511.master.005.patch, > HBASE-15511.master.006.patch, HBASE-15511.master.007.patch, > HBASE-15511.master.008.patch, HBASE-15511.master.009.patch, > HBASE-15511.master.010.patch, HBASE-15511.master.011.patch, > HBASE-15511.master.012.patch, HBASE-15511.master.013.patch, > HBASE-15511.master.014.patch > > > The current ClusterStatus response returns too much information about the > load per region and replication cluster wide. Sometimes that response can be > quite large (10s or 100s of MBs) and methods like getServerSize() or > getRegionsCount() don't really need the full response. One possibility is to > provide a scope (or filter) for the ClusterStatus requests to limit the > response back to the client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124585#comment-16124585
]
Reid Chan commented on HBASE-18554:
---
Other mutations: Delete, Put and Increment all throw
IOException(WrongRowIOException).
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18271) Shade netty
[
https://issues.apache.org/jira/browse/HBASE-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124583#comment-16124583
]
Hadoop QA commented on HBASE-18271:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
5s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
4s{color} | {color:red} The patch generated 5 new + 493 unchanged - 5 fixed =
498 total (was 498) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
7s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 55s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
33s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hbase-prefix-tree in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 28s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
54s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
15s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}142m 19s{color}
| {color:red} root in the patch failed. {color} |
|
[jira] [Commented] (HBASE-18526) FIFOCompactionPolicy pre-check uses wrong scope
[
https://issues.apache.org/jira/browse/HBASE-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124565#comment-16124565
]
Lars George commented on HBASE-18526:
-
lgtm, +1, thanks Vlad
> FIFOCompactionPolicy pre-check uses wrong scope
> ---
>
> Key: HBASE-18526
> URL: https://issues.apache.org/jira/browse/HBASE-18526
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.3.1
>Reporter: Lars George
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18526-v1.patch
>
>
> See https://issues.apache.org/jira/browse/HBASE-14468
> It adds this check to {{HMaster.checkCompactionPolicy()}}:
> {code}
> // 1. Check TTL
> if (hcd.getTimeToLive() == HColumnDescriptor.DEFAULT_TTL) {
> message = "Default TTL is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 2. Check min versions
> if (hcd.getMinVersions() > 0) {
> message = "MIN_VERSION > 0 is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 3. blocking file count
> String sbfc = htd.getConfigurationValue(HStore.BLOCKING_STOREFILES_KEY);
> if (sbfc != null) {
> blockingFileCount = Integer.parseInt(sbfc);
> }
> if (blockingFileCount < 1000) {
> message =
> "blocking file count '" + HStore.BLOCKING_STOREFILES_KEY + "' "
> + blockingFileCount
> + " is below recommended minimum of 1000";
> throw new IOException(message);
> }
> {code}
> Why does it only check the blocking file count on the HTD level, while
> others are check on the HCD level? Doing this for example fails
> because of it:
> {noformat}
> hbase(main):008:0> create 'ttltable', { NAME => 'cf1', TTL => 300,
> CONFIGURATION => { 'hbase.hstore.defaultengine.compactionpolicy.class'
> => 'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
> 'hbase.hstore.blockingStoreFiles' => 2000 } }
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: blocking file
> count 'hbase.hstore.blockingStoreFiles' 10 is below recommended
> minimum of 1000 Set hbase.table.sanity.checks to false at conf or
> table descriptor if you want to bypass sanity checks
> at
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1782)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1663)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1545)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58549)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> Caused by: java.io.IOException: blocking file count
> 'hbase.hstore.blockingStoreFiles' 10 is below recommended minimum of
> 1000
> at
> org.apache.hadoop.hbase.master.HMaster.checkCompactionPolicy(HMaster.java:1773)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1661)
> ... 7 more
> {noformat}
> The check should be performed on the column family level instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18555) Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and Query
[
https://issues.apache.org/jira/browse/HBASE-18555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124560#comment-16124560
]
Hudson commented on HBASE-18555:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3518 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3518/])
HBASE-18555: Remove redundant familyMap.put() from addxxx() of (jerryjch: rev
173dce73471da005fb6780a7e7b65b43bad481e2)
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Append.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/Increment.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Put.java
> Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and
> Query
> ---
>
> Key: HBASE-18555
> URL: https://issues.apache.org/jira/browse/HBASE-18555
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18555.master.000.patch,
> HBASE-18555.master.001.patch, HBASE-18555.master.002.patch
>
>
> In addxxx() functions of Mutation(Append, Delete, Increment and Put) and
> Query(Get and Scan), there are redundant Map#put() calls which could be
> removed to improve the performance.
> For example, in Put#addColumn() and addImmutable(), after getting the cell
> list of the given family and add the cell into the list, the code puts
> (key=family, value=list) into familyMap.
> In addColumn(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value);
> list.add(kv);
> familyMap.put(CellUtil.cloneFamily(kv), list); // <-- here
> return this;
> {code}
> In addImmutable(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value, tag);
> list.add(kv);
> familyMap.put(family, list); // <-- here
> return this;
> {code}
> I think those put() for Map only take effect when getCellList(family) returns
> a new allocated ArrayList. When the list for a family already exist, put()
> for Map will update the value to the reference of the list, but actually, the
> reference of the list is not changed.
> Those put() do not do any harm in terms of the correctness when they are
> here. But it could be removed to improve the performance. familyMap searches
> for key and set the new value and return the old value. Those operation take
> some time but actually are not needed.
> The put() could be moved into Mutation#getCellList(family)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18557) change splitable to mergeable in MergeTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-18557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124559#comment-16124559
]
Hudson commented on HBASE-18557:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3518 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3518/])
HBASE-18557: Change splitable to mergeable in MergeTableRegionsProcedure
(jerryjch: rev 95e883967cbb383b48d8fae548fb55b88c7f0529)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
> change splitable to mergeable in MergeTableRegionsProcedure
> ---
>
> Key: HBASE-18557
> URL: https://issues.apache.org/jira/browse/HBASE-18557
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18557-master-v1.patch, HBASE-18557-master-v1.patch
>
>
> {code}
> private boolean isMergeable(final MasterProcedureEnv env, final RegionState
> rs)
> throws IOException {
> GetRegionInfoResponse response =
> Util.getRegionInfoResponse(env, rs.getServerName(), rs.getRegion());
> return response.hasSplittable() && response.getSplittable();
> }
> {code}
> should be mergeable
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124555#comment-16124555
]
Reid Chan commented on HBASE-18554:
---
Ya, whether to throw exception made me confused.
So, which one should be chosen, making behavioral change, or not throwing
exception until 3.0, and throwing runtime exception temporarily to keep api
compatibility?
Btw, there is a WrongRowIOException.
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18555) Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and Query
[
https://issues.apache.org/jira/browse/HBASE-18555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124551#comment-16124551
]
Hudson commented on HBASE-18555:
SUCCESS: Integrated in Jenkins build HBase-2.0 #315 (See
[https://builds.apache.org/job/HBase-2.0/315/])
HBASE-18555: Remove redundant familyMap.put() from addxxx() of (jerryjch: rev
8cebf7f1a818a419ef32a48a24c3b5df27dd980d)
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/Increment.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Append.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Put.java
> Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and
> Query
> ---
>
> Key: HBASE-18555
> URL: https://issues.apache.org/jira/browse/HBASE-18555
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18555.master.000.patch,
> HBASE-18555.master.001.patch, HBASE-18555.master.002.patch
>
>
> In addxxx() functions of Mutation(Append, Delete, Increment and Put) and
> Query(Get and Scan), there are redundant Map#put() calls which could be
> removed to improve the performance.
> For example, in Put#addColumn() and addImmutable(), after getting the cell
> list of the given family and add the cell into the list, the code puts
> (key=family, value=list) into familyMap.
> In addColumn(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value);
> list.add(kv);
> familyMap.put(CellUtil.cloneFamily(kv), list); // <-- here
> return this;
> {code}
> In addImmutable(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value, tag);
> list.add(kv);
> familyMap.put(family, list); // <-- here
> return this;
> {code}
> I think those put() for Map only take effect when getCellList(family) returns
> a new allocated ArrayList. When the list for a family already exist, put()
> for Map will update the value to the reference of the list, but actually, the
> reference of the list is not changed.
> Those put() do not do any harm in terms of the correctness when they are
> here. But it could be removed to improve the performance. familyMap searches
> for key and set the new value and return the old value. Those operation take
> some time but actually are not needed.
> The put() could be moved into Mutation#getCellList(family)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124549#comment-16124549
]
Chia-Ping Tsai commented on HBASE-18554:
I'm not a fan of eating exception. We can make behavioral change (throw
unchecked exception?).Also make it as Deprecated with comment saying "Will
throw IOException in 3.0"
BTW, why not throw illegalargumentexception directly? The IOE can't represent
the incorrect data.
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124544#comment-16124544
]
Hadoop QA commented on HBASE-18554:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 13s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
28s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18554 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12881607/HBASE-18554.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux cd4aee98ba64 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 173dce7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8057/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8057/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
>
[jira] [Updated] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-18554:
--
Assignee: Reid Chan
Status: Patch Available (was: Open)
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124539#comment-16124539
]
Reid Chan edited comment on HBASE-18554 at 8/12/17 10:51 AM:
-
Remove {code}@SuppressWarnings("unchecked"){code}
for it's no longer unchecked; add a row check for a passed cell and ignore it
if fails the check;
{code}
if (!Bytes.equals(this.row, CellUtil.cloneRow(cell))) {
// If not the same row, just ignore it and return.
return this;
}
{code}
no unit test provided.
was (Author: reidchan):
Remove {code@SuppressWarnings("unchecked")}
for it's no longer unchecked; add a row check for a passed cell and ignore it
if fails the check; no unit test provided.
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124539#comment-16124539
]
Reid Chan edited comment on HBASE-18554 at 8/12/17 10:50 AM:
-
Remove {code@SuppressWarnings("unchecked")}
for it's no longer unchecked; add a row check for a passed cell and ignore it
if fails the check; no unit test provided.
was (Author: reidchan):
Remove {code:@SuppressWarnings("unchecked")}
for it's no longer unchecked; add a row check for a passed cell and ignore it
if fails the check; no unit test provided.
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124539#comment-16124539
]
Reid Chan edited comment on HBASE-18554 at 8/12/17 10:49 AM:
-
Remove {code:@SuppressWarnings("unchecked")}
for it's no longer unchecked; add a row check for a passed cell and ignore it
if fails the check; no unit test provided.
was (Author: reidchan):
Remove {code:@SuppressWarnings("unchecked")} for it's no longer unchecked; add
a row check for a passed cell and ignore it if fails the check; no unit test
provided.
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18554) Append#add doesn't check the row of passed cell
[
https://issues.apache.org/jira/browse/HBASE-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-18554:
--
Attachment: HBASE-18554.master.001.patch
Remove {code:@SuppressWarnings("unchecked")} for it's no longer unchecked; add
a row check for a passed cell and ignore it if fails the check; no unit test
provided.
> Append#add doesn't check the row of passed cell
> ---
>
> Key: HBASE-18554
> URL: https://issues.apache.org/jira/browse/HBASE-18554
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
> Labels: beginner
> Attachments: HBASE-18554.master.001.patch
>
>
> {code}
> @SuppressWarnings("unchecked")
> public Append add(final Cell cell) {
> // Presume it is KeyValue for now.
> byte [] family = CellUtil.cloneFamily(cell);
> List list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<>(1);
> }
> // find where the new entry should be placed in the List
> list.add(cell);
> this.familyMap.put(family, list);
> return this;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18271) Shade netty
[ https://issues.apache.org/jira/browse/HBASE-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124527#comment-16124527 ] stack commented on HBASE-18271: --- .004 Fixes. I like that shellcheck thing. Tried to make it happy. The hadoop failed compile checks are good ones finding a io.netty reference buried in the MR class loading. > Shade netty > --- > > Key: HBASE-18271 > URL: https://issues.apache.org/jira/browse/HBASE-18271 > Project: HBase > Issue Type: Sub-task > Components: rpc >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18271.master.001.patch, > HBASE-18271.master.002.patch, HBASE-18271.master.003.patch, > HBASE-18271.master.004.patch, > HBASE-18271-Shade-netty-Purge-mention-of-netty-all.003.patch > > > Our new prefatory project, hbase-thirdparty, includes a relocated netty > 4.1.12. This issue is about changing references in hbase to make use of this > shaded netty. This way we will take ourselves out of the clashing library > saga and change netty as we see fit. > One kink is the inclusion inside the netty-all jar of an .so. We need to make > sure it gets loaded and that on linux we are doing native epoll; all should > be in place to do this but need to add a system property to bin/hbase for > netty to pick up. TODO as part of this issue. See tail of upgrade guava > sibling issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18271) Shade netty
[ https://issues.apache.org/jira/browse/HBASE-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18271: -- Attachment: HBASE-18271.master.004.patch > Shade netty > --- > > Key: HBASE-18271 > URL: https://issues.apache.org/jira/browse/HBASE-18271 > Project: HBase > Issue Type: Sub-task > Components: rpc >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18271.master.001.patch, > HBASE-18271.master.002.patch, HBASE-18271.master.003.patch, > HBASE-18271.master.004.patch, > HBASE-18271-Shade-netty-Purge-mention-of-netty-all.003.patch > > > Our new prefatory project, hbase-thirdparty, includes a relocated netty > 4.1.12. This issue is about changing references in hbase to make use of this > shaded netty. This way we will take ourselves out of the clashing library > saga and change netty as we see fit. > One kink is the inclusion inside the netty-all jar of an .so. We need to make > sure it gets loaded and that on linux we are doing native epoll; all should > be in place to do this but need to add a system property to bin/hbase for > netty to pick up. TODO as part of this issue. See tail of upgrade guava > sibling issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18555) Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and Query
[
https://issues.apache.org/jira/browse/HBASE-18555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124526#comment-16124526
]
Hudson commented on HBASE-18555:
FAILURE: Integrated in Jenkins build HBase-1.4 #851 (See
[https://builds.apache.org/job/HBase-1.4/851/])
HBASE-18555: Remove redundant familyMap.put() from addxxx() of (jerryjch: rev
7bd2795ee927df841a8c0ce3574598e9a6811739)
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Append.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/Increment.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Put.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
> Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and
> Query
> ---
>
> Key: HBASE-18555
> URL: https://issues.apache.org/jira/browse/HBASE-18555
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18555.master.000.patch,
> HBASE-18555.master.001.patch, HBASE-18555.master.002.patch
>
>
> In addxxx() functions of Mutation(Append, Delete, Increment and Put) and
> Query(Get and Scan), there are redundant Map#put() calls which could be
> removed to improve the performance.
> For example, in Put#addColumn() and addImmutable(), after getting the cell
> list of the given family and add the cell into the list, the code puts
> (key=family, value=list) into familyMap.
> In addColumn(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value);
> list.add(kv);
> familyMap.put(CellUtil.cloneFamily(kv), list); // <-- here
> return this;
> {code}
> In addImmutable(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value, tag);
> list.add(kv);
> familyMap.put(family, list); // <-- here
> return this;
> {code}
> I think those put() for Map only take effect when getCellList(family) returns
> a new allocated ArrayList. When the list for a family already exist, put()
> for Map will update the value to the reference of the list, but actually, the
> reference of the list is not changed.
> Those put() do not do any harm in terms of the correctness when they are
> here. But it could be removed to improve the performance. familyMap searches
> for key and set the new value and return the old value. Those operation take
> some time but actually are not needed.
> The put() could be moved into Mutation#getCellList(family)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18579) Enable core dump by default for docker
[ https://issues.apache.org/jira/browse/HBASE-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18579: --- Attachment: 18579.v2.txt Verified patch v2 where core dump is generated under /tmp > Enable core dump by default for docker > -- > > Key: HBASE-18579 > URL: https://issues.apache.org/jira/browse/HBASE-18579 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Minor > Attachments: 18579.v1.txt, 18579.v2.txt > > > In recent debugging experience, I found that by default the ulimit value > prohibits the generation of core dump. This makes debugging difficult. > We should enable core dump generation by default. > https://www.akadia.com/services/ora_enable_core.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18541) [C++] Segfaults from JNI
[
https://issues.apache.org/jira/browse/HBASE-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124523#comment-16124523
]
Ted Yu commented on HBASE-18541:
Managed to generate core dump where:
{code}
Core was generated by
`/usr/src/hbase/hbase-native-client/buck-out/gen/core/retry-test
--gtest_color=n'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x7fe32bc33135 in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64//jre/lib/amd64/server/libjvm.so
[Current thread is 1 (Thread 0x7fe32c343840 (LWP 19436))]
(gdb) bt
#0 0x7fe32bc33135 in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64//jre/lib/amd64/server/libjvm.so
#1 0x7fe31885053e in ?? ()
#2 0xe0d55920 in ?? ()
#3 0xfefd3110 in ?? ()
#4 0x7ffdd7ce7210 in ?? ()
#5 0x7fe32b66dfed in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64//jre/lib/amd64/server/libjvm.so
#6 0x7fe318282b10 in ?? ()
#7 0x029426a0 in ?? ()
#8 0x7fe318282b10 in ?? ()
#9 0x0010 in ?? ()
#10 0x7fe32b4b32fd in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64//jre/lib/amd64/server/libjvm.so
#11 0x7fe318282b10 in ?? ()
#12 0x0010 in ?? ()
#13 0x7fe31840c5c8 in ?? ()
#14 0xfefd3110 in ?? ()
{code}
there was no method from native client shown above.
> [C++] Segfaults from JNI
>
>
> Key: HBASE-18541
> URL: https://issues.apache.org/jira/browse/HBASE-18541
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Ted Yu
>
> retry-test and multi-retry-test fails flakily when run with
> {code}
> buck test --all --no-results-cache
> {code}
> or when run in a loop:
> {code}
> for i in `seq 1 10`; do buck test --no-results-cache core:retry-test || break
> 1; done
> {code}
> The problem seems to be within the JNI internals and usually happens at the
> create table method call. I was not able to inspect much, but the comments in
> our mini-cluster indicate that we may need to use global references instead
> of local ones. I suspect the problem happens when there is a GC run for the
> test since the failure happens usually after some time (but almost always in
> create table method).
> [~ted_yu] do you mind taking a look at this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18271) Shade netty
[
https://issues.apache.org/jira/browse/HBASE-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124519#comment-16124519
]
Hadoop QA commented on HBASE-18271:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
5s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
5s{color} | {color:red} The patch generated 6 new + 492 unchanged - 6 fixed =
498 total (was 498) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
7s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
41s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
16s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.2. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
51s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.3. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
22s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m
48s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.5. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
25s{color} | {color:red} hbase-client generated 2 new + 0 unchanged - 0 fixed =
2 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 3m
6s{color} | {color:red} root generated 2 new + 23 unchanged - 0 fixed = 25
total (was 23) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
[jira] [Updated] (HBASE-18577) shaded client includes several non-relocated third party dependencies
[ https://issues.apache.org/jira/browse/HBASE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18577: Affects Version/s: 1.1.2 > shaded client includes several non-relocated third party dependencies > - > > Key: HBASE-18577 > URL: https://issues.apache.org/jira/browse/HBASE-18577 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0, 1.1.2, 1.3.0, 2.0.0-alpha-1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18577.WIP.0.patch, HBASE-18577.WIP.-1.patch > > > we have some unexpected unrelocated third party dependencies in our shaded > artifacts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18577) shaded client includes several non-relocated third party dependencies
[ https://issues.apache.org/jira/browse/HBASE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18577: Fix Version/s: 1.1.12 > shaded client includes several non-relocated third party dependencies > - > > Key: HBASE-18577 > URL: https://issues.apache.org/jira/browse/HBASE-18577 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0, 1.3.0, 2.0.0-alpha-1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18577.WIP.0.patch, HBASE-18577.WIP.-1.patch > > > we have some unexpected unrelocated third party dependencies in our shaded > artifacts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18577) shaded client includes several non-relocated third party dependencies
[ https://issues.apache.org/jira/browse/HBASE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18577: Attachment: HBASE-18577.WIP.0.patch -01 * relocates codahale metrics, commons-crypto, and curator * leaves as exposed dependencies jaxb-api, jersey, jaxb-impl, javax inject ( :( ) * removes more web apps * ignores default xml configs from hadoop/hbase * ignores hadoop version-properties files this is still a WIP. the test still fails and there are more dependencies to work on. > shaded client includes several non-relocated third party dependencies > - > > Key: HBASE-18577 > URL: https://issues.apache.org/jira/browse/HBASE-18577 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0, 1.3.0, 2.0.0-alpha-1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18577.WIP.0.patch, HBASE-18577.WIP.-1.patch > > > we have some unexpected unrelocated third party dependencies in our shaded > artifacts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18577) shaded client includes several non-relocated third party dependencies
[
https://issues.apache.org/jira/browse/HBASE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124492#comment-16124492
]
Sean Busbey commented on HBASE-18577:
-
test is run with:
{{mvn -DskipTests -pl hbase-shaded/hbase-shaded-check-invariants -am -Prelease
verify}}
> shaded client includes several non-relocated third party dependencies
> -
>
> Key: HBASE-18577
> URL: https://issues.apache.org/jira/browse/HBASE-18577
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 1.3.0, 2.0.0-alpha-1
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18577.WIP.-1.patch
>
>
> we have some unexpected unrelocated third party dependencies in our shaded
> artifacts.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18577) shaded client includes several non-relocated third party dependencies
[ https://issues.apache.org/jira/browse/HBASE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18577: Fix Version/s: 2.0.0-alpha-2 1.2.7 1.5.0 1.3.2 1.4.0 3.0.0 > shaded client includes several non-relocated third party dependencies > - > > Key: HBASE-18577 > URL: https://issues.apache.org/jira/browse/HBASE-18577 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0, 1.3.0, 2.0.0-alpha-1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18577.WIP.-1.patch > > > we have some unexpected unrelocated third party dependencies in our shaded > artifacts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18577) shaded client includes several non-relocated third party dependencies
[ https://issues.apache.org/jira/browse/HBASE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18577: Affects Version/s: 1.2.0 1.3.0 2.0.0-alpha-1 > shaded client includes several non-relocated third party dependencies > - > > Key: HBASE-18577 > URL: https://issues.apache.org/jira/browse/HBASE-18577 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0, 1.3.0, 2.0.0-alpha-1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18577.WIP.-1.patch > > > we have some unexpected unrelocated third party dependencies in our shaded > artifacts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18555) Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and Query
[
https://issues.apache.org/jira/browse/HBASE-18555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124488#comment-16124488
]
Jerry He commented on HBASE-18555:
--
001 is committed with no test change.
> Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and
> Query
> ---
>
> Key: HBASE-18555
> URL: https://issues.apache.org/jira/browse/HBASE-18555
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18555.master.000.patch,
> HBASE-18555.master.001.patch, HBASE-18555.master.002.patch
>
>
> In addxxx() functions of Mutation(Append, Delete, Increment and Put) and
> Query(Get and Scan), there are redundant Map#put() calls which could be
> removed to improve the performance.
> For example, in Put#addColumn() and addImmutable(), after getting the cell
> list of the given family and add the cell into the list, the code puts
> (key=family, value=list) into familyMap.
> In addColumn(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value);
> list.add(kv);
> familyMap.put(CellUtil.cloneFamily(kv), list); // <-- here
> return this;
> {code}
> In addImmutable(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value, tag);
> list.add(kv);
> familyMap.put(family, list); // <-- here
> return this;
> {code}
> I think those put() for Map only take effect when getCellList(family) returns
> a new allocated ArrayList. When the list for a family already exist, put()
> for Map will update the value to the reference of the list, but actually, the
> reference of the list is not changed.
> Those put() do not do any harm in terms of the correctness when they are
> here. But it could be removed to improve the performance. familyMap searches
> for key and set the new value and return the old value. Those operation take
> some time but actually are not needed.
> The put() could be moved into Mutation#getCellList(family)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18555) Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and Query
[
https://issues.apache.org/jira/browse/HBASE-18555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerry He updated HBASE-18555:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0-alpha-2
1.5.0
3.0.0
Status: Resolved (was: Patch Available)
Committed to branch-1, branch-2 and master.
Thanks for the patch. Thanks for the review.
> Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and
> Query
> ---
>
> Key: HBASE-18555
> URL: https://issues.apache.org/jira/browse/HBASE-18555
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18555.master.000.patch,
> HBASE-18555.master.001.patch, HBASE-18555.master.002.patch
>
>
> In addxxx() functions of Mutation(Append, Delete, Increment and Put) and
> Query(Get and Scan), there are redundant Map#put() calls which could be
> removed to improve the performance.
> For example, in Put#addColumn() and addImmutable(), after getting the cell
> list of the given family and add the cell into the list, the code puts
> (key=family, value=list) into familyMap.
> In addColumn(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value);
> list.add(kv);
> familyMap.put(CellUtil.cloneFamily(kv), list); // <-- here
> return this;
> {code}
> In addImmutable(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value, tag);
> list.add(kv);
> familyMap.put(family, list); // <-- here
> return this;
> {code}
> I think those put() for Map only take effect when getCellList(family) returns
> a new allocated ArrayList. When the list for a family already exist, put()
> for Map will update the value to the reference of the list, but actually, the
> reference of the list is not changed.
> Those put() do not do any harm in terms of the correctness when they are
> here. But it could be removed to improve the performance. familyMap searches
> for key and set the new value and return the old value. Those operation take
> some time but actually are not needed.
> The put() could be moved into Mutation#getCellList(family)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
