[GitHub] [hbase] Apache-HBase commented on issue #830: HBASE-23281: Track meta region locations in masters
Apache-HBase commented on issue #830: HBASE-23281: Track meta region locations in masters URL: https://github.com/apache/hbase/pull/830#issuecomment-560988000 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 52s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 17 new or modified test files. | ||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 56s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | compile | 1m 49s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | checkstyle | 2m 25s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | shadedjars | 5m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +0 :ok: | spotbugs | 4m 58s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 46s | HBASE-18095/client-locate-meta-no-zookeeper passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 50s | the patch passed | | +1 :green_heart: | compile | 2m 16s | the patch passed | | +1 :green_heart: | javac | 2m 16s | the patch passed | | +1 :green_heart: | checkstyle | 2m 52s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 6m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 22m 43s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 28s | the patch passed | | +1 :green_heart: | findbugs | 8m 3s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 17s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 1m 2s | hbase-zookeeper in the patch passed. | | -1 :x: | unit | 311m 19s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 1m 16s | The patch does not generate ASF License warnings. | | | | 402m 40s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestFromClientSide | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | | hadoop.hbase.master.TestAssignmentManagerMetrics | | | hadoop.hbase.client.TestFromClientSide3 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-830/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/830 | | JIRA Issue | HBASE-23281 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux bece045780fe 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-830/out/precommit/personality/provided.sh | | git revision | HBASE-18095/client-locate-meta-no-zookeeper / efab11afc5 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-830/13/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-830/13/testReport/ | | Max. process+thread count | 5000 (vs. ulimit of 1) | | modules | C: hbase-client hbase-zookeeper hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-830/13/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to
[jira] [Commented] (HBASE-23281) Track meta region changes on masters
[
https://issues.apache.org/jira/browse/HBASE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986589#comment-16986589
]
HBase QA commented on HBASE-23281:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
52s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 17 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-18095/client-locate-meta-no-zookeeper Compile
Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
56s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
25s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
58s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
46s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
24s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
22m 43s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
17s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
2s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}311m 19s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}402m 40s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestFromClientSide |
| |
[jira] [Updated] (HBASE-23345) Duplicate code about deciding whether a table need replicate
[
https://issues.apache.org/jira/browse/HBASE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-23345:
---
Status: Patch Available (was: Open)
> Duplicate code about deciding whether a table need replicate
> -
>
> Key: HBASE-23345
> URL: https://issues.apache.org/jira/browse/HBASE-23345
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Major
> Attachments: HBASE-23345.branch-2.001.patch,
> HBASE-23345.branch-2.002.patch
>
>
> There is duplicate code in ReplicationUtils.contains and
> ReplicationPeerConfig.needToReplicate about deciding whether a table need
> replicate to the peer cluster.
> And the implementation of ReplicationPeerConfig.needToReplicate has a bug, it
> return false when replicateAllUserTables flag is true and excludeTableCFsMap
> contains not all of cfs.
>
> We should copy the code from ReplicationUtils.contains to
> ReplicationPeerConfig.needToReplicate, and delete
> ReplicationUtils.contains.Because ReplicationUtils is from module
> replication, ReplicationPeerConfig is from module client, and module
> replication depends on module client.
>
> In the following code, ReplicationPeerConfig.needToReplicate return false,
> when replicateAllUserTables flag is true and excludeTableCFsMap contains not
> all of cfs.
> {code:java}
> public boolean needToReplicate(TableName table) {
> if (replicateAllUserTables) {
> ..
> if (excludeTableCFsMap != null && excludeTableCFsMap.containsKey(table)) {
> return false;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23345) Table need to replication unless all of cfs are excluded
[
https://issues.apache.org/jira/browse/HBASE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-23345:
---
Priority: Minor (was: Major)
> Table need to replication unless all of cfs are excluded
>
>
> Key: HBASE-23345
> URL: https://issues.apache.org/jira/browse/HBASE-23345
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Minor
> Attachments: HBASE-23345.branch-2.001.patch,
> HBASE-23345.branch-2.002.patch
>
>
> There is duplicate code in ReplicationUtils.contains and
> ReplicationPeerConfig.needToReplicate about deciding whether a table need
> replicate to the peer cluster.
> And the implementation of ReplicationPeerConfig.needToReplicate has a bug, it
> return false when replicateAllUserTables flag is true and excludeTableCFsMap
> contains not all of cfs.
>
> We should copy the code from ReplicationUtils.contains to
> ReplicationPeerConfig.needToReplicate, and delete
> ReplicationUtils.contains.Because ReplicationUtils is from module
> replication, ReplicationPeerConfig is from module client, and module
> replication depends on module client.
>
> In the following code, ReplicationPeerConfig.needToReplicate return false,
> when replicateAllUserTables flag is true and excludeTableCFsMap contains not
> all of cfs.
> {code:java}
> public boolean needToReplicate(TableName table) {
> if (replicateAllUserTables) {
> ..
> if (excludeTableCFsMap != null && excludeTableCFsMap.containsKey(table)) {
> return false;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23345) Table need to replication unless all of cfs are excluded
[
https://issues.apache.org/jira/browse/HBASE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-23345:
---
Summary: Table need to replication unless all of cfs are excluded (was:
Duplicate code about deciding whether a table need replicate)
> Table need to replication unless all of cfs are excluded
>
>
> Key: HBASE-23345
> URL: https://issues.apache.org/jira/browse/HBASE-23345
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Major
> Attachments: HBASE-23345.branch-2.001.patch,
> HBASE-23345.branch-2.002.patch
>
>
> There is duplicate code in ReplicationUtils.contains and
> ReplicationPeerConfig.needToReplicate about deciding whether a table need
> replicate to the peer cluster.
> And the implementation of ReplicationPeerConfig.needToReplicate has a bug, it
> return false when replicateAllUserTables flag is true and excludeTableCFsMap
> contains not all of cfs.
>
> We should copy the code from ReplicationUtils.contains to
> ReplicationPeerConfig.needToReplicate, and delete
> ReplicationUtils.contains.Because ReplicationUtils is from module
> replication, ReplicationPeerConfig is from module client, and module
> replication depends on module client.
>
> In the following code, ReplicationPeerConfig.needToReplicate return false,
> when replicateAllUserTables flag is true and excludeTableCFsMap contains not
> all of cfs.
> {code:java}
> public boolean needToReplicate(TableName table) {
> if (replicateAllUserTables) {
> ..
> if (excludeTableCFsMap != null && excludeTableCFsMap.containsKey(table)) {
> return false;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23345) Table need to replication unless all of cfs are excluded
[
https://issues.apache.org/jira/browse/HBASE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-23345:
---
Fix Version/s: 2.1.9
2.2.3
2.3.0
3.0.0
> Table need to replication unless all of cfs are excluded
>
>
> Key: HBASE-23345
> URL: https://issues.apache.org/jira/browse/HBASE-23345
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
> Attachments: HBASE-23345.branch-2.001.patch,
> HBASE-23345.branch-2.002.patch
>
>
> There is duplicate code in ReplicationUtils.contains and
> ReplicationPeerConfig.needToReplicate about deciding whether a table need
> replicate to the peer cluster.
> And the implementation of ReplicationPeerConfig.needToReplicate has a bug, it
> return false when replicateAllUserTables flag is true and excludeTableCFsMap
> contains not all of cfs.
>
> We should copy the code from ReplicationUtils.contains to
> ReplicationPeerConfig.needToReplicate, and delete
> ReplicationUtils.contains.Because ReplicationUtils is from module
> replication, ReplicationPeerConfig is from module client, and module
> replication depends on module client.
>
> In the following code, ReplicationPeerConfig.needToReplicate return false,
> when replicateAllUserTables flag is true and excludeTableCFsMap contains not
> all of cfs.
> {code:java}
> public boolean needToReplicate(TableName table) {
> if (replicateAllUserTables) {
> ..
> if (excludeTableCFsMap != null && excludeTableCFsMap.containsKey(table)) {
> return false;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23355) Add a conf key to decide which operations can trigger prefetch
[
https://issues.apache.org/jira/browse/HBASE-23355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986576#comment-16986576
]
chenxu commented on HBASE-23355:
So here we should ignore the flush and compaction case when do prefetch, and
only consider open and bulkload?
> Add a conf key to decide which operations can trigger prefetch
> --
>
> Key: HBASE-23355
> URL: https://issues.apache.org/jira/browse/HBASE-23355
> Project: HBase
> Issue Type: Improvement
>Reporter: chenxu
>Assignee: chenxu
>Priority: Major
>
> Copy the comment from anoop under
> https://issues.apache.org/jira/browse/HBASE-23066?focusedCommentId=16984140=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16984140
> {quote}On a side note, (Not related to this issue) when we have cache on
> write ON as well as prefetch also On, do we do the caching part for the
> flushed files twice? When it is written, its already been added to cache.
> Later as part of HFile reader open, the prefetch threads will again do a read
> and add to cache!
> {quote}
> It's better to have a conf key to decide which operations(flush, compaction,
> regionOpen, bulkload) can trigger the prefetch. In this way, if cacheonwrite
> is enabled, we can exclude the flush and compaction operations when doing
> prefetch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22096) /storeFile.jsp shows CorruptHFileException when the storeFile is a reference file
[ https://issues.apache.org/jira/browse/HBASE-22096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-22096: - Fix Version/s: 3.0.0 > /storeFile.jsp shows CorruptHFileException when the storeFile is a reference > file > - > > Key: HBASE-22096 > URL: https://issues.apache.org/jira/browse/HBASE-22096 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0 > > Attachments: screanshot.png > > > When the storeFile is a reference file, /storeFile.jsp for the storeFile > shows the following error: > !screanshot.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] binlijin commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
binlijin commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-561037779 try use the following import: import java.util.List; import org.apache.hadoop.hbase.HBaseInterfaceAudience; import org.apache.hadoop.hbase.wal.WAL; import org.apache.yetus.audience.InterfaceAudience; import org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting; This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] binlijin edited a comment on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
binlijin edited a comment on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-561037779 @sandeepvinayak try use the following import: import java.util.List; import org.apache.hadoop.hbase.HBaseInterfaceAudience; import org.apache.hadoop.hbase.wal.WAL; import org.apache.yetus.audience.InterfaceAudience; import org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting; This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
Apache-HBase commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-560961101 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 49s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | checkstyle | 1m 30s | master passed | | +1 :green_heart: | shadedjars | 5m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | | +0 :ok: | spotbugs | 4m 36s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 35s | master passed | | -0 :warning: | patch | 4m 43s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 23s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | -1 :x: | checkstyle | 1m 29s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 59s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 6s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | | +1 :green_heart: | findbugs | 4m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 162m 17s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 223m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/837 | | JIRA Issue | HBASE-23309 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 80d6c2f2ae21 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-837/out/precommit/personality/provided.sh | | git revision | master / 97e0107000 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/6/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/6/testReport/ | | Max. process+thread count | 4578 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/6/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 merged pull request #888: HBASE-22096 /storeFile.jsp shows CorruptHFileException when the store…
brfrn169 merged pull request #888: HBASE-22096 /storeFile.jsp shows CorruptHFileException when the store… URL: https://github.com/apache/hbase/pull/888 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on issue #888: HBASE-22096 /storeFile.jsp shows CorruptHFileException when the store…
brfrn169 commented on issue #888: HBASE-22096 /storeFile.jsp shows CorruptHFileException when the store… URL: https://github.com/apache/hbase/pull/888#issuecomment-561018427 Merged. Thank you for reviewing this! @binlijin This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986668#comment-16986668
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #198 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/198/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/198//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/198//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/198//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986505#comment-16986505
]
HBase QA commented on HBASE-23309:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
36s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
35s{color} | {color:green} master passed {color} |
| {color:orange}-0{color} | {color:orange} patch {color} | {color:orange} 4m
43s{color} | {color:orange} Used diff version of patch file. Binary files and
potentially other changes not applied. Please rebase and squash commits if
necessary. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
29s{color} | {color:red} hbase-server: The patch generated 1 new + 0 unchanged
- 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
59s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 6s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}162m
17s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}223m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.4 Server=19.03.4 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/837 |
| JIRA Issue | HBASE-23309 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 80d6c2f2ae21 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12
10:36:11 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-23066) Allow cache on write during compactions when prefetching is enabled
[ https://issues.apache.org/jira/browse/HBASE-23066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986567#comment-16986567 ] Anoop Sam John commented on HBASE-23066: bq.have a CACHE_BLOCKS_ON_COMPACTION as part of ColumnFamilyDescriptor. The latest patch what Ram published yesterday is having this aspect also. We were discussing it offline. The allow cache on write for compacted files should be a global config as well as one can override it at Table or CF levels. On HTD or HCD level, user can do setConfigure with same config name and override the global value. > Allow cache on write during compactions when prefetching is enabled > --- > > Key: HBASE-23066 > URL: https://issues.apache.org/jira/browse/HBASE-23066 > Project: HBase > Issue Type: Improvement > Components: Compaction, regionserver >Affects Versions: 1.4.10 >Reporter: Jacob LeBlanc >Assignee: Jacob LeBlanc >Priority: Minor > Fix For: 2.3.0, 1.6.0 > > Attachments: HBASE-23066.patch, performance_results.png, > prefetchCompactedBlocksOnWrite.patch > > > In cases where users care a lot about read performance for tables that are > small enough to fit into a cache (or the cache is large enough), > prefetchOnOpen can be enabled to make the entire table available in cache > after the initial region opening is completed. Any new data can also be > guaranteed to be in cache with the cacheBlocksOnWrite setting. > However, the missing piece is when all blocks are evicted after a compaction. > We found very poor performance after compactions for tables under heavy read > load and a slower backing filesystem (S3). After a compaction the prefetching > threads need to compete with threads servicing read requests and get > constantly blocked as a result. > This is a proposal to introduce a new cache configuration option that would > cache blocks on write during compaction for any column family that has > prefetch enabled. This would virtually guarantee all blocks are kept in cache > after the initial prefetch on open is completed allowing for guaranteed > steady read performance despite a slow backing file system. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] infraio merged pull request #881: HBASE-23345 Duplicate code about deciding whether a table need replicate
infraio merged pull request #881: HBASE-23345 Duplicate code about deciding whether a table need replicate URL: https://github.com/apache/hbase/pull/881 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23352) Allow chaos monkeys to access cmd line params, and improve FillDiskCommandAction
[
https://issues.apache.org/jira/browse/HBASE-23352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986463#comment-16986463
]
Hudson commented on HBASE-23352:
Results for branch branch-2.2
[build #711 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/711/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/711//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/711//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/711//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Allow chaos monkeys to access cmd line params, and improve
> FillDiskCommandAction
>
>
> Key: HBASE-23352
> URL: https://issues.apache.org/jira/browse/HBASE-23352
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Affects Versions: 2.2.2
>Reporter: Szabolcs Bukros
>Assignee: Szabolcs Bukros
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> When integration tests are run through hbase cli the properties passed as cmd
> line params does not reach the chaos monkies. It is possible to define a
> property file, but it would be more flexible if we could also pick up
> properties from the command line.
> Also I would like to improve FillDiskCommandAction, to stop the remote
> process if the call times out before it could have finished or was run
> without a size parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23349) Expose max refCount among all compacted store files of a region
[
https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986530#comment-16986530
]
HBase QA commented on HBASE-23349:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} prototool {color} | {color:blue} 0m
0s{color} | {color:blue} prototool was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 7m
29s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 17m
56s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 30m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 7m
50s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 28s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
11m 47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 31m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red}
[jira] [Commented] (HBASE-23298) Refactor LogRecoveredEditsOutputSink and BoundedLogWriterCreationOutputSink
[
https://issues.apache.org/jira/browse/HBASE-23298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986556#comment-16986556
]
Hudson commented on HBASE-23298:
Results for branch branch-2
[build #2372 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Refactor LogRecoveredEditsOutputSink and BoundedLogWriterCreationOutputSink
> ---
>
> Key: HBASE-23298
> URL: https://issues.apache.org/jira/browse/HBASE-23298
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-23298.branch-2.001.patch
>
>
> I do some refactor work in HBASE-23286. Move them to a new issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23352) Allow chaos monkeys to access cmd line params, and improve FillDiskCommandAction
[
https://issues.apache.org/jira/browse/HBASE-23352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986554#comment-16986554
]
Hudson commented on HBASE-23352:
Results for branch branch-2
[build #2372 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Allow chaos monkeys to access cmd line params, and improve
> FillDiskCommandAction
>
>
> Key: HBASE-23352
> URL: https://issues.apache.org/jira/browse/HBASE-23352
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Affects Versions: 2.2.2
>Reporter: Szabolcs Bukros
>Assignee: Szabolcs Bukros
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> When integration tests are run through hbase cli the properties passed as cmd
> line params does not reach the chaos monkies. It is possible to define a
> property file, but it would be more flexible if we could also pick up
> properties from the command line.
> Also I would like to improve FillDiskCommandAction, to stop the remote
> process if the call times out before it could have finished or was run
> without a size parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23335) Improve cost functions array copy in StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-23335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986555#comment-16986555
]
Hudson commented on HBASE-23335:
Results for branch branch-2
[build #2372 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2372//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Improve cost functions array copy in StochasticLoadBalancer
> ---
>
> Key: HBASE-23335
> URL: https://issues.apache.org/jira/browse/HBASE-23335
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> While copying temp array of cost functions to current cost functions array,
> instead of copying each element manually, we can use System.arraycopy() for
> better performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23356) When construct StoreScanner throw exceptions it is possible to left some KeyValueScanner not closed.
Lijin Bin created HBASE-23356: - Summary: When construct StoreScanner throw exceptions it is possible to left some KeyValueScanner not closed. Key: HBASE-23356 URL: https://issues.apache.org/jira/browse/HBASE-23356 Project: HBase Issue Type: Bug Reporter: Lijin Bin -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23355) Add a conf key to decide which operations can trigger prefetch
[
https://issues.apache.org/jira/browse/HBASE-23355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986570#comment-16986570
]
Anoop Sam John commented on HBASE-23355:
I believe the prefetch feature was added with intent to fetch it on Region open
only. When we go through that old Jira it look so. We have cache on write
config which will govern caching on flush. (This is kind of prefetch only). The
new Jira will do the same for compaction also. So dont think we need new
prefetchOnFlush also. This might be an overkill IMHO.
> Add a conf key to decide which operations can trigger prefetch
> --
>
> Key: HBASE-23355
> URL: https://issues.apache.org/jira/browse/HBASE-23355
> Project: HBase
> Issue Type: Improvement
>Reporter: chenxu
>Assignee: chenxu
>Priority: Major
>
> Copy the comment from anoop under
> https://issues.apache.org/jira/browse/HBASE-23066?focusedCommentId=16984140=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16984140
> {quote}On a side note, (Not related to this issue) when we have cache on
> write ON as well as prefetch also On, do we do the caching part for the
> flushed files twice? When it is written, its already been added to cache.
> Later as part of HFile reader open, the prefetch threads will again do a read
> and add to cache!
> {quote}
> It's better to have a conf key to decide which operations(flush, compaction,
> regionOpen, bulkload) can trigger the prefetch. In this way, if cacheonwrite
> is enabled, we can exclude the flush and compaction operations when doing
> prefetch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] binlijin opened a new pull request #891: HBASE-23356 When construct StoreScanner throw exceptions it is possib…
binlijin opened a new pull request #891: HBASE-23356 When construct StoreScanner throw exceptions it is possib… URL: https://github.com/apache/hbase/pull/891 …le to left some KeyValueScanner not closed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (HBASE-23356) When construct StoreScanner throw exceptions it is possible to left some KeyValueScanner not closed.
[ https://issues.apache.org/jira/browse/HBASE-23356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijin Bin reassigned HBASE-23356: - Assignee: Lijin Bin > When construct StoreScanner throw exceptions it is possible to left some > KeyValueScanner not closed. > > > Key: HBASE-23356 > URL: https://issues.apache.org/jira/browse/HBASE-23356 > Project: HBase > Issue Type: Bug >Reporter: Lijin Bin >Assignee: Lijin Bin >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23356) When construct StoreScanner throw exceptions it is possible to left some KeyValueScanner not closed.
[
https://issues.apache.org/jira/browse/HBASE-23356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lijin Bin updated HBASE-23356:
--
Description:
{code}
try {
// Pass columns to try to filter out unnecessary StoreFiles.
List scanners = selectScannersFrom(store,
store.getScanners(cacheBlocks, scanUsePread, false, matcher,
scan.getStartRow(),
scan.includeStartRow(), scan.getStopRow(), scan.includeStopRow(),
this.readPt));
// Seek all scanners to the start of the Row (or if the exact matching row
// key does not exist, then to the start of the next matching Row).
// Always check bloom filter to optimize the top row seek for delete
// family marker.
seekScanners(scanners, matcher.getStartKey(), explicitColumnQuery &&
lazySeekEnabledGlobally,
parallelSeekEnabled);
..
} catch (IOException e) {
// remove us from the HStore#changedReaderObservers here or we'll have no
chance to
// and might cause memory leak
store.deleteChangedReaderObserver(this);
throw e;
}
{code}
Like the above, if the seekScanners(scanners,...) throws exception, the
scanners do not closed, we need to close the scanners.
> When construct StoreScanner throw exceptions it is possible to left some
> KeyValueScanner not closed.
>
>
> Key: HBASE-23356
> URL: https://issues.apache.org/jira/browse/HBASE-23356
> Project: HBase
> Issue Type: Bug
>Reporter: Lijin Bin
>Assignee: Lijin Bin
>Priority: Major
>
> {code}
> try {
> // Pass columns to try to filter out unnecessary StoreFiles.
> List scanners = selectScannersFrom(store,
> store.getScanners(cacheBlocks, scanUsePread, false, matcher,
> scan.getStartRow(),
> scan.includeStartRow(), scan.getStopRow(), scan.includeStopRow(),
> this.readPt));
> // Seek all scanners to the start of the Row (or if the exact matching
> row
> // key does not exist, then to the start of the next matching Row).
> // Always check bloom filter to optimize the top row seek for delete
> // family marker.
> seekScanners(scanners, matcher.getStartKey(), explicitColumnQuery &&
> lazySeekEnabledGlobally,
> parallelSeekEnabled);
>
> ..
> } catch (IOException e) {
> // remove us from the HStore#changedReaderObservers here or we'll have
> no chance to
> // and might cause memory leak
> store.deleteChangedReaderObserver(this);
> throw e;
> }
> {code}
> Like the above, if the seekScanners(scanners,...) throws exception, the
> scanners do not closed, we need to close the scanners.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23345) Table need to replication unless all of cfs are excluded
[
https://issues.apache.org/jira/browse/HBASE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986625#comment-16986625
]
Guanghao Zhang commented on HBASE-23345:
Need to change the Jira state to "Patch Available" to trigger HBase QA.
> Table need to replication unless all of cfs are excluded
>
>
> Key: HBASE-23345
> URL: https://issues.apache.org/jira/browse/HBASE-23345
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
> Attachments: HBASE-23345.branch-2.001.patch,
> HBASE-23345.branch-2.002.patch, HBASE-23345.branch-2.002.patch
>
>
> There is duplicate code in ReplicationUtils.contains and
> ReplicationPeerConfig.needToReplicate about deciding whether a table need
> replicate to the peer cluster.
> And the implementation of ReplicationPeerConfig.needToReplicate has a bug, it
> return false when replicateAllUserTables flag is true and excludeTableCFsMap
> contains not all of cfs.
>
> We should copy the code from ReplicationUtils.contains to
> ReplicationPeerConfig.needToReplicate, and delete
> ReplicationUtils.contains.Because ReplicationUtils is from module
> replication, ReplicationPeerConfig is from module client, and module
> replication depends on module client.
>
> In the following code, ReplicationPeerConfig.needToReplicate return false,
> when replicateAllUserTables flag is true and excludeTableCFsMap contains not
> all of cfs.
> {code:java}
> public boolean needToReplicate(TableName table) {
> if (replicateAllUserTables) {
> ..
> if (excludeTableCFsMap != null && excludeTableCFsMap.containsKey(table)) {
> return false;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23345) Table need to replication unless all of cfs are excluded
[
https://issues.apache.org/jira/browse/HBASE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-23345:
---
Attachment: HBASE-23345.branch-2.002.patch
> Table need to replication unless all of cfs are excluded
>
>
> Key: HBASE-23345
> URL: https://issues.apache.org/jira/browse/HBASE-23345
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
> Attachments: HBASE-23345.branch-2.001.patch,
> HBASE-23345.branch-2.002.patch, HBASE-23345.branch-2.002.patch
>
>
> There is duplicate code in ReplicationUtils.contains and
> ReplicationPeerConfig.needToReplicate about deciding whether a table need
> replicate to the peer cluster.
> And the implementation of ReplicationPeerConfig.needToReplicate has a bug, it
> return false when replicateAllUserTables flag is true and excludeTableCFsMap
> contains not all of cfs.
>
> We should copy the code from ReplicationUtils.contains to
> ReplicationPeerConfig.needToReplicate, and delete
> ReplicationUtils.contains.Because ReplicationUtils is from module
> replication, ReplicationPeerConfig is from module client, and module
> replication depends on module client.
>
> In the following code, ReplicationPeerConfig.needToReplicate return false,
> when replicateAllUserTables flag is true and excludeTableCFsMap contains not
> all of cfs.
> {code:java}
> public boolean needToReplicate(TableName table) {
> if (replicateAllUserTables) {
> ..
> if (excludeTableCFsMap != null && excludeTableCFsMap.containsKey(table)) {
> return false;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22096) /storeFile.jsp shows CorruptHFileException when the storeFile is a reference file
[ https://issues.apache.org/jira/browse/HBASE-22096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-22096: - Fix Version/s: 2.2.3 2.1.8 2.3.0 > /storeFile.jsp shows CorruptHFileException when the storeFile is a reference > file > - > > Key: HBASE-22096 > URL: https://issues.apache.org/jira/browse/HBASE-22096 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.1.8, 2.2.3 > > Attachments: screanshot.png > > > When the storeFile is a reference file, /storeFile.jsp for the storeFile > shows the following error: > !screanshot.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #891: HBASE-23356 When construct StoreScanner throw exceptions it is possib…
Apache-HBase commented on issue #891: HBASE-23356 When construct StoreScanner throw exceptions it is possib… URL: https://github.com/apache/hbase/pull/891#issuecomment-561028877 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 27s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | checkstyle | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 4m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | master passed | | +0 :ok: | spotbugs | 4m 13s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 10s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 56s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | checkstyle | 1m 17s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 42s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | | +1 :green_heart: | findbugs | 4m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 164m 32s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 221m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-891/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/891 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d011a4d24352 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-891/out/precommit/personality/provided.sh | | git revision | master / 97e0107000 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-891/1/testReport/ | | Max. process+thread count | 4283 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-891/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #890: HBASE-23349 : Expose max refCount among all compacted store files and…
Apache-HBase commented on issue #890: HBASE-23349 : Expose max refCount among all compacted store files and… URL: https://github.com/apache/hbase/pull/890#issuecomment-560987082 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 40s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 15s | master passed | | +1 :green_heart: | compile | 3m 0s | master passed | | +1 :green_heart: | checkstyle | 2m 31s | master passed | | +0 :ok: | refguide | 5m 29s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 4m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 4s | master passed | | +0 :ok: | spotbugs | 3m 42s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 24m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 39s | the patch passed | | +1 :green_heart: | compile | 3m 23s | the patch passed | | +1 :green_heart: | cc | 3m 23s | the patch passed | | +1 :green_heart: | javac | 3m 23s | the patch passed | | +1 :green_heart: | checkstyle | 2m 47s | the patch passed | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 5m 37s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 4m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 8s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 8m 39s | the patch passed | | +1 :green_heart: | javadoc | 5m 11s | the patch passed | | +1 :green_heart: | findbugs | 25m 1s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 221m 19s | root in the patch failed. | | +1 :green_heart: | asflicense | 5m 1s | The patch does not generate ASF License warnings. | | | | 363m 28s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.snapshot.TestExportSnapshotNoCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/890 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml cc hbaseprotoc prototool | | uname | Linux 614ef3bacda9 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-890/out/precommit/personality/provided.sh | | git revision | master / 97e0107000 | | Default Java | 1.8.0_181 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/2/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/2/artifact/out/patch-site/book.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/2/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/2/testReport/ | | Max. process+thread count | 5578 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-hadoop2-compat hbase-protocol hbase-client hbase-server . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus
[jira] [Updated] (HBASE-22096) /storeFile.jsp shows CorruptHFileException when the storeFile is a reference file
[ https://issues.apache.org/jira/browse/HBASE-22096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-22096: - Fix Version/s: (was: 2.1.8) 2.1.9 > /storeFile.jsp shows CorruptHFileException when the storeFile is a reference > file > - > > Key: HBASE-22096 > URL: https://issues.apache.org/jira/browse/HBASE-22096 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9 > > Attachments: screanshot.png > > > When the storeFile is a reference file, /storeFile.jsp for the storeFile > shows the following error: > !screanshot.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-561025397 @wchevreuil There is only checkstyle warning and to me that doesn't look right. I am not sure what's wrong with the order of imports now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sandeepvinayak edited a comment on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak edited a comment on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-561025397 @wchevreuil There is only one checkstyle warning and to me that doesn't look right. I am not sure what's wrong with the order of imports now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23349) Expose max refCount among all compacted store files of a region
[ https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-23349: - Attachment: HBASE-23349.master.002.patch > Expose max refCount among all compacted store files of a region > --- > > Key: HBASE-23349 > URL: https://issues.apache.org/jira/browse/HBASE-23349 > Project: HBase > Issue Type: Task >Affects Versions: 3.0.0, 2.3.0, 1.6.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-23349.master.000.patch, > HBASE-23349.master.001.patch, HBASE-23349.master.002.patch > > > We should expose a region level metric that represents max refCount among > refCounts of all compacted store files under the region. For successful > archival of compacted store files, it is important for this metric count to > be 0 eventually if not immediately. If it is >0 for a considerably high > amount of time, it indicates some issue i.e. reader refCount leak on some > compacted store files and in such case, archival would not be successful. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #881: HBASE-23345 Duplicate code about deciding whether a table need replicate
Apache-HBase commented on issue #881: HBASE-23345 Duplicate code about deciding whether a table need replicate URL: https://github.com/apache/hbase/pull/881#issuecomment-560509935 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 59s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 13s | master passed | | +1 :green_heart: | compile | 1m 40s | master passed | | +1 :green_heart: | checkstyle | 2m 4s | master passed | | +1 :green_heart: | shadedjars | 4m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 15s | master passed | | +0 :ok: | spotbugs | 4m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 38s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 51s | the patch passed | | +1 :green_heart: | compile | 1m 40s | the patch passed | | +1 :green_heart: | javac | 1m 40s | the patch passed | | +1 :green_heart: | checkstyle | 2m 6s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 37s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 15s | the patch passed | | +1 :green_heart: | findbugs | 6m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 53s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 28s | hbase-replication in the patch passed. | | +1 :green_heart: | unit | 165m 43s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 1m 38s | The patch does not generate ASF License warnings. | | | | 236m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-881/9/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/881 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 8762ec84f65d 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-881/out/precommit/personality/provided.sh | | git revision | master / 97e0107000 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-881/9/testReport/ | | Max. process+thread count | 4644 (vs. ulimit of 1) | | modules | C: hbase-client hbase-replication hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-881/9/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23066) Allow cache on write during compactions when prefetching is enabled
[
https://issues.apache.org/jira/browse/HBASE-23066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986152#comment-16986152
]
Jacob LeBlanc commented on HBASE-23066:
---
Apologies for the delayed response on my part.
Just as an FYI, we manually deployed this patch (my original version) to our
production environment and have been running it with good results for about a
month and a half.
{quote}With this new config, what we do is write to cache along with the HFile
create itself. Blocks are added to cache as and when it is written to the
HFile. So its aggressive. Ya it helps to make the new File data available from
time 0 itself. The concern is this in a way demands 2x cache size. Because the
compacting files data might be already there in the cache. While the new file
write, those old files are still valid. The new one is not even committed by
the RS.
{quote}
Without evictblocksonclose enabled (it looks like it is disabled by default)
then wouldn't the old file data still be in cache even after compaction is
finished? Granted once compaction is done it will no longer be accessed, age
out, and be evicted if necessary but the same amount of data is read into the
cache both with and without this new setting, is it not? When prefetching is
enabled the only difference is the caching of the new file is done a little bit
earlier, but other than that it seems caching requirements are the same. I'm
not sure I understand why 2x cache size is needed - perhaps I am missing
something. Having eviceblocksonclose enabled does change things and means you
would need 2x cache size compared to normal as you change the ordering of
caching/evicting.
{quote}Also IMHO no need to check that based on whether prefetch is on or not!
Make this conf name and doc clear what it is doing and what is the size
expectations.
{quote}
Coming from the perspective of my organization's requirements - this would not
work well for us as we only want data to be cached on compaction for tables
where prefetching is enabled:
The clear intention of enabling prefetching on a table is to keep as much data
in the read cache as possible to ensure consistently fast reading, but without
this configuration there are consistently huge drops in read performance
whenever compaction is done because large parts of the table are essentially
dropped from the cache (actually the pre-compaction data is still there unless
evictblocksonclose is enabled, but the pre-compaction data is for the old file
names which will never be accessed again after compaction is finished so it's
the same as dropping the data). This configuration is to mitigate that effect
to better achieve read performance sought by prefetching. The intention is
*not* just to cache everything that gets compacted.
So caching all compacted data on all tables does not meet this requirement and
in fact would cause problems if it were to be used. In our use cases we have
several tables where we write and compact a lot but where we don't want to
prefetch those tables into our cache. Caching all blocks on compaction would
cause big problems where we'd evict data we care about in favor of data we will
never/rarely read.
An alternative to having this setting contingent on prefetching would be to
have a CACHE_BLOCKS_ON_COMPACTION as part of ColumnFamilyDescriptor. Then we
could choose to turn it on for the same CFs where we also have prefetching.
This seems like a bigger code/documentation change, whereas my original
intention on this patch was to keep it small and focused for the only use case
I could think of (why else would someone want to cache blocks during compaction
except if they were prefetching?). But if a per-column family setting is
preferred, then I could try making those changes.
I welcome input from you experts. Thanks!
> Allow cache on write during compactions when prefetching is enabled
> ---
>
> Key: HBASE-23066
> URL: https://issues.apache.org/jira/browse/HBASE-23066
> Project: HBase
> Issue Type: Improvement
> Components: Compaction, regionserver
>Affects Versions: 1.4.10
>Reporter: Jacob LeBlanc
>Assignee: Jacob LeBlanc
>Priority: Minor
> Fix For: 2.3.0, 1.6.0
>
> Attachments: HBASE-23066.patch, performance_results.png,
> prefetchCompactedBlocksOnWrite.patch
>
>
> In cases where users care a lot about read performance for tables that are
> small enough to fit into a cache (or the cache is large enough),
> prefetchOnOpen can be enabled to make the entire table available in cache
> after the initial region opening is completed. Any new data can also be
> guaranteed to be in cache with the cacheBlocksOnWrite setting.
> However, the missing piece is when all blocks are evicted after a
[jira] [Updated] (HBASE-23298) Refactor LogRecoveredEditsOutputSink and BoundedLogWriterCreationOutputSink
[ https://issues.apache.org/jira/browse/HBASE-23298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-23298: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to branch-2 and master. Thanks [~stack] and [~zhangduo] for reviewing. > Refactor LogRecoveredEditsOutputSink and BoundedLogWriterCreationOutputSink > --- > > Key: HBASE-23298 > URL: https://issues.apache.org/jira/browse/HBASE-23298 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-23298.branch-2.001.patch > > > I do some refactor work in HBASE-23286. Move them to a new issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985926#comment-16985926
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #197 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/197/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/197//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/197//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/197//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23298) Refactor LogRecoveredEditsOutputSink and BoundedLogWriterCreationOutputSink
[
https://issues.apache.org/jira/browse/HBASE-23298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985950#comment-16985950
]
HBase QA commented on HBASE-23298:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
22s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
7s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
5s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 13s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}163m
44s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}216m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1055/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23298 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12987265/HBASE-23298.branch-2.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8a93070a5897 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / 399fc8d098 |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/1055/testReport/ |
| Max. process+thread count | 4470 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output
[GitHub] [hbase] Apache-HBase commented on issue #888: HBASE-22096 /storeFile.jsp shows CorruptHFileException when the store…
Apache-HBase commented on issue #888: HBASE-22096 /storeFile.jsp shows CorruptHFileException when the store… URL: https://github.com/apache/hbase/pull/888#issuecomment-560298205 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 52s | master passed | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 163m 22s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 177m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-888/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/888 | | Optional Tests | dupname asflicense javac javadoc unit | | uname | Linux 4262a99d8c55 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-888/out/precommit/personality/provided.sh | | git revision | master / ad607a1d76 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-888/1/testReport/ | | Max. process+thread count | 4715 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-888/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bsglz commented on issue #873: HBASE-23334 The table-lock node of zk is not needed since HBASE-16786
bsglz commented on issue #873: HBASE-23334 The table-lock node of zk is not needed since HBASE-16786 URL: https://github.com/apache/hbase/pull/873#issuecomment-560346729 @guangxuCheng Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bsglz commented on issue #764: HBASE-23223 Support the offsetLock of bucketCache to use strong ref
bsglz commented on issue #764: HBASE-23223 Support the offsetLock of bucketCache to use strong ref URL: https://github.com/apache/hbase/pull/764#issuecomment-560347023 @wchevreuil Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ramkrish86 opened a new pull request #889: HBASE-23066 - Allow cache on write during compactions when prefetchin…
ramkrish86 opened a new pull request #889: HBASE-23066 - Allow cache on write during compactions when prefetchin… URL: https://github.com/apache/hbase/pull/889 I have updated the PR (created a new one) with the comments. Now the config is not dependent on the prefetch block option. Just that we can use it as a toggle config. So once cache on compaction is enabled, the blocks are cached. The threshold level change will do in another subtask. (https://issues.apache.org/jira/browse/HBASE-23350) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #849: HBASE-23320 Upgrade surefire plugin to 3.0.0-M4
Apache-HBase commented on issue #849: HBASE-23320 Upgrade surefire plugin to 3.0.0-M4 URL: https://github.com/apache/hbase/pull/849#issuecomment-560373981 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 27s | master passed | | +1 :green_heart: | compile | 3m 9s | master passed | | +1 :green_heart: | shadedjars | 4m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 58s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 21s | the patch passed | | +1 :green_heart: | compile | 3m 5s | the patch passed | | +1 :green_heart: | javac | 3m 5s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 4m 43s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 59s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 2m 54s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 327m 17s | root in the patch failed. | | +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. | | | | 387m 26s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestSyncReplicationStandbyKillMaster | | | hadoop.hbase.client.TestFromClientSide3 | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.client.TestFromClientSide | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-849/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/849 | | Optional Tests | dupname asflicense javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux c14ed3b350df 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-849/out/precommit/personality/provided.sh | | git revision | master / ad607a1d76 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-849/6/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-849/6/testReport/ | | Max. process+thread count | 4816 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-849/6/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on issue #887: HBASE-23337 Release scripts should rely on maven for deploy.
busbey commented on issue #887: HBASE-23337 Release scripts should rely on maven for deploy. URL: https://github.com/apache/hbase/pull/887#issuecomment-560374385 Yeah there is one already This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #889: HBASE-23066 - Allow cache on write during compactions when prefetchin…
Apache-HBase commented on issue #889: HBASE-23066 - Allow cache on write during compactions when prefetchin… URL: https://github.com/apache/hbase/pull/889#issuecomment-560439974 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 30s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | checkstyle | 1m 22s | master passed | | +1 :green_heart: | shadedjars | 5m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | | +0 :ok: | spotbugs | 5m 4s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 2s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 58s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | -1 :x: | checkstyle | 1m 18s | hbase-server: The patch generated 3 new + 46 unchanged - 0 fixed = 49 total (was 46) | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 43s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 47s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | | +1 :green_heart: | findbugs | 4m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 171m 38s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 230m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-889/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/889 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 46e98e00faee 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-889/out/precommit/personality/provided.sh | | git revision | master / ad607a1d76 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-889/1/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-889/1/testReport/ | | Max. process+thread count | 4555 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-889/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23349) Expose max refCount among all compacted store files of a region
[ https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-23349: - Attachment: HBASE-23349.master.001.patch > Expose max refCount among all compacted store files of a region > --- > > Key: HBASE-23349 > URL: https://issues.apache.org/jira/browse/HBASE-23349 > Project: HBase > Issue Type: Task >Affects Versions: 3.0.0, 2.3.0, 1.6.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-23349.master.000.patch, > HBASE-23349.master.001.patch > > > We should expose a region level metric that represents max refCount among > refCounts of all compacted store files under the region. For successful > archival of compacted store files, it is important for this metric count to > be 0 eventually if not immediately. If it is >0 for a considerably high > amount of time, it indicates some issue i.e. reader refCount leak on some > compacted store files and in such case, archival would not be successful. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani opened a new pull request #890: HBASE-23349 : Expose max refCount among all compacted store files and…
virajjasani opened a new pull request #890: HBASE-23349 : Expose max refCount among all compacted store files and… URL: https://github.com/apache/hbase/pull/890 … reopen region for threshold value This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23349) Expose max refCount among all compacted store files of a region
[
https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986034#comment-16986034
]
Viraj Jasani commented on HBASE-23349:
--
{quote}Because the metric report the max ref count across all the store files
in the Region. This list will not include the compacted away files.
{quote}
Agree, because of this, our auto-reopen of region is not happening based on max
refCount among compacted store files.
Sure, let me update the patch and include this change as part of this Jira,
also I have linked the previous issues.
FYI [~apurtell]
Thanks
> Expose max refCount among all compacted store files of a region
> ---
>
> Key: HBASE-23349
> URL: https://issues.apache.org/jira/browse/HBASE-23349
> Project: HBase
> Issue Type: Task
>Affects Versions: 3.0.0, 2.3.0, 1.6.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
> Attachments: HBASE-23349.master.000.patch
>
>
> We should expose a region level metric that represents max refCount among
> refCounts of all compacted store files under the region. For successful
> archival of compacted store files, it is important for this metric count to
> be 0 eventually if not immediately. If it is >0 for a considerably high
> amount of time, it indicates some issue i.e. reader refCount leak on some
> compacted store files and in such case, archival would not be successful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] busbey merged pull request #887: HBASE-23337 Release scripts should rely on maven for deploy.
busbey merged pull request #887: HBASE-23337 Release scripts should rely on maven for deploy. URL: https://github.com/apache/hbase/pull/887 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23342) Handle NPE while closing compressingStream
[
https://issues.apache.org/jira/browse/HBASE-23342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986076#comment-16986076
]
Hudson commented on HBASE-23342:
Results for branch master
[build #1554 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1554/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Handle NPE while closing compressingStream
> --
>
> Key: HBASE-23342
> URL: https://issues.apache.org/jira/browse/HBASE-23342
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0, 2.2.3
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Trivial
> Fix For: 3.0.0
>
>
> EncodedDataBlock.getCompressedSize() can produce NPE while closing
> compressingStream if createCompressionStream() throws IOException.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23335) Improve cost functions array copy in StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-23335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986078#comment-16986078
]
Hudson commented on HBASE-23335:
Results for branch master
[build #1554 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1554/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Improve cost functions array copy in StochasticLoadBalancer
> ---
>
> Key: HBASE-23335
> URL: https://issues.apache.org/jira/browse/HBASE-23335
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> While copying temp array of cost functions to current cost functions array,
> instead of copying each element manually, we can use System.arraycopy() for
> better performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23352) Allow chaos monkeys to access cmd line params, and improve FillDiskCommandAction
[
https://issues.apache.org/jira/browse/HBASE-23352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986075#comment-16986075
]
Hudson commented on HBASE-23352:
Results for branch master
[build #1554 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1554/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Allow chaos monkeys to access cmd line params, and improve
> FillDiskCommandAction
>
>
> Key: HBASE-23352
> URL: https://issues.apache.org/jira/browse/HBASE-23352
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Affects Versions: 2.2.2
>Reporter: Szabolcs Bukros
>Assignee: Szabolcs Bukros
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> When integration tests are run through hbase cli the properties passed as cmd
> line params does not reach the chaos monkies. It is possible to define a
> property file, but it would be more flexible if we could also pick up
> properties from the command line.
> Also I would like to improve FillDiskCommandAction, to stop the remote
> process if the call times out before it could have finished or was run
> without a size parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23298) Refactor LogRecoveredEditsOutputSink and BoundedLogWriterCreationOutputSink
[
https://issues.apache.org/jira/browse/HBASE-23298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986077#comment-16986077
]
Hudson commented on HBASE-23298:
Results for branch master
[build #1554 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1554/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1554//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Refactor LogRecoveredEditsOutputSink and BoundedLogWriterCreationOutputSink
> ---
>
> Key: HBASE-23298
> URL: https://issues.apache.org/jira/browse/HBASE-23298
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-23298.branch-2.001.patch
>
>
> I do some refactor work in HBASE-23286. Move them to a new issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] busbey commented on a change in pull request #843: HBASE-23303 Add security headers to REST server/info page
busbey commented on a change in pull request #843: HBASE-23303 Add security headers to REST server/info page URL: https://github.com/apache/hbase/pull/843#discussion_r352618440 ## File path: hbase-http/src/main/java/org/apache/hadoop/hbase/http/SecurityHeadersFilter.java ## @@ -0,0 +1,83 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hbase.http; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.HBaseInterfaceAudience; +import org.apache.yetus.audience.InterfaceAudience; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.servlet.Filter; +import javax.servlet.FilterChain; +import javax.servlet.FilterConfig; +import javax.servlet.ServletException; +import javax.servlet.ServletRequest; +import javax.servlet.ServletResponse; +import javax.servlet.http.HttpServletResponse; +import java.io.IOException; +import java.util.HashMap; +import java.util.Map; + +@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.CONFIG) Review comment: LP(Config) is supposed to mean the class name is exposed in user configs. If the user can no longer specify which filters then it'd be fine to make it Private. Otherwise we'd need a layer of indirection so that folks don't give classnames. I think it'd also be fine for this to go to shipping branches of we expressly checked for the old classname in configs and did the appropriate thing to put the same protection in place. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #890: HBASE-23349 : Expose max refCount among all compacted store files and…
Apache-HBase commented on issue #890: HBASE-23349 : Expose max refCount among all compacted store files and… URL: https://github.com/apache/hbase/pull/890#issuecomment-560657127 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 41s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 12s | master passed | | +1 :green_heart: | compile | 3m 3s | master passed | | +1 :green_heart: | checkstyle | 2m 44s | master passed | | +1 :green_heart: | shadedjars | 4m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 3s | master passed | | +0 :ok: | spotbugs | 4m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 10m 4s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 55s | the patch passed | | +1 :green_heart: | compile | 3m 6s | the patch passed | | +1 :green_heart: | cc | 3m 6s | the patch passed | | +1 :green_heart: | javac | 3m 6s | the patch passed | | +1 :green_heart: | checkstyle | 2m 41s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 40s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 3m 1s | the patch passed | | +1 :green_heart: | javadoc | 2m 0s | the patch passed | | +1 :green_heart: | findbugs | 10m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 42s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 34s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 0m 39s | hbase-hadoop2-compat in the patch passed. | | +1 :green_heart: | unit | 0m 27s | hbase-protocol in the patch passed. | | +1 :green_heart: | unit | 1m 53s | hbase-client in the patch passed. | | -1 :x: | unit | 311m 9s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 2m 52s | The patch does not generate ASF License warnings. | | | | 402m 36s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.master.TestSplitWALManager | | | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.client.TestAsyncTableBatch | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/890 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 5238e0bff80e 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-890/out/precommit/personality/provided.sh | | git revision | master / 97e0107000 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/1/testReport/ | | Max. process+thread count | 4709 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-hadoop-compat hbase-hadoop2-compat hbase-protocol hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-890/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by |
[jira] [Assigned] (HBASE-22457) Harden the HBase HFile reader reference counting
[ https://issues.apache.org/jira/browse/HBASE-22457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl reassigned HBASE-22457: - Assignee: (was: Lars Hofhansl) > Harden the HBase HFile reader reference counting > > > Key: HBASE-22457 > URL: https://issues.apache.org/jira/browse/HBASE-22457 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl >Priority: Major > Attachments: 22457-random-1.5.txt > > > The problem that any coprocessor hook that replaces a passed scanner without > closing it can cause an incorrect reference count. > This was bad and wrong before of course, but now it has pretty bad > consequences, since an incorrect reference could will prevent HFiles from > being archived indefinitely. > All hooks that are passed a scanner and return a scanner are suspect, since > the returned scanner may or may not close the passed scanner: > * preCompact > * preCompactScannerOpen > * preFlush > * preFlushScannerOpen > * preScannerOpen > * preStoreScannerOpen > * preStoreFileReaderOpen...? (not sure about this one, it could mess with the > reader) > I sampled the Phoenix and also Tephra code, and found a few instances where > this is happening. > And for those I filed issued: TEPHRA-300, PHOENIX-5291 > (We're not using Tephra) > The Phoenix ones should be rare. In our case we are seeing readers with > refCount > 1000. > Perhaps there are other issues, a path where not all exceptions are caught > and scanner is left open that way perhaps. (Generally I am not a fan of > reference counting in complex systems - it's too easy to miss something. But > that's a different discussion. :) ). > Let's brainstorm some way in which we can harden this. > [~ram_krish], [~anoop.hbase], [~apurtell] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on issue #879: HBASE-23343 Thrift Resolve FQDN from SPNEGO principal
saintstack commented on issue #879: HBASE-23343 Thrift Resolve FQDN from SPNEGO principal URL: https://github.com/apache/hbase/pull/879#issuecomment-560576386 Fixed by HBASE-23312 . Closing. Thanks @karthikhw This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sandeepvinayak commented on a change in pull request #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak commented on a change in pull request #837: HBASE-23309: Adding
the flexibility to ChainWalEntryFilter to filter the whole entry if all cells
get filtered
URL: https://github.com/apache/hbase/pull/837#discussion_r352887181
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/CustomChainWALEntryFilter.java
##
@@ -0,0 +1,68 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.replication;
+
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.hbase.wal.WAL;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.yetus.audience.InterfaceAudience;
+
+import java.util.List;
+
+/**
+ * A {@link ChainWALEntryFilter} for providing more flexible options
+ */
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.REPLICATION)
+public class CustomChainWALEntryFilter extends ChainWALEntryFilter {
Review comment:
@wchevreuil I addressed your comments. Can you please review it again?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack closed pull request #879: HBASE-23343 Thrift Resolve FQDN from SPNEGO principal
saintstack closed pull request #879: HBASE-23343 Thrift Resolve FQDN from SPNEGO principal URL: https://github.com/apache/hbase/pull/879 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] karthikhw commented on issue #879: HBASE-23343 Thrift Resolve FQDN from SPNEGO principal
karthikhw commented on issue #879: HBASE-23343 Thrift Resolve FQDN from SPNEGO principal URL: https://github.com/apache/hbase/pull/879#issuecomment-560543075 Thank you very much, @saintstack sir. Sorry if am wrong. I simply thought to put all security-related stuff in one commonplace in HBase SecurityUtil. I think we can close this pull request @saintstack. The same issue addressed via HBASE-23312 which is pushed on the same day. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23349) Expose max refCount among all compacted store files of a region
[
https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986363#comment-16986363
]
HBase QA commented on HBASE-23349:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} prototool {color} | {color:blue} 0m
0s{color} | {color:blue} prototool was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
59s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
29s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 10m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
2s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 13s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
2m 46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
41s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green}
[GitHub] [hbase] SteNicholas closed pull request #679: HBASE-23113 IPC Netty Optimization
SteNicholas closed pull request #679: HBASE-23113 IPC Netty Optimization URL: https://github.com/apache/hbase/pull/679 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
