[jira] [Updated] (HBASE-28498) fix spotless:check errors
[ https://issues.apache.org/jira/browse/HBASE-28498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28498: - Affects Version/s: 2.6.0 > fix spotless:check errors > - > > Key: HBASE-28498 > URL: https://issues.apache.org/jira/browse/HBASE-28498 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.6.0 >Reporter: Nikita Pande >Assignee: Nikita Pande >Priority: Major > Labels: pull-request-available > > I have observed that mvn spotless:check gives error on branch-2.6 > {noformat} > The following files had format violations: > [ERROR] RELEASENOTES.md{noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28485) Re-use ZstdDecompressCtx/ZstdCompressCtx for performance
[ https://issues.apache.org/jira/browse/HBASE-28485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28485: - Resolution: Fixed Status: Resolved (was: Patch Available) Commit applied to branch-2.5+. Thanks [~charlesconnell] for the contribution and [~apurtell] for the review! > Re-use ZstdDecompressCtx/ZstdCompressCtx for performance > > > Key: HBASE-28485 > URL: https://issues.apache.org/jira/browse/HBASE-28485 > Project: HBase > Issue Type: Improvement >Reporter: Charles Connell >Assignee: Charles Connell >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > Attachments: async-prof-flamegraph-cpu_event-1712150670836-cpu.html, > async-prof-pid-1324144-cpu-1.html > > > The zstd documentation > [recommends|https://facebook.github.io/zstd/zstd_manual.html#Chapter4] > re-using context objects when possible, because their creation has some > expense. They can be more cheaply reset than re-created. In > {{ZstdDecompressor}} and {{{}ZstdCompressor{}}}, we create a new context > object for every call to {{decompress()}} and {{{}compress(){}}}. In CPU > profiles I've taken at my company, the constructor of {{ZstdDecompressCtx}} > can sometimes represent 10-25% of the time spent in zstd decompression, which > itself is 5-10% of a RegionServer's total CPU time. Avoiding this performance > penalty won't lead to any massive performance boost, but is a nice little win. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28485) Re-use ZstdDecompressCtx/ZstdCompressCtx for performance
[ https://issues.apache.org/jira/browse/HBASE-28485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28485: - Fix Version/s: 2.7.0 3.0.0-beta-2 2.6.1 2.5.9 > Re-use ZstdDecompressCtx/ZstdCompressCtx for performance > > > Key: HBASE-28485 > URL: https://issues.apache.org/jira/browse/HBASE-28485 > Project: HBase > Issue Type: Improvement >Reporter: Charles Connell >Assignee: Charles Connell >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > Attachments: async-prof-flamegraph-cpu_event-1712150670836-cpu.html, > async-prof-pid-1324144-cpu-1.html > > > The zstd documentation > [recommends|https://facebook.github.io/zstd/zstd_manual.html#Chapter4] > re-using context objects when possible, because their creation has some > expense. They can be more cheaply reset than re-created. In > {{ZstdDecompressor}} and {{{}ZstdCompressor{}}}, we create a new context > object for every call to {{decompress()}} and {{{}compress(){}}}. In CPU > profiles I've taken at my company, the constructor of {{ZstdDecompressCtx}} > can sometimes represent 10-25% of the time spent in zstd decompression, which > itself is 5-10% of a RegionServer's total CPU time. Avoiding this performance > penalty won't lead to any massive performance boost, but is a nice little win. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28485) Re-use ZstdDecompressCtx/ZstdCompressCtx for performance
[ https://issues.apache.org/jira/browse/HBASE-28485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28485: - Fix Version/s: 4.0.0-alpha-1 > Re-use ZstdDecompressCtx/ZstdCompressCtx for performance > > > Key: HBASE-28485 > URL: https://issues.apache.org/jira/browse/HBASE-28485 > Project: HBase > Issue Type: Improvement >Reporter: Charles Connell >Assignee: Charles Connell >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > Attachments: async-prof-flamegraph-cpu_event-1712150670836-cpu.html, > async-prof-pid-1324144-cpu-1.html > > > The zstd documentation > [recommends|https://facebook.github.io/zstd/zstd_manual.html#Chapter4] > re-using context objects when possible, because their creation has some > expense. They can be more cheaply reset than re-created. In > {{ZstdDecompressor}} and {{{}ZstdCompressor{}}}, we create a new context > object for every call to {{decompress()}} and {{{}compress(){}}}. In CPU > profiles I've taken at my company, the constructor of {{ZstdDecompressCtx}} > can sometimes represent 10-25% of the time spent in zstd decompression, which > itself is 5-10% of a RegionServer's total CPU time. Avoiding this performance > penalty won't lead to any massive performance boost, but is a nice little win. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28439) Remove ZooKeeper as a means of creating a client connection
Nick Dimiduk created HBASE-28439: Summary: Remove ZooKeeper as a means of creating a client connection Key: HBASE-28439 URL: https://issues.apache.org/jira/browse/HBASE-28439 Project: HBase Issue Type: Task Components: Client Affects Versions: 4.0.0-alpha-1 Reporter: Nick Dimiduk Following up the discussion and decision around HBASE-23324, we will remove ZooKeeper as a point of entry for client connections. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-23324) Deprecate clients that connect to Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-23324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824532#comment-17824532 ] Nick Dimiduk commented on HBASE-23324: -- Deprecation pushed to branch-2.6+. We can take on removal in another ticket. > Deprecate clients that connect to Zookeeper > --- > > Key: HBASE-23324 > URL: https://issues.apache.org/jira/browse/HBASE-23324 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > Sort out on what schedule we'll deprecated and then delete the old > client-to-zookeeper path. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-23324) Deprecate clients that connect to Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-23324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23324: - Resolution: Fixed Status: Resolved (was: Patch Available) > Deprecate clients that connect to Zookeeper > --- > > Key: HBASE-23324 > URL: https://issues.apache.org/jira/browse/HBASE-23324 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > Sort out on what schedule we'll deprecated and then delete the old > client-to-zookeeper path. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-23324) Deprecate clients that connect to Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-23324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23324: - Affects Version/s: 2.6.0 4.0.0-alpha-1 3.0.0-beta-2 > Deprecate clients that connect to Zookeeper > --- > > Key: HBASE-23324 > URL: https://issues.apache.org/jira/browse/HBASE-23324 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > Sort out on what schedule we'll deprecated and then delete the old > client-to-zookeeper path. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-23324) Deprecate clients that connect to Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-23324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23324: - Fix Version/s: 2.6.0 4.0.0-alpha-1 3.0.0-beta-2 Status: Patch Available (was: Open) > Deprecate clients that connect to Zookeeper > --- > > Key: HBASE-23324 > URL: https://issues.apache.org/jira/browse/HBASE-23324 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > Sort out on what schedule we'll deprecated and then delete the old > client-to-zookeeper path. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-23324) Deprecate clients that connect to Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-23324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23324: - Release Note: ZooKeeper is becoming an internal implementation detail, removed from our public interface. Connecting to a cluster via ZooKeeper quorum will be considered deprecated starting in 2.6. Our default connection mechanism will switch to via RPC in 3.0 And finally we intend to remove the ZooKeeper connection mechanism from client-facing APIs in 4.0. > Deprecate clients that connect to Zookeeper > --- > > Key: HBASE-23324 > URL: https://issues.apache.org/jira/browse/HBASE-23324 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Sort out on what schedule we'll deprecated and then delete the old > client-to-zookeeper path. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-23324) Deprecate clients that connect to Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-23324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-23324: Assignee: Nick Dimiduk > Deprecate clients that connect to Zookeeper > --- > > Key: HBASE-23324 > URL: https://issues.apache.org/jira/browse/HBASE-23324 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Sort out on what schedule we'll deprecated and then delete the old > client-to-zookeeper path. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28379) Upgrade thirdparty dep to 4.1.6
[ https://issues.apache.org/jira/browse/HBASE-28379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28379. -- Resolution: Fixed This is merged. FYI [~bbeaudreault]. > Upgrade thirdparty dep to 4.1.6 > --- > > Key: HBASE-28379 > URL: https://issues.apache.org/jira/browse/HBASE-28379 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > Adopt the next hbase-thirdparty release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28379) Upgrade thirdparty dep to 4.1.6
[ https://issues.apache.org/jira/browse/HBASE-28379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28379: - Fix Version/s: 3.0.0-beta-2 > Upgrade thirdparty dep to 4.1.6 > --- > > Key: HBASE-28379 > URL: https://issues.apache.org/jira/browse/HBASE-28379 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > Adopt the next hbase-thirdparty release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-28379) Upgrade thirdparty dep to 4.1.6
[ https://issues.apache.org/jira/browse/HBASE-28379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-28379 started by Nick Dimiduk. > Upgrade thirdparty dep to 4.1.6 > --- > > Key: HBASE-28379 > URL: https://issues.apache.org/jira/browse/HBASE-28379 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1 > > > Adopt the next hbase-thirdparty release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28379) Upgrade thirdparty dep to 4.1.6
[ https://issues.apache.org/jira/browse/HBASE-28379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28379: - Fix Version/s: 2.6.0 > Upgrade thirdparty dep to 4.1.6 > --- > > Key: HBASE-28379 > URL: https://issues.apache.org/jira/browse/HBASE-28379 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1 > > > Adopt the next hbase-thirdparty release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28379) Upgrade thirdparty dep to 4.1.6
[ https://issues.apache.org/jira/browse/HBASE-28379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28379: - Fix Version/s: 4.0.0-alpha-1 > Upgrade thirdparty dep to 4.1.6 > --- > > Key: HBASE-28379 > URL: https://issues.apache.org/jira/browse/HBASE-28379 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > > Adopt the next hbase-thirdparty release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28407) [thirdparty] Update release instructions
[ https://issues.apache.org/jira/browse/HBASE-28407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28407: - Fix Version/s: thirdparty-4.1.7 Resolution: Fixed Status: Resolved (was: Patch Available) > [thirdparty] Update release instructions > > > Key: HBASE-28407 > URL: https://issues.apache.org/jira/browse/HBASE-28407 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.7 > > > Our release instructions for hbase-thirdparty are out of date. Update them > based on recent experience. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28414) create-release should spotless:apply after making any file changes
[ https://issues.apache.org/jira/browse/HBASE-28414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28414: - Affects Version/s: 4.0.0-alpha-1 Status: Patch Available (was: Open) > create-release should spotless:apply after making any file changes > -- > > Key: HBASE-28414 > URL: https://issues.apache.org/jira/browse/HBASE-28414 > Project: HBase > Issue Type: Task > Components: create-release >Affects Versions: 4.0.0-alpha-1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > Looks like the release notes generator can sometimes leave whitespace in its > changes, as is currently the case on branch-2.5. We should be a little more > careful about this in the create-release scripts. Anything that performs and > commits some change should run spotless:apply over the affected files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28414) create-release should spotless:apply after making any file changes
[ https://issues.apache.org/jira/browse/HBASE-28414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-28414: Assignee: Nick Dimiduk > create-release should spotless:apply after making any file changes > -- > > Key: HBASE-28414 > URL: https://issues.apache.org/jira/browse/HBASE-28414 > Project: HBase > Issue Type: Task > Components: create-release >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > Looks like the release notes generator can sometimes leave whitespace in its > changes, as is currently the case on branch-2.5. We should be a little more > careful about this in the create-release scripts. Anything that performs and > commits some change should run spotless:apply over the affected files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28403) Improve debugging for failures in procedure tests
[ https://issues.apache.org/jira/browse/HBASE-28403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28403: - Resolution: Fixed Status: Resolved (was: Patch Available) > Improve debugging for failures in procedure tests > - > > Key: HBASE-28403 > URL: https://issues.apache.org/jira/browse/HBASE-28403 > Project: HBase > Issue Type: Task > Components: proc-v2, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2, 2.5.9 > > > We see unit test failures in Jenkins that look like this: > {noformat} > java.lang.IllegalArgumentException: run queue not empty > at > org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100) > at > org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85) > at > org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180) > {noformat} > This isn't enough information to debug the situation. The test code in > question looks reasonable enough -- it clears the object for re-use between > tests. However, somewhere between stop/clear/start we miss something. Add > some toString implementations and dump the objects in the preconditions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28354) RegionSizeCalculator throws NPE when regions are in transition
[ https://issues.apache.org/jira/browse/HBASE-28354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28354: - Fix Version/s: 2.5.9 (was: 2.5.8) > RegionSizeCalculator throws NPE when regions are in transition > -- > > Key: HBASE-28354 > URL: https://issues.apache.org/jira/browse/HBASE-28354 > Project: HBase > Issue Type: Bug >Reporter: Bryan Beaudreault >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2, 2.5.9 > > > When a region is in transition, it may briefly have a null ServerName in > meta. The RegionSizeCalculator calls RegionLocator.getAllRegionLocations() > and does not handle the possibility that a RegionLocation.getServerName() > could be null. The ServerName is eventually passed into an Admin call, which > results in an NPE. > This has come up in other contexts. For example, taking a look at > getAllRegionLocations() impl, we have checks to ensure that we don't call > null server names. We need to similarly handle the possibility of nulls in > RegionSizeCalculator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28403) Improve debugging for failures in procedure tests
[ https://issues.apache.org/jira/browse/HBASE-28403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28403: - Fix Version/s: 2.5.9 > Improve debugging for failures in procedure tests > - > > Key: HBASE-28403 > URL: https://issues.apache.org/jira/browse/HBASE-28403 > Project: HBase > Issue Type: Task > Components: proc-v2, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2, 2.5.9 > > > We see unit test failures in Jenkins that look like this: > {noformat} > java.lang.IllegalArgumentException: run queue not empty > at > org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100) > at > org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85) > at > org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180) > {noformat} > This isn't enough information to debug the situation. The test code in > question looks reasonable enough -- it clears the object for re-use between > tests. However, somewhere between stop/clear/start we miss something. Add > some toString implementations and dump the objects in the preconditions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28414) create-release should spotless:apply after making any file changes
Nick Dimiduk created HBASE-28414: Summary: create-release should spotless:apply after making any file changes Key: HBASE-28414 URL: https://issues.apache.org/jira/browse/HBASE-28414 Project: HBase Issue Type: Task Components: create-release Reporter: Nick Dimiduk Looks like the release notes generator can sometimes leave whitespace in its changes, as is currently the case on branch-2.5. We should be a little more careful about this in the create-release scripts. Anything that performs and commits some change should run spotless:apply over the affected files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28413) Fix race condition in TestCleanerChore.retriesIOExceptionInStatus
[ https://issues.apache.org/jira/browse/HBASE-28413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28413: - Affects Version/s: 3.0.0-alpha-4 2.6.0 4.0.0-alpha-1 > Fix race condition in TestCleanerChore.retriesIOExceptionInStatus > - > > Key: HBASE-28413 > URL: https://issues.apache.org/jira/browse/HBASE-28413 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 2.6.0, 3.0.0-alpha-4, 4.0.0-alpha-1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > We occasionally get a test failure in > TestCleanerChore.retriesIOExceptionInStatus. For example, from a [recent PR > build|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5725/1/testReport/org.apache.hadoop.hbase.master.cleaner/TestCleanerChore/precommit_checks___yetus_jdk11_hadoop3_checks___retriesIOExceptionInStatus/] > on branch-2.6, > {noformat} > java.util.concurrent.ExecutionException: java.io.IOException: whomp whomp. > at > java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) > at > java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) > at > org.apache.hadoop.hbase.master.cleaner.TestCleanerChore.retriesIOExceptionInStatus(TestCleanerChore.java:163) > ... > Caused by: java.io.IOException: whomp whomp. > at > org.apache.hadoop.hbase.master.cleaner.TestCleanerChore$1.listStatus(TestCleanerChore.java:134) > at > org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:475) > at > org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$chore$0(CleanerChore.java:258) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > ... 1 more > {noformat} > This looks like a race condition where the chore manages an entire execution > between when the flag is flipped and when the test thread gets back around to > continuing execution. Make the test a little more pessimistic about its view > of the world. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28413) Fix race condition in TestCleanerChore.retriesIOExceptionInStatus
[ https://issues.apache.org/jira/browse/HBASE-28413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822412#comment-17822412 ] Nick Dimiduk commented on HBASE-28413: -- Looking at the other test changes from HBASE-27321, I think that this is the only one that has such a race. FYI [~zhangduo]. > Fix race condition in TestCleanerChore.retriesIOExceptionInStatus > - > > Key: HBASE-28413 > URL: https://issues.apache.org/jira/browse/HBASE-28413 > Project: HBase > Issue Type: Test > Components: test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > We occasionally get a test failure in > TestCleanerChore.retriesIOExceptionInStatus. For example, from a [recent PR > build|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5725/1/testReport/org.apache.hadoop.hbase.master.cleaner/TestCleanerChore/precommit_checks___yetus_jdk11_hadoop3_checks___retriesIOExceptionInStatus/] > on branch-2.6, > {noformat} > java.util.concurrent.ExecutionException: java.io.IOException: whomp whomp. > at > java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) > at > java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) > at > org.apache.hadoop.hbase.master.cleaner.TestCleanerChore.retriesIOExceptionInStatus(TestCleanerChore.java:163) > ... > Caused by: java.io.IOException: whomp whomp. > at > org.apache.hadoop.hbase.master.cleaner.TestCleanerChore$1.listStatus(TestCleanerChore.java:134) > at > org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:475) > at > org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$chore$0(CleanerChore.java:258) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > ... 1 more > {noformat} > This looks like a race condition where the chore manages an entire execution > between when the flag is flipped and when the test thread gets back around to > continuing execution. Make the test a little more pessimistic about its view > of the world. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28413) Fix race condition in TestCleanerChore.retriesIOExceptionInStatus
[ https://issues.apache.org/jira/browse/HBASE-28413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28413: - Status: Patch Available (was: In Progress) > Fix race condition in TestCleanerChore.retriesIOExceptionInStatus > - > > Key: HBASE-28413 > URL: https://issues.apache.org/jira/browse/HBASE-28413 > Project: HBase > Issue Type: Test > Components: test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > We occasionally get a test failure in > TestCleanerChore.retriesIOExceptionInStatus. For example, from a [recent PR > build|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5725/1/testReport/org.apache.hadoop.hbase.master.cleaner/TestCleanerChore/precommit_checks___yetus_jdk11_hadoop3_checks___retriesIOExceptionInStatus/] > on branch-2.6, > {noformat} > java.util.concurrent.ExecutionException: java.io.IOException: whomp whomp. > at > java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) > at > java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) > at > org.apache.hadoop.hbase.master.cleaner.TestCleanerChore.retriesIOExceptionInStatus(TestCleanerChore.java:163) > ... > Caused by: java.io.IOException: whomp whomp. > at > org.apache.hadoop.hbase.master.cleaner.TestCleanerChore$1.listStatus(TestCleanerChore.java:134) > at > org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:475) > at > org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$chore$0(CleanerChore.java:258) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > ... 1 more > {noformat} > This looks like a race condition where the chore manages an entire execution > between when the flag is flipped and when the test thread gets back around to > continuing execution. Make the test a little more pessimistic about its view > of the world. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28403) Improve debugging for failures in procedure tests
[ https://issues.apache.org/jira/browse/HBASE-28403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28403: - Fix Version/s: 2.6.0 > Improve debugging for failures in procedure tests > - > > Key: HBASE-28403 > URL: https://issues.apache.org/jira/browse/HBASE-28403 > Project: HBase > Issue Type: Task > Components: proc-v2, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We see unit test failures in Jenkins that look like this: > {noformat} > java.lang.IllegalArgumentException: run queue not empty > at > org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100) > at > org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85) > at > org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180) > {noformat} > This isn't enough information to debug the situation. The test code in > question looks reasonable enough -- it clears the object for re-use between > tests. However, somewhere between stop/clear/start we miss something. Add > some toString implementations and dump the objects in the preconditions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-28413) Fix race condition in TestCleanerChore.retriesIOExceptionInStatus
[ https://issues.apache.org/jira/browse/HBASE-28413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-28413 started by Nick Dimiduk. > Fix race condition in TestCleanerChore.retriesIOExceptionInStatus > - > > Key: HBASE-28413 > URL: https://issues.apache.org/jira/browse/HBASE-28413 > Project: HBase > Issue Type: Test > Components: test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > We occasionally get a test failure in > TestCleanerChore.retriesIOExceptionInStatus. For example, from a [recent PR > build|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5725/1/testReport/org.apache.hadoop.hbase.master.cleaner/TestCleanerChore/precommit_checks___yetus_jdk11_hadoop3_checks___retriesIOExceptionInStatus/] > on branch-2.6, > {noformat} > java.util.concurrent.ExecutionException: java.io.IOException: whomp whomp. > at > java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) > at > java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) > at > org.apache.hadoop.hbase.master.cleaner.TestCleanerChore.retriesIOExceptionInStatus(TestCleanerChore.java:163) > ... > Caused by: java.io.IOException: whomp whomp. > at > org.apache.hadoop.hbase.master.cleaner.TestCleanerChore$1.listStatus(TestCleanerChore.java:134) > at > org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:475) > at > org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$chore$0(CleanerChore.java:258) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > ... 1 more > {noformat} > This looks like a race condition where the chore manages an entire execution > between when the flag is flipped and when the test thread gets back around to > continuing execution. Make the test a little more pessimistic about its view > of the world. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28413) Fix race condition in TestCleanerChore.retriesIOExceptionInStatus
Nick Dimiduk created HBASE-28413: Summary: Fix race condition in TestCleanerChore.retriesIOExceptionInStatus Key: HBASE-28413 URL: https://issues.apache.org/jira/browse/HBASE-28413 Project: HBase Issue Type: Test Components: test Reporter: Nick Dimiduk Assignee: Nick Dimiduk We occasionally get a test failure in TestCleanerChore.retriesIOExceptionInStatus. For example, from a [recent PR build|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5725/1/testReport/org.apache.hadoop.hbase.master.cleaner/TestCleanerChore/precommit_checks___yetus_jdk11_hadoop3_checks___retriesIOExceptionInStatus/] on branch-2.6, {noformat} java.util.concurrent.ExecutionException: java.io.IOException: whomp whomp. at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) at org.apache.hadoop.hbase.master.cleaner.TestCleanerChore.retriesIOExceptionInStatus(TestCleanerChore.java:163) ... Caused by: java.io.IOException: whomp whomp. at org.apache.hadoop.hbase.master.cleaner.TestCleanerChore$1.listStatus(TestCleanerChore.java:134) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:475) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$chore$0(CleanerChore.java:258) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ... 1 more {noformat} This looks like a race condition where the chore manages an entire execution between when the flag is flipped and when the test thread gets back around to continuing execution. Make the test a little more pessimistic about its view of the world. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28354) RegionSizeCalculator throws NPE when regions are in transition
[ https://issues.apache.org/jira/browse/HBASE-28354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28354: - Fix Version/s: 2.5.8 > RegionSizeCalculator throws NPE when regions are in transition > -- > > Key: HBASE-28354 > URL: https://issues.apache.org/jira/browse/HBASE-28354 > Project: HBase > Issue Type: Bug >Reporter: Bryan Beaudreault >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 2.5.8, 3.0.0-beta-2 > > > When a region is in transition, it may briefly have a null ServerName in > meta. The RegionSizeCalculator calls RegionLocator.getAllRegionLocations() > and does not handle the possibility that a RegionLocation.getServerName() > could be null. The ServerName is eventually passed into an Admin call, which > results in an NPE. > This has come up in other contexts. For example, taking a look at > getAllRegionLocations() impl, we have checks to ensure that we don't call > null server names. We need to similarly handle the possibility of nulls in > RegionSizeCalculator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28354) RegionSizeCalculator throws NPE when regions are in transition
[ https://issues.apache.org/jira/browse/HBASE-28354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28354. -- Resolution: Fixed Committed to branches 2.5+. Thanks for the contribution [~aalhour]. > RegionSizeCalculator throws NPE when regions are in transition > -- > > Key: HBASE-28354 > URL: https://issues.apache.org/jira/browse/HBASE-28354 > Project: HBase > Issue Type: Bug >Reporter: Bryan Beaudreault >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2, 2.5.8 > > > When a region is in transition, it may briefly have a null ServerName in > meta. The RegionSizeCalculator calls RegionLocator.getAllRegionLocations() > and does not handle the possibility that a RegionLocation.getServerName() > could be null. The ServerName is eventually passed into an Admin call, which > results in an NPE. > This has come up in other contexts. For example, taking a look at > getAllRegionLocations() impl, we have checks to ensure that we don't call > null server names. We need to similarly handle the possibility of nulls in > RegionSizeCalculator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28354) RegionSizeCalculator throws NPE when regions are in transition
[ https://issues.apache.org/jira/browse/HBASE-28354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28354: - Fix Version/s: 2.6.0 (was: 2.7.0) > RegionSizeCalculator throws NPE when regions are in transition > -- > > Key: HBASE-28354 > URL: https://issues.apache.org/jira/browse/HBASE-28354 > Project: HBase > Issue Type: Bug >Reporter: Bryan Beaudreault >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > When a region is in transition, it may briefly have a null ServerName in > meta. The RegionSizeCalculator calls RegionLocator.getAllRegionLocations() > and does not handle the possibility that a RegionLocation.getServerName() > could be null. The ServerName is eventually passed into an Admin call, which > results in an NPE. > This has come up in other contexts. For example, taking a look at > getAllRegionLocations() impl, we have checks to ensure that we don't call > null server names. We need to similarly handle the possibility of nulls in > RegionSizeCalculator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28403) Improve debugging for failures in procedure tests
[ https://issues.apache.org/jira/browse/HBASE-28403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28403: - Fix Version/s: 3.0.0-beta-2 > Improve debugging for failures in procedure tests > - > > Key: HBASE-28403 > URL: https://issues.apache.org/jira/browse/HBASE-28403 > Project: HBase > Issue Type: Task > Components: proc-v2, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 3.0.0-beta-2 > > > We see unit test failures in Jenkins that look like this: > {noformat} > java.lang.IllegalArgumentException: run queue not empty > at > org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100) > at > org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85) > at > org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180) > {noformat} > This isn't enough information to debug the situation. The test code in > question looks reasonable enough -- it clears the object for re-use between > tests. However, somewhere between stop/clear/start we miss something. Add > some toString implementations and dump the objects in the preconditions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28354) RegionSizeCalculator throws NPE when regions are in transition
[ https://issues.apache.org/jira/browse/HBASE-28354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28354: - Fix Version/s: 2.7.0 > RegionSizeCalculator throws NPE when regions are in transition > -- > > Key: HBASE-28354 > URL: https://issues.apache.org/jira/browse/HBASE-28354 > Project: HBase > Issue Type: Bug >Reporter: Bryan Beaudreault >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > When a region is in transition, it may briefly have a null ServerName in > meta. The RegionSizeCalculator calls RegionLocator.getAllRegionLocations() > and does not handle the possibility that a RegionLocation.getServerName() > could be null. The ServerName is eventually passed into an Admin call, which > results in an NPE. > This has come up in other contexts. For example, taking a look at > getAllRegionLocations() impl, we have checks to ensure that we don't call > null server names. We need to similarly handle the possibility of nulls in > RegionSizeCalculator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28354) RegionSizeCalculator throws NPE when regions are in transition
[ https://issues.apache.org/jira/browse/HBASE-28354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28354: - Fix Version/s: 3.0.0-beta-2 > RegionSizeCalculator throws NPE when regions are in transition > -- > > Key: HBASE-28354 > URL: https://issues.apache.org/jira/browse/HBASE-28354 > Project: HBase > Issue Type: Bug >Reporter: Bryan Beaudreault >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 3.0.0-beta-2 > > > When a region is in transition, it may briefly have a null ServerName in > meta. The RegionSizeCalculator calls RegionLocator.getAllRegionLocations() > and does not handle the possibility that a RegionLocation.getServerName() > could be null. The ServerName is eventually passed into an Admin call, which > results in an NPE. > This has come up in other contexts. For example, taking a look at > getAllRegionLocations() impl, we have checks to ensure that we don't call > null server names. We need to similarly handle the possibility of nulls in > RegionSizeCalculator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28342. -- Resolution: Fixed > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Improvement > Components: master >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28354) RegionSizeCalculator throws NPE when regions are in transition
[ https://issues.apache.org/jira/browse/HBASE-28354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28354: - Fix Version/s: 4.0.0-alpha-1 > RegionSizeCalculator throws NPE when regions are in transition > -- > > Key: HBASE-28354 > URL: https://issues.apache.org/jira/browse/HBASE-28354 > Project: HBase > Issue Type: Bug >Reporter: Bryan Beaudreault >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > > When a region is in transition, it may briefly have a null ServerName in > meta. The RegionSizeCalculator calls RegionLocator.getAllRegionLocations() > and does not handle the possibility that a RegionLocation.getServerName() > could be null. The ServerName is eventually passed into an Admin call, which > results in an NPE. > This has come up in other contexts. For example, taking a look at > getAllRegionLocations() impl, we have checks to ensure that we don't call > null server names. We need to similarly handle the possibility of nulls in > RegionSizeCalculator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28403) Improve debugging for failures in procedure tests
[ https://issues.apache.org/jira/browse/HBASE-28403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28403: - Fix Version/s: 4.0.0-alpha-1 > Improve debugging for failures in procedure tests > - > > Key: HBASE-28403 > URL: https://issues.apache.org/jira/browse/HBASE-28403 > Project: HBase > Issue Type: Task > Components: proc-v2, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > > We see unit test failures in Jenkins that look like this: > {noformat} > java.lang.IllegalArgumentException: run queue not empty > at > org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100) > at > org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85) > at > org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180) > {noformat} > This isn't enough information to debug the situation. The test code in > question looks reasonable enough -- it clears the object for re-use between > tests. However, somewhere between stop/clear/start we miss something. Add > some toString implementations and dump the objects in the preconditions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28403) Improve debugging for failures in procedure tests
[ https://issues.apache.org/jira/browse/HBASE-28403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822086#comment-17822086 ] Nick Dimiduk commented on HBASE-28403: -- [~apurtell] do you have any concerns with this landing on branch-2.5 ? Thanks. > Improve debugging for failures in procedure tests > - > > Key: HBASE-28403 > URL: https://issues.apache.org/jira/browse/HBASE-28403 > Project: HBase > Issue Type: Task > Components: proc-v2, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > We see unit test failures in Jenkins that look like this: > {noformat} > java.lang.IllegalArgumentException: run queue not empty > at > org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100) > at > org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85) > at > org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180) > {noformat} > This isn't enough information to debug the situation. The test code in > question looks reasonable enough -- it clears the object for re-use between > tests. However, somewhere between stop/clear/start we miss something. Add > some toString implementations and dump the objects in the preconditions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-23324) Deprecate clients that connect to Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-23324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822073#comment-17822073 ] Nick Dimiduk commented on HBASE-23324: -- https://lists.apache.org/thread/ksw4tb8h22ojwmbn7pqwc7gox70vgzgr > Deprecate clients that connect to Zookeeper > --- > > Key: HBASE-23324 > URL: https://issues.apache.org/jira/browse/HBASE-23324 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Priority: Major > > Sort out on what schedule we'll deprecated and then delete the old > client-to-zookeeper path. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-23324) Deprecate clients that connect to Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-23324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822028#comment-17822028 ] Nick Dimiduk commented on HBASE-23324: -- I haven't thought about this since the issue was filed. Yes, I think that we could talk through a deprecation path. Let me start a DISCUSS on the mailing list. > Deprecate clients that connect to Zookeeper > --- > > Key: HBASE-23324 > URL: https://issues.apache.org/jira/browse/HBASE-23324 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Priority: Major > > Sort out on what schedule we'll deprecated and then delete the old > client-to-zookeeper path. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reopened HBASE-28342: -- Sorry, I missed something in review. Looking more closely at the other exceptions thrown, I think we should make two small changes. First, the DecommissionedHostRejectedException should be in the `org.apache.hadoop.hbase.ipc` package (still in the hbase-server module). Second, It should be annotated as `@InterfaceAudience.Public` because it's part of our RPC protocol. > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Improvement > Components: master >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28342: - Release Note: This change introduces the configuration `hbase.master.reject.decommissioned.hosts`. When this property is set to `true`, region servers added to the [decommissioning hosts list](https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Admin.html#decommissionRegionServers-java.util.List-boolean-) will be checked by hostname only (not taking into consideration RPC port or startcode). When a region server with a hostname that matches the list attempts to join the cluster, the Master will reject its application by responding with the new `DecommissionedHostRejectedException`. > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Improvement > Components: master >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28342: - Component/s: master > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Improvement > Components: master >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28342: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to master, branch-3, branch-2, branch-2.6. Thanks for the contribution [~aalhour]! > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Improvement >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28399) region size can be wrong from RegionSizeCalculator
[ https://issues.apache.org/jira/browse/HBASE-28399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821677#comment-17821677 ] Nick Dimiduk commented on HBASE-28399: -- spark.hadoopRDD.ignoreEmptySplits seems like a dangerous optimization for a system with no guarantees over its data sources... For HBase anyway, getSplits() is a best effort operation, we should disable or warn very loudly if this configuration is present. > region size can be wrong from RegionSizeCalculator > -- > > Key: HBASE-28399 > URL: https://issues.apache.org/jira/browse/HBASE-28399 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 3.0.0-beta-1 >Reporter: ruanhui >Assignee: ruanhui >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > > The RegionSizeCalculator calculates region byte size using the following > method > {code:java} > private static final long MEGABYTE = 1024L * 1024L; > long regionSizeBytes = > ((long) regionLoad.getStoreFileSize().get(Size.Unit.MEGABYTE)) * MEGABYTE; > {code} > However, this method will lose accuracy. For example, the result of > {code:java} > ((long) new Size(1, Size.Unit.BYTE).get(Size.Unit.MEGABYTE)) * MEGABYTE {code} > is 0. This will result in a TableInputSplit with a length of 0, but in fact > this TableInputSplit has a small amount of data. > > This TableInputSplit will be ignored if we enable > spark.hadoopRDD.ignoreEmptySplits. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27312) Update create-release to work with maven-gpg-plugin-3.0.1 and gnupg >= 2.1.x
[ https://issues.apache.org/jira/browse/HBASE-27312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821592#comment-17821592 ] Nick Dimiduk commented on HBASE-27312: -- Yes, that could be this issue. But now that I re-read what I wrote in MGPG-92, I think I have it backwards. My current understanding is that the "extra" socket is the one with restricted permissions, the one for which overriding the pinentry method is forbidden. I believe the description on this jira is correct. > Update create-release to work with maven-gpg-plugin-3.0.1 and gnupg >= 2.1.x > > > Key: HBASE-27312 > URL: https://issues.apache.org/jira/browse/HBASE-27312 > Project: HBase > Issue Type: Task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 3.0.0-alpha-4 > > > I've run into some trouble with `create-release`. The issue seems to come > down to changes to maven-release-plugin via MGPG-79 and working with the > `agent-extra-socket`. After MGPG-79, maven-release-plugin uses > `--pinentry-mode error` in a non-interactive setting, which is not > permissible over the restricted permissions exposed via `agent-extra-socket`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28048) RSProcedureDispatcher to abort executing request after configurable retries
[ https://issues.apache.org/jira/browse/HBASE-28048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821590#comment-17821590 ] Nick Dimiduk commented on HBASE-28048: -- An alternative to the liveliness check would be to tie procedure execution into events processed by the {{ServerManager}}. When Master recognizes that the {{ServerName}} that is the target of a {{RemoteProcedure}} has left the cluster, the remote procedure must be failed. How the failure is handled is up to each procedure. > RSProcedureDispatcher to abort executing request after configurable retries > --- > > Key: HBASE-28048 > URL: https://issues.apache.org/jira/browse/HBASE-28048 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-4, 2.4.17, 2.5.5 >Reporter: Viraj Jasani >Priority: Major > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > In a recent incident, we observed that RSProcedureDispatcher continues > executing region open/close procedures with unbounded retries even in the > presence of known failures like GSS initiate failure: > > {code:java} > 2023-08-25 02:21:02,821 WARN [ispatcher-pool-40777] > procedure.RSProcedureDispatcher - request to rs1,61020,1692930044498 failed > due to java.io.IOException: Call to address=rs1:61020 failed on local > exception: java.io.IOException: > org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException: > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): GSS > initiate failed, try=0, retrying... {code} > > > If the remote execution results in IOException, the dispatcher attempts to > schedule the procedure for further retries: > > {code:java} > private boolean scheduleForRetry(IOException e) { > LOG.debug("Request to {} failed, try={}", serverName, > numberOfAttemptsSoFar, e); > // Should we wait a little before retrying? If the server is starting > it's yes. > ... > ... > ... > numberOfAttemptsSoFar++; > // Add some backoff here as the attempts rise otherwise if a stuck > condition, will fill logs > // with failed attempts. None of our backoff classes -- RetryCounter or > ClientBackoffPolicy > // -- fit here nicely so just do something simple; increment by > rsRpcRetryInterval millis * > // retry^2 on each try > // up to max of 10 seconds (don't want to back off too much in case of > situation change). > submitTask(this, > Math.min(rsRpcRetryInterval * (this.numberOfAttemptsSoFar * > this.numberOfAttemptsSoFar), > 10 * 1000), > TimeUnit.MILLISECONDS); > return true; > } > {code} > > > Even though we try to provide backoff while retrying, max wait time is 10s: > > {code:java} > submitTask(this, > Math.min(rsRpcRetryInterval * (this.numberOfAttemptsSoFar * > this.numberOfAttemptsSoFar), > 10 * 1000), > TimeUnit.MILLISECONDS); {code} > > > This results in endless loop of retries, until either the underlying issue is > fixed (e.g. krb issue in this case) or regionserver is killed and the ongoing > open/close region procedure (and perhaps entire SCP) for the affected > regionserver is sidelined manually. > {code:java} > 2023-08-25 03:04:18,918 WARN [ispatcher-pool-41274] > procedure.RSProcedureDispatcher - request to rs1,61020,1692930044498 failed > due to java.io.IOException: Call to address=rs1:61020 failed on local > exception: java.io.IOException: > org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException: > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): GSS > initiate failed, try=217, retrying... > 2023-08-25 03:04:18,916 WARN [ispatcher-pool-41280] > procedure.RSProcedureDispatcher - request to rs1,61020,1692930044498 failed > due to java.io.IOException: Call to address=rs1:61020 failed on local > exception: java.io.IOException: > org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException: > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): GSS > initiate failed, try=193, retrying... > 2023-08-25 03:04:28,968 WARN [ispatcher-pool-41315] > procedure.RSProcedureDispatcher - request to rs1,61020,1692930044498 failed > due to java.io.IOException: Call to address=rs1:61020 failed on local > exception: java.io.IOException: > org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException: > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): GSS > initiate failed, try=266, retrying... > 2023-08-25 03:04:28,969 WARN [ispatcher-pool-41240] > procedure.RSProcedureDispatcher - request to rs1,61020,1692930044498 failed > due to java.io.IOException: Call to address=rs1:61020 failed on local > exception: java.io.IOException: >
[jira] [Commented] (HBASE-26372) [JDK17] Jenkins build support
[ https://issues.apache.org/jira/browse/HBASE-26372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821587#comment-17821587 ] Nick Dimiduk commented on HBASE-26372: -- Looking good ! > [JDK17] Jenkins build support > - > > Key: HBASE-26372 > URL: https://issues.apache.org/jira/browse/HBASE-26372 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Attachments: Screenshot from 2024-02-23 22-58-51.png > > > We'll need to update our build infrastructure to include a JDK17 environment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28342: - Issue Type: Improvement (was: Bug) > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Improvement >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28407) [thirdparty] Update release instructions
[ https://issues.apache.org/jira/browse/HBASE-28407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28407: - Status: Patch Available (was: Open) > [thirdparty] Update release instructions > > > Key: HBASE-28407 > URL: https://issues.apache.org/jira/browse/HBASE-28407 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Our release instructions for hbase-thirdparty are out of date. Update them > based on recent experience. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28407) [thirdparty] Update release instructions
Nick Dimiduk created HBASE-28407: Summary: [thirdparty] Update release instructions Key: HBASE-28407 URL: https://issues.apache.org/jira/browse/HBASE-28407 Project: HBase Issue Type: Task Components: thirdparty Reporter: Nick Dimiduk Assignee: Nick Dimiduk Our release instructions for hbase-thirdparty are out of date. Update them based on recent experience. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28279) Bump up jetty-server, jetty-http to 9.4.53.v20231009
[ https://issues.apache.org/jira/browse/HBASE-28279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28279: - Issue Type: Task (was: Bug) > Bump up jetty-server, jetty-http to 9.4.53.v20231009 > > > Key: HBASE-28279 > URL: https://issues.apache.org/jira/browse/HBASE-28279 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: thirdparty-4.1.6 > > > Bump up jetty-server to 9.4.53.v20231009 to avoid CVE CVE-2023-36478 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28374) [hbase-thirdparty] bump deps for 4.1.6 release
[ https://issues.apache.org/jira/browse/HBASE-28374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28374: - Component/s: thirdparty > [hbase-thirdparty] bump deps for 4.1.6 release > -- > > Key: HBASE-28374 > URL: https://issues.apache.org/jira/browse/HBASE-28374 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > * protobuf 3.24.3 -> 3.25.2 > * guava 32.1.2-jre -> 33.0.0-jre > * commons-cli 1.5.0 -> 1.6.0 > * jetty 9.4.53.v20231009 -> 9.4.54.v20240208 > * jersey 2.40 -> 2.41 > * javassist 3.29.2-GA -> 3.30.2-GA > * jackson-jaxrs-json-provider 2.15.2 -> 2.16.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28360) [hbase-thirdparty] Upgrade Netty to 4.1.107.Final
[ https://issues.apache.org/jira/browse/HBASE-28360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28360: - Component/s: thirdparty > [hbase-thirdparty] Upgrade Netty to 4.1.107.Final > - > > Key: HBASE-28360 > URL: https://issues.apache.org/jira/browse/HBASE-28360 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Fix For: thirdparty-4.1.6 > > > https://netty.io/news/2024/02/13/4-1-107-Final.html -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28374) [hbase-thirdparty] bump deps for 4.1.6 release
[ https://issues.apache.org/jira/browse/HBASE-28374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28374. -- Resolution: Fixed > [hbase-thirdparty] bump deps for 4.1.6 release > -- > > Key: HBASE-28374 > URL: https://issues.apache.org/jira/browse/HBASE-28374 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > * protobuf 3.24.3 -> 3.25.2 > * guava 32.1.2-jre -> 33.0.0-jre > * commons-cli 1.5.0 -> 1.6.0 > * jetty 9.4.53.v20231009 -> 9.4.54.v20240208 > * jersey 2.40 -> 2.41 > * javassist 3.29.2-GA -> 3.30.2-GA > * jackson-jaxrs-json-provider 2.15.2 -> 2.16.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28403) Improve debugging for failures in procedure tests
[ https://issues.apache.org/jira/browse/HBASE-28403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-28403: Assignee: Nick Dimiduk > Improve debugging for failures in procedure tests > - > > Key: HBASE-28403 > URL: https://issues.apache.org/jira/browse/HBASE-28403 > Project: HBase > Issue Type: Task > Components: proc-v2, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > We see unit test failures in Jenkins that look like this: > {noformat} > java.lang.IllegalArgumentException: run queue not empty > at > org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100) > at > org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85) > at > org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180) > {noformat} > This isn't enough information to debug the situation. The test code in > question looks reasonable enough -- it clears the object for re-use between > tests. However, somewhere between stop/clear/start we miss something. Add > some toString implementations and dump the objects in the preconditions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28403) Improve debugging for failures in procedure tests
[ https://issues.apache.org/jira/browse/HBASE-28403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28403: - Status: Patch Available (was: Open) > Improve debugging for failures in procedure tests > - > > Key: HBASE-28403 > URL: https://issues.apache.org/jira/browse/HBASE-28403 > Project: HBase > Issue Type: Task > Components: proc-v2, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > We see unit test failures in Jenkins that look like this: > {noformat} > java.lang.IllegalArgumentException: run queue not empty > at > org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132) > at > org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100) > at > org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85) > at > org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180) > {noformat} > This isn't enough information to debug the situation. The test code in > question looks reasonable enough -- it clears the object for re-use between > tests. However, somewhere between stop/clear/start we miss something. Add > some toString implementations and dump the objects in the preconditions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28403) Improve debugging for failures in procedure tests
Nick Dimiduk created HBASE-28403: Summary: Improve debugging for failures in procedure tests Key: HBASE-28403 URL: https://issues.apache.org/jira/browse/HBASE-28403 Project: HBase Issue Type: Task Components: proc-v2, test Reporter: Nick Dimiduk We see unit test failures in Jenkins that look like this: {noformat} java.lang.IllegalArgumentException: run queue not empty at org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665) at org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132) at org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100) at org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85) at org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180) {noformat} This isn't enough information to debug the situation. The test code in question looks reasonable enough -- it clears the object for re-use between tests. However, somewhere between stop/clear/start we miss something. Add some toString implementations and dump the objects in the preconditions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28393) Update Apache Parent POM to version 31
[ https://issues.apache.org/jira/browse/HBASE-28393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820014#comment-17820014 ] Nick Dimiduk commented on HBASE-28393: -- I'm in favor of working through the parent pom version upgrade, at least for master and branch-3. It'll require an audit of all the plugins we've pinned to a specific version. Reading through the new parent pom, I think we might also adopt its style of naming dependency version properties. For example, {{version.maven-antrun-plugin}}. > Update Apache Parent POM to version 31 > -- > > Key: HBASE-28393 > URL: https://issues.apache.org/jira/browse/HBASE-28393 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > > Bump to https://github.com/apache/maven-apache-parent/releases/tag/apache-31 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28393) Update Apache Parent POM to version 31
[ https://issues.apache.org/jira/browse/HBASE-28393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819993#comment-17819993 ] Nick Dimiduk commented on HBASE-28393: -- I generated effective-pom before and after changing the parent, but the diff is massive. https://gist.github.com/ndimiduk/196b75a53aeab2a88fffad0035d57115/revisions#diff-9c5fb3d1b7e3b0f54bc5c4182965c4fe1f9023d449017cece3005d3f90e8e4d8 > Update Apache Parent POM to version 31 > -- > > Key: HBASE-28393 > URL: https://issues.apache.org/jira/browse/HBASE-28393 > Project: HBase > Issue Type: Task > Components: build, dependencies >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > > Bump to https://github.com/apache/maven-apache-parent/releases/tag/apache-31 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-26372) [JDK17] Jenkins build support
[ https://issues.apache.org/jira/browse/HBASE-26372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819973#comment-17819973 ] Nick Dimiduk commented on HBASE-26372: -- bq. This has been handled as part of https://github.com/apache/hbase/pull/5689. PR#5689 addresses nightly builds but we also need pre-commit checks updated, right? > [JDK17] Jenkins build support > - > > Key: HBASE-26372 > URL: https://issues.apache.org/jira/browse/HBASE-26372 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Rajeshbabu Chintaguntla >Priority: Major > > We'll need to update our build infrastructure to include a JDK17 environment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28342: - Status: Patch Available (was: Open) > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Bug >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819972#comment-17819972 ] Nick Dimiduk commented on HBASE-28342: -- [~bbeaudreault] are you good with bringing this one back to branch-2.6? > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Bug >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28342: - Fix Version/s: 2.6.0 4.0.0-alpha-1 3.0.0-beta-2 > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Bug >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28379) Upgrade thirdparty dep to 4.1.6
[ https://issues.apache.org/jira/browse/HBASE-28379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819153#comment-17819153 ] Nick Dimiduk commented on HBASE-28379: -- Testing this PR found HBASE-28387. We should fix that one so that the test suite can proceed. > Upgrade thirdparty dep to 4.1.6 > --- > > Key: HBASE-28379 > URL: https://issues.apache.org/jira/browse/HBASE-28379 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > > Adopt the next hbase-thirdparty release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28387) Broken test TestHRegionWithInMemoryFlush
[ https://issues.apache.org/jira/browse/HBASE-28387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28387: - Attachment: TEST-org.apache.hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush.xml > Broken test TestHRegionWithInMemoryFlush > > > Key: HBASE-28387 > URL: https://issues.apache.org/jira/browse/HBASE-28387 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 4.0.0-alpha-1 >Reporter: Nick Dimiduk >Priority: Major > Attachments: > TEST-org.apache.hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush.xml > > > {{TestHRegionWithInMemoryFlush}} is broken in Jenkins > ([PR35693|https://github.com/apache/hbase/pull/5693]) and in local testing. > It times out while waiting for HMaster to startup. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28387) Broken test TestHRegionWithInMemoryFlush
Nick Dimiduk created HBASE-28387: Summary: Broken test TestHRegionWithInMemoryFlush Key: HBASE-28387 URL: https://issues.apache.org/jira/browse/HBASE-28387 Project: HBase Issue Type: Task Components: test Affects Versions: 4.0.0-alpha-1 Reporter: Nick Dimiduk {{TestHRegionWithInMemoryFlush}} is broken in Jenkins ([PR35693|https://github.com/apache/hbase/pull/5693]) and in local testing. It times out while waiting for HMaster to startup. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28379) Upgrade thirdparty dep to 4.1.6
Nick Dimiduk created HBASE-28379: Summary: Upgrade thirdparty dep to 4.1.6 Key: HBASE-28379 URL: https://issues.apache.org/jira/browse/HBASE-28379 Project: HBase Issue Type: Task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Adopt the next hbase-thirdparty release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28374) [hbase-thirdparty] bump deps for 4.1.6 release
[ https://issues.apache.org/jira/browse/HBASE-28374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28374: - Description: * protobuf 3.24.3 -> 3.25.2 * guava 32.1.2-jre -> 33.0.0-jre * commons-cli 1.5.0 -> 1.6.0 * jetty 9.4.53.v20231009 -> 9.4.54.v20240208 * jersey 2.40 -> 2.41 * javassist 3.29.2-GA -> 3.30.2-GA * jackson-jaxrs-json-provider 2.15.2 -> 2.16.1 was: * protobuf 3.24.3 -> 3.25.2 * guava 32.1.2-jre33.0.0-jre * commons-cli 1.6.0 * jetty 9.4.54.v20240208 * jersey 2.41 * javassist 3.30.2-GA * jackson-jaxrs-json-provider 2.16.1 > [hbase-thirdparty] bump deps for 4.1.6 release > -- > > Key: HBASE-28374 > URL: https://issues.apache.org/jira/browse/HBASE-28374 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > * protobuf 3.24.3 -> 3.25.2 > * guava 32.1.2-jre -> 33.0.0-jre > * commons-cli 1.5.0 -> 1.6.0 > * jetty 9.4.53.v20231009 -> 9.4.54.v20240208 > * jersey 2.40 -> 2.41 > * javassist 3.29.2-GA -> 3.30.2-GA > * jackson-jaxrs-json-provider 2.15.2 -> 2.16.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28374) [hbase-thirdparty] bump deps for 4.1.6 release
[ https://issues.apache.org/jira/browse/HBASE-28374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28374: - Description: * protobuf 3.24.3 -> 3.25.2 * guava 32.1.2-jre33.0.0-jre * commons-cli 1.6.0 * jetty 9.4.54.v20240208 * jersey 2.41 * javassist 3.30.2-GA * jackson-jaxrs-json-provider 2.16.1 > [hbase-thirdparty] bump deps for 4.1.6 release > -- > > Key: HBASE-28374 > URL: https://issues.apache.org/jira/browse/HBASE-28374 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > * protobuf 3.24.3 -> 3.25.2 > * guava 32.1.2-jre33.0.0-jre > * commons-cli 1.6.0 > * jetty 9.4.54.v20240208 > * jersey 2.41 > * javassist 3.30.2-GA > * jackson-jaxrs-json-provider 2.16.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28372) [hbase-thirdparty] Bump protobuf version to 3.25.2
[ https://issues.apache.org/jira/browse/HBASE-28372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28372: - Fix Version/s: (was: thirdparty-4.1.6) Status: Open (was: Patch Available) > [hbase-thirdparty] Bump protobuf version to 3.25.2 > -- > > Key: HBASE-28372 > URL: https://issues.apache.org/jira/browse/HBASE-28372 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > There's an update to protobuf. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28372) [hbase-thirdparty] Bump protobuf version to 3.25.2
[ https://issues.apache.org/jira/browse/HBASE-28372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28372. -- Resolution: Duplicate > [hbase-thirdparty] Bump protobuf version to 3.25.2 > -- > > Key: HBASE-28372 > URL: https://issues.apache.org/jira/browse/HBASE-28372 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > There's an update to protobuf. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28374) [hbase-thirdparty] bump deps for 4.1.6 release
[ https://issues.apache.org/jira/browse/HBASE-28374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28374: - Fix Version/s: (was: thirdparty-4.1.6) Status: Open (was: Patch Available) > [hbase-thirdparty] bump deps for 4.1.6 release > -- > > Key: HBASE-28374 > URL: https://issues.apache.org/jira/browse/HBASE-28374 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28374) [hbase-thirdparty] bump deps for 4.1.6 release
[ https://issues.apache.org/jira/browse/HBASE-28374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28374: - Fix Version/s: thirdparty-4.1.6 > [hbase-thirdparty] bump deps for 4.1.6 release > -- > > Key: HBASE-28374 > URL: https://issues.apache.org/jira/browse/HBASE-28374 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28373) [hbase-thirdparty] upgrade Guava to 33.0.0-jre
[ https://issues.apache.org/jira/browse/HBASE-28373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28373: - Status: Open (was: Patch Available) > [hbase-thirdparty] upgrade Guava to 33.0.0-jre > -- > > Key: HBASE-28373 > URL: https://issues.apache.org/jira/browse/HBASE-28373 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28373) [hbase-thirdparty] upgrade Guava to 33.0.0-jre
[ https://issues.apache.org/jira/browse/HBASE-28373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28373. -- Resolution: Duplicate > [hbase-thirdparty] upgrade Guava to 33.0.0-jre > -- > > Key: HBASE-28373 > URL: https://issues.apache.org/jira/browse/HBASE-28373 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28374) [hbase-thirdparty] bump deps for 4.1.6 release
[ https://issues.apache.org/jira/browse/HBASE-28374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28374: - Summary: [hbase-thirdparty] bump deps for 4.1.6 release (was: [hbase-thirdparty] bump remaining deps for 4.1.6 release) > [hbase-thirdparty] bump deps for 4.1.6 release > -- > > Key: HBASE-28374 > URL: https://issues.apache.org/jira/browse/HBASE-28374 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28373) [hbase-thirdparty] upgrade Guava to 33.0.0-jre
[ https://issues.apache.org/jira/browse/HBASE-28373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28373: - Fix Version/s: (was: thirdparty-4.1.6) > [hbase-thirdparty] upgrade Guava to 33.0.0-jre > -- > > Key: HBASE-28373 > URL: https://issues.apache.org/jira/browse/HBASE-28373 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28374) [hbase-thirdparty] bump remaining deps for 4.1.6 release
[ https://issues.apache.org/jira/browse/HBASE-28374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28374: - Fix Version/s: thirdparty-4.1.6 Status: Patch Available (was: Open) > [hbase-thirdparty] bump remaining deps for 4.1.6 release > > > Key: HBASE-28374 > URL: https://issues.apache.org/jira/browse/HBASE-28374 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28374) [hbase-thirdparty] bump remaining deps for 4.1.6 release
Nick Dimiduk created HBASE-28374: Summary: [hbase-thirdparty] bump remaining deps for 4.1.6 release Key: HBASE-28374 URL: https://issues.apache.org/jira/browse/HBASE-28374 Project: HBase Issue Type: Task Reporter: Nick Dimiduk Assignee: Nick Dimiduk -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28372) [hbase-thirdparty] Bump protobuf version to 3.25.2
[ https://issues.apache.org/jira/browse/HBASE-28372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28372: - Status: Patch Available (was: Open) > [hbase-thirdparty] Bump protobuf version to 3.25.2 > -- > > Key: HBASE-28372 > URL: https://issues.apache.org/jira/browse/HBASE-28372 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > There's an update to protobuf. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28373) [hbase-thirdparty] upgrade Guava to 33.0.0-jre
[ https://issues.apache.org/jira/browse/HBASE-28373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28373: - Fix Version/s: thirdparty-4.1.6 > [hbase-thirdparty] upgrade Guava to 33.0.0-jre > -- > > Key: HBASE-28373 > URL: https://issues.apache.org/jira/browse/HBASE-28373 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28369) Add os-maven-plugin to hbase-thirdparty build
[ https://issues.apache.org/jira/browse/HBASE-28369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28369. -- Resolution: Fixed > Add os-maven-plugin to hbase-thirdparty build > - > > Key: HBASE-28369 > URL: https://issues.apache.org/jira/browse/HBASE-28369 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > A small nice thing we don't yet have in the hbase-thirdparty maven > configuration. This extension prints the current operating system environment > at the start of a build. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28373) [hbase-thirdparty] upgrade Guava to 33.0.0-jre
[ https://issues.apache.org/jira/browse/HBASE-28373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28373: - Status: Patch Available (was: Open) > [hbase-thirdparty] upgrade Guava to 33.0.0-jre > -- > > Key: HBASE-28373 > URL: https://issues.apache.org/jira/browse/HBASE-28373 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28373) [hbase-thirdparty] upgrade Guava to 33.0.0-jre
Nick Dimiduk created HBASE-28373: Summary: [hbase-thirdparty] upgrade Guava to 33.0.0-jre Key: HBASE-28373 URL: https://issues.apache.org/jira/browse/HBASE-28373 Project: HBase Issue Type: Task Reporter: Nick Dimiduk Assignee: Nick Dimiduk -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28372) [hbase-thirdparty] Bump protobuf version to 3.25.2
[ https://issues.apache.org/jira/browse/HBASE-28372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28372: - Summary: [hbase-thirdparty] Bump protobuf version to 3.25.2 (was: Bump protobuf version to 3.25.2) > [hbase-thirdparty] Bump protobuf version to 3.25.2 > -- > > Key: HBASE-28372 > URL: https://issues.apache.org/jira/browse/HBASE-28372 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > There's an update to protobuf. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28372) Bump protobuf version to 3.25.2
[ https://issues.apache.org/jira/browse/HBASE-28372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28372: - Summary: Bump protobuf version to 3.25.2 (was: [third-party] Bump protobuf version to 3.25.2) > Bump protobuf version to 3.25.2 > --- > > Key: HBASE-28372 > URL: https://issues.apache.org/jira/browse/HBASE-28372 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > There's an update to protobuf. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28372) [third-party] Bump protobuf version to 3.25.2
Nick Dimiduk created HBASE-28372: Summary: [third-party] Bump protobuf version to 3.25.2 Key: HBASE-28372 URL: https://issues.apache.org/jira/browse/HBASE-28372 Project: HBase Issue Type: Task Components: thirdparty Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: thirdparty-4.1.6 There's an update to protobuf. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28369) Add os-maven-plugin to hbase-thirdparty build
[ https://issues.apache.org/jira/browse/HBASE-28369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28369: - Fix Version/s: thirdparty-4.1.6 > Add os-maven-plugin to hbase-thirdparty build > - > > Key: HBASE-28369 > URL: https://issues.apache.org/jira/browse/HBASE-28369 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > A small nice thing we don't yet have in the hbase-thirdparty maven > configuration. This extension prints the current operating system environment > at the start of a build. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28369) Add os-maven-plugin to hbase-thirdparty build
Nick Dimiduk created HBASE-28369: Summary: Add os-maven-plugin to hbase-thirdparty build Key: HBASE-28369 URL: https://issues.apache.org/jira/browse/HBASE-28369 Project: HBase Issue Type: Task Components: thirdparty Reporter: Nick Dimiduk Assignee: Nick Dimiduk A small nice thing we don't yet have in the hbase-thirdparty maven configuration. This extension prints the current operating system environment at the start of a build. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28325) Enable infra automation to add label on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815334#comment-17815334 ] Nick Dimiduk commented on HBASE-28325: -- Thanks [~zhangduo] ! > Enable infra automation to add label on a Jira when a new PR is posted > -- > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Duo Zhang >Priority: Minor > Fix For: 4.0.0-alpha-1 > > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28325) Enable infra automation to add label on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-28325: Assignee: Duo Zhang (was: Nick Dimiduk) > Enable infra automation to add label on a Jira when a new PR is posted > -- > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Duo Zhang >Priority: Minor > Fix For: 4.0.0-alpha-1 > > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28325) Enable infra automation to comment on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810769#comment-17810769 ] Nick Dimiduk commented on HBASE-28325: -- {quote} bq. Or somehow get the old ones auto-hidden. I manually hide the old comments as outdated, so yeah, this is on my list as well. It should be possible to add a step to the Jenkins job that reads the old comments, parses the ones that have links containing a previous build-id, and then hiding them. Or maybe this should be a Yetus feature. Either way, I haven't tried this yet... {quote} I filed YETUS-1245. > Enable infra automation to comment on a Jira when a new PR is posted > > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28325) Enable infra automation to comment on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810767#comment-17810767 ] Nick Dimiduk commented on HBASE-28325: -- I reverted the original commit. bq. Does multijdk build run different jdk builds in parallel? Otherwise it will double the pre commit time... Hmm true, yes, it likely does run them sequentially. Even if it could run them in parallel, the resource utilizastion on the worker would be ... a lot. bq. What i'd really love to do is get them on GH Actions, if that were possible. Yetus has nice, easy support for GH Actions ... but last I looked into it, the ASF isn't really ready for us to use GH Actions. Like the bad old days of Jenkins, they have a single pool of resources for all projects. Since several have adopted use of actions, the processing queue is often saturated. That said, Apache has accepted donations specifically ear-marked for HBase. We use this money today for our dedicated Jenkins infrastructure. I'm sure that we could pursue a similar distribution toward GH Action ; I don't know if our account with GitHub can be partitioned that way. That would be a question for the users@infra list or an INFRA ticket. bq. Or somehow get the old ones auto-hidden. I manually hide the old comments as outdated, so yeah, this is on my list as well. It should be possible to add a step to the Jenkins job that reads the old comments, parses the ones that have links containing a previous build-id, and then hiding them. Or maybe this should be a Yetus feature. Either way, I haven't tried this yet... > Enable infra automation to comment on a Jira when a new PR is posted > > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28325) Enable infra automation to comment on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28325: - Fix Version/s: (was: 4.0.0-alpha-1) > Enable infra automation to comment on a Jira when a new PR is posted > > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28325) Enable infra automation to comment on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809970#comment-17809970 ] Nick Dimiduk commented on HBASE-28325: -- The 3x posting on pre-commit is because Yetus's multi-jdk feature didn't work for us back when we added testing of jdk11. There have been a couple yetus releases since then and it's on my todo list to try out multi-jdk on a more recent version. > Enable infra automation to comment on a Jira when a new PR is posted > > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 4.0.0-alpha-1 > > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (HBASE-28325) Enable infra automation to comment on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reopened HBASE-28325: -- I don't want a comment for every PR comment, only a comment when the PR is detected/linked. Asking for clarification on INFRA-25382. > Enable infra automation to comment on a Jira when a new PR is posted > > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 4.0.0-alpha-1 > > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28325) Enable infra automation to comment on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809945#comment-17809945 ] Nick Dimiduk commented on HBASE-28325: -- {quote} ndimiduk commented on PR #5645: URL: https://github.com/apache/hbase/pull/5645#issuecomment-1906048012 Thanks folks. {quote} Oh no. That's exactly what I don't want. > Enable infra automation to comment on a Jira when a new PR is posted > > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 4.0.0-alpha-1 > > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28325) Enable infra automation to comment on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28325: - Fix Version/s: 4.0.0-alpha-1 Resolution: Fixed Status: Resolved (was: Patch Available) > Enable infra automation to comment on a Jira when a new PR is posted > > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 4.0.0-alpha-1 > > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)