[jira] [Resolved] (HBASE-28632) Make -h arg respected by hbck2 and exit if unrecognized arguments are passed
[
https://issues.apache.org/jira/browse/HBASE-28632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-28632.
--
Fix Version/s: hbase-operator-tools-1.3.0
Resolution: Fixed
Thanks [~ksravista] for the contribution and [~rmdmattingly] for the review
help.
> Make -h arg respected by hbck2 and exit if unrecognized arguments are passed
>
>
> Key: HBASE-28632
> URL: https://issues.apache.org/jira/browse/HBASE-28632
> Project: HBase
> Issue Type: Bug
>Reporter: Sravi Kommineni
>Assignee: Sravi Kommineni
>Priority: Major
> Fix For: hbase-operator-tools-1.3.0
>
>
> The -h argument in hbck is not respected and instead of displaying the
> argument usage guide, the command continued to execute. Any unrecognized
> arguments should cause an exception and exit.
>
> example:
> {code:java}
> $ hbck2 addFsRegionsMissingInMeta -h
> OpenJDK 64-Bit Server VM warning: Ignoring option --illegal-access=permit;
> support was removed in 17.0
> 19:09:08.831 [main] INFO org.apache.hadoop.conf.Configuration.deprecation -
> hbase.client.pause.cqtbe is deprecated. Instead, use
> hbase.client.pause.server.overloaded
> ERROR: Unrecognized option: -h
> FOR USAGE, use the -h or --help option {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28682) ITBLL and other MR-based integration tests should heartbeat often
Nick Dimiduk created HBASE-28682:
Summary: ITBLL and other MR-based integration tests should
heartbeat often
Key: HBASE-28682
URL: https://issues.apache.org/jira/browse/HBASE-28682
Project: HBase
Issue Type: Test
Components: integration tests, mapreduce
Reporter: Nick Dimiduk
We have this little note in our ITBLL harness,
{noformat}
// If we cause enough chaos, RPC requests might get into long backoffs.
During this
// time, it won't send keep alives to the map/reduce context. So increase
the timeout
// a bunch
{noformat}
Investigating, the ITBLL Generator's persist method updates the MR context
progress only every 100 puts. You'd think that would be enough, but given
chaos, it really isn't. What if we update progress with every put? Digging
through MR source code, it seems that calling the context.progress() method
only sets an AtomicBoolean that a progress update needs sent, actual sending of
progress reports is gated by {{mapreduce.task.progress-report.interval}}, or 1%
of {{mapreduce.task.timeout}}, which defaults to 1% of 300_000ms, or 3 seconds.
So yeah, we should probably update this AtomicBool much more often in chaotic
jobs, as doing so is effectively free and will improve reliability.
But still, every put is perhaps excessive. What if we add a pre-flush hook to
(Async)BufferedMutator so that a MR job can set this progress flag right before
the client disappears down into a retry loop? I bet other applications would
find such a hook useful as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28642) Hide old PR comments when posting new
Nick Dimiduk created HBASE-28642: Summary: Hide old PR comments when posting new Key: HBASE-28642 URL: https://issues.apache.org/jira/browse/HBASE-28642 Project: HBase Issue Type: Task Components: build, community Reporter: Nick Dimiduk It would be really nice if the build bot would hide the old commits when it posts new ones. When a PR has been open for a while, we end up with more build-bot activity than human activity and it's easy to lose human comments. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28605) Add ErrorProne ban on Hadoop shaded thirdparty jars
[ https://issues.apache.org/jira/browse/HBASE-28605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28605. -- Fix Version/s: 2.7.0 3.0.0-beta-2 2.6.1 2.5.9 Resolution: Fixed Pushed to branch-2.5+. Thanks for the quick reviews. > Add ErrorProne ban on Hadoop shaded thirdparty jars > --- > > Key: HBASE-28605 > URL: https://issues.apache.org/jira/browse/HBASE-28605 > Project: HBase > Issue Type: Task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > Over on HBASE-28568 we got tripped up because we pulled in the shaded Guava > provided by Hadoop. This wasn't noticed until the backport to branch-2, which > builds against hadoop-2. We should make this a compile time failure. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28605) Add ErrorProne ban on Hadoop shaded thirdparty jars
Nick Dimiduk created HBASE-28605: Summary: Add ErrorProne ban on Hadoop shaded thirdparty jars Key: HBASE-28605 URL: https://issues.apache.org/jira/browse/HBASE-28605 Project: HBase Issue Type: Task Components: build Reporter: Nick Dimiduk Over on HBASE-28568 we got tripped up because we pulled in the shaded Guava provided by Hadoop. This wasn't noticed until the backport to branch-2, which builds against hadoop-2. We should make this a compile time failure. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28602) Incremental backup fails when WALs move
Nick Dimiduk created HBASE-28602:
Summary: Incremental backup fails when WALs move
Key: HBASE-28602
URL: https://issues.apache.org/jira/browse/HBASE-28602
Project: HBase
Issue Type: Bug
Components: backuprestore
Affects Versions: 3.0.0-beta-1, 2.6.0, 4.0.0-alpha-1, 2.7.0
Reporter: Nick Dimiduk
The incremental back process appears to collect a set of WAL files to operate
over and then proceed to do so. In between a file moves. This causes the backup
to fail. This is reproducible as a flakey unit test, as we see in
TestIncrementalBackup.TestIncBackupRestore,
{noformat}
java.io.IOException: java.io.FileNotFoundException: File
hdfs://localhost:39577/user/jenkins/test-data/f51646e4-e3e0-ef30-df2b-aa2a22ed41c3/WALs/94f4fe62ee7a,40249,1715620734331/94f4fe62ee7a%2C40249%2C1715620734331.94f4fe62ee7a%2C40249%2C1715620734331.regiongroup-0.1715620773674
does not exist.
at
org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:289)
at
org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:595)
at
org.apache.hadoop.hbase.backup.TestIncrementalBackup.TestIncBackupRestore(TestIncrementalBackup.java:169)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.io.FileNotFoundException: File
hdfs://localhost:39577/user/jenkins/test-data/f51646e4-e3e0-ef30-df2b-aa2a22ed41c3/WALs/94f4fe62ee7a,40249,1715620734331/94f4fe62ee7a%2C40249%2C1715620734331.94f4fe62ee7a%2C40249%2C1715620734331.regiongroup-0.1715620773674
does not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.(DistributedFileSystem.java:1282)
at
org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.(DistributedFileSystem.java:1256)
at
org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1201)
at
org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1197)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:1215)
at
[jira] [Created] (HBASE-28601) Enable setting memcache on-heap sizes in bytes
Nick Dimiduk created HBASE-28601:
Summary: Enable setting memcache on-heap sizes in bytes
Key: HBASE-28601
URL: https://issues.apache.org/jira/browse/HBASE-28601
Project: HBase
Issue Type: Improvement
Components: regionserver
Reporter: Nick Dimiduk
Specifying blockcache and memcache sizes as a percentage of heap is not always
ideal. Sometimes it's easier to specify exact values rather than backing into a
percentage. Let's introduce new configuration settings (perhaps named similarly
to {{hbase.bucketcache.size}}) that accept byte values. Even nicer would be if
these settings accepted human-friendly byte values like {{512m}} or {{10g}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28600) Enable setting blockcache on-heap sizes in bytes
Nick Dimiduk created HBASE-28600:
Summary: Enable setting blockcache on-heap sizes in bytes
Key: HBASE-28600
URL: https://issues.apache.org/jira/browse/HBASE-28600
Project: HBase
Issue Type: Improvement
Components: regionserver
Reporter: Nick Dimiduk
Specifying blockcache and memcache sizes as a percentage of heap is not always
ideal. Sometimes it's easier to specify exact values rather than backing into a
percentage. Let's introduce new configuration settings (perhaps named similarly
to {{hbase.bucketcache.size}}) that accept byte values. Even nicer would be if
these settings accepted human-friendly byte values like {{512m}} or {{10g}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28593) Update "Releasing Apache HBase" section in the book to document `do-release.sh`
Nick Dimiduk created HBASE-28593:
Summary: Update "Releasing Apache HBase" section in the book to
document `do-release.sh`
Key: HBASE-28593
URL: https://issues.apache.org/jira/browse/HBASE-28593
Project: HBase
Issue Type: Task
Components: community
Reporter: Nick Dimiduk
Manually rolling release candidates is a thing of the past. Let's update this
section of the book to describe how to use the automation in
{{dev-support/create-release}} and throw out these old manual instructions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28594) Add a new "Promoting Release Candidate"
Nick Dimiduk created HBASE-28594: Summary: Add a new "Promoting Release Candidate" Key: HBASE-28594 URL: https://issues.apache.org/jira/browse/HBASE-28594 Project: HBase Issue Type: Task Reporter: Nick Dimiduk -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28573) Update compatibility report generator to ignore o.a.h.hbase.shaded packages
Nick Dimiduk created HBASE-28573: Summary: Update compatibility report generator to ignore o.a.h.hbase.shaded packages Key: HBASE-28573 URL: https://issues.apache.org/jira/browse/HBASE-28573 Project: HBase Issue Type: Task Components: community Reporter: Nick Dimiduk This is a small change that will make reviewing release candidates a little easier. Right now that compatibility report includes classes that we shade. So when we shaded upgrade 3rd party dependencies, they show up in this report as an incompatible change. Changes to these classes do not affect users so there's no reason to consider them wrt compatibility. We should update the reporting tool to exclude this package. For example, https://dist.apache.org/repos/dist/dev/hbase/2.6.0RC4/api_compare_2.5.0_to_2.6.0RC4.html#Binary_Removed -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28439) Remove ZooKeeper as a means of creating a client connection
Nick Dimiduk created HBASE-28439: Summary: Remove ZooKeeper as a means of creating a client connection Key: HBASE-28439 URL: https://issues.apache.org/jira/browse/HBASE-28439 Project: HBase Issue Type: Task Components: Client Affects Versions: 4.0.0-alpha-1 Reporter: Nick Dimiduk Following up the discussion and decision around HBASE-23324, we will remove ZooKeeper as a point of entry for client connections. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28379) Upgrade thirdparty dep to 4.1.6
[ https://issues.apache.org/jira/browse/HBASE-28379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28379. -- Resolution: Fixed This is merged. FYI [~bbeaudreault]. > Upgrade thirdparty dep to 4.1.6 > --- > > Key: HBASE-28379 > URL: https://issues.apache.org/jira/browse/HBASE-28379 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > Adopt the next hbase-thirdparty release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28414) create-release should spotless:apply after making any file changes
Nick Dimiduk created HBASE-28414: Summary: create-release should spotless:apply after making any file changes Key: HBASE-28414 URL: https://issues.apache.org/jira/browse/HBASE-28414 Project: HBase Issue Type: Task Components: create-release Reporter: Nick Dimiduk Looks like the release notes generator can sometimes leave whitespace in its changes, as is currently the case on branch-2.5. We should be a little more careful about this in the create-release scripts. Anything that performs and commits some change should run spotless:apply over the affected files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28413) Fix race condition in TestCleanerChore.retriesIOExceptionInStatus
Nick Dimiduk created HBASE-28413:
Summary: Fix race condition in
TestCleanerChore.retriesIOExceptionInStatus
Key: HBASE-28413
URL: https://issues.apache.org/jira/browse/HBASE-28413
Project: HBase
Issue Type: Test
Components: test
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
We occasionally get a test failure in
TestCleanerChore.retriesIOExceptionInStatus. For example, from a [recent PR
build|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5725/1/testReport/org.apache.hadoop.hbase.master.cleaner/TestCleanerChore/precommit_checks___yetus_jdk11_hadoop3_checks___retriesIOExceptionInStatus/]
on branch-2.6,
{noformat}
java.util.concurrent.ExecutionException: java.io.IOException: whomp whomp.
at
java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
at
java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at
org.apache.hadoop.hbase.master.cleaner.TestCleanerChore.retriesIOExceptionInStatus(TestCleanerChore.java:163)
...
Caused by: java.io.IOException: whomp whomp.
at
org.apache.hadoop.hbase.master.cleaner.TestCleanerChore$1.listStatus(TestCleanerChore.java:134)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:475)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$chore$0(CleanerChore.java:258)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
... 1 more
{noformat}
This looks like a race condition where the chore manages an entire execution
between when the flag is flipped and when the test thread gets back around to
continuing execution. Make the test a little more pessimistic about its view of
the world.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28354) RegionSizeCalculator throws NPE when regions are in transition
[ https://issues.apache.org/jira/browse/HBASE-28354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28354. -- Resolution: Fixed Committed to branches 2.5+. Thanks for the contribution [~aalhour]. > RegionSizeCalculator throws NPE when regions are in transition > -- > > Key: HBASE-28354 > URL: https://issues.apache.org/jira/browse/HBASE-28354 > Project: HBase > Issue Type: Bug >Reporter: Bryan Beaudreault >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2, 2.5.8 > > > When a region is in transition, it may briefly have a null ServerName in > meta. The RegionSizeCalculator calls RegionLocator.getAllRegionLocations() > and does not handle the possibility that a RegionLocation.getServerName() > could be null. The ServerName is eventually passed into an Admin call, which > results in an NPE. > This has come up in other contexts. For example, taking a look at > getAllRegionLocations() impl, we have checks to ensure that we don't call > null server names. We need to similarly handle the possibility of nulls in > RegionSizeCalculator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28342. -- Resolution: Fixed > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Improvement > Components: master >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (HBASE-28342) Decommissioned hosts should be rejected by the HMaster
[ https://issues.apache.org/jira/browse/HBASE-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reopened HBASE-28342: -- Sorry, I missed something in review. Looking more closely at the other exceptions thrown, I think we should make two small changes. First, the DecommissionedHostRejectedException should be in the `org.apache.hadoop.hbase.ipc` package (still in the hbase-server module). Second, It should be annotated as `@InterfaceAudience.Public` because it's part of our RPC protocol. > Decommissioned hosts should be rejected by the HMaster > -- > > Key: HBASE-28342 > URL: https://issues.apache.org/jira/browse/HBASE-28342 > Project: HBase > Issue Type: Improvement > Components: master >Reporter: Ahmad Alhour >Assignee: Ahmad Alhour >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 4.0.0-alpha-1, 3.0.0-beta-2 > > > We had an issue with a cluster, internally at HubSpot, where a decommissioned > RegionServer was still being picked up by the HMaster. The host the > RegionServer was living on was impaired, and we couldn't correctly kill the > RegionServer, so the HMaster would periodically hear back from the host and > remove it from its dead host's list. > We would like to implement a fix so that this doesn't happen. We're thinking > of adding a boolean flag to the Decommission RegionServer Admin API that > signifies ignoring the startcode of the servername, when the boolean is True > the host will be rejected every time it comes back even if it had a different > startcode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28407) [thirdparty] Update release instructions
Nick Dimiduk created HBASE-28407: Summary: [thirdparty] Update release instructions Key: HBASE-28407 URL: https://issues.apache.org/jira/browse/HBASE-28407 Project: HBase Issue Type: Task Components: thirdparty Reporter: Nick Dimiduk Assignee: Nick Dimiduk Our release instructions for hbase-thirdparty are out of date. Update them based on recent experience. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28374) [hbase-thirdparty] bump deps for 4.1.6 release
[ https://issues.apache.org/jira/browse/HBASE-28374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28374. -- Resolution: Fixed > [hbase-thirdparty] bump deps for 4.1.6 release > -- > > Key: HBASE-28374 > URL: https://issues.apache.org/jira/browse/HBASE-28374 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > * protobuf 3.24.3 -> 3.25.2 > * guava 32.1.2-jre -> 33.0.0-jre > * commons-cli 1.5.0 -> 1.6.0 > * jetty 9.4.53.v20231009 -> 9.4.54.v20240208 > * jersey 2.40 -> 2.41 > * javassist 3.29.2-GA -> 3.30.2-GA > * jackson-jaxrs-json-provider 2.15.2 -> 2.16.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28403) Improve debugging for failures in procedure tests
Nick Dimiduk created HBASE-28403:
Summary: Improve debugging for failures in procedure tests
Key: HBASE-28403
URL: https://issues.apache.org/jira/browse/HBASE-28403
Project: HBase
Issue Type: Task
Components: proc-v2, test
Reporter: Nick Dimiduk
We see unit test failures in Jenkins that look like this:
{noformat}
java.lang.IllegalArgumentException: run queue not empty
at
org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:332)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:665)
at
org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:132)
at
org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.restart(ProcedureTestingUtility.java:100)
at
org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.restartMasterProcedureExecutor(MasterProcedureTestingUtility.java:85)
at
org.apache.hadoop.hbase.master.assignment.TestRollbackSCP.testFailAndRollback(TestRollbackSCP.java:180)
{noformat}
This isn't enough information to debug the situation. The test code in question
looks reasonable enough -- it clears the object for re-use between tests.
However, somewhere between stop/clear/start we miss something. Add some
toString implementations and dump the objects in the preconditions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28387) Broken test TestHRegionWithInMemoryFlush
Nick Dimiduk created HBASE-28387:
Summary: Broken test TestHRegionWithInMemoryFlush
Key: HBASE-28387
URL: https://issues.apache.org/jira/browse/HBASE-28387
Project: HBase
Issue Type: Task
Components: test
Affects Versions: 4.0.0-alpha-1
Reporter: Nick Dimiduk
{{TestHRegionWithInMemoryFlush}} is broken in Jenkins
([PR35693|https://github.com/apache/hbase/pull/5693]) and in local testing. It
times out while waiting for HMaster to startup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28379) Upgrade thirdparty dep to 4.1.6
Nick Dimiduk created HBASE-28379: Summary: Upgrade thirdparty dep to 4.1.6 Key: HBASE-28379 URL: https://issues.apache.org/jira/browse/HBASE-28379 Project: HBase Issue Type: Task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Adopt the next hbase-thirdparty release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28372) [hbase-thirdparty] Bump protobuf version to 3.25.2
[ https://issues.apache.org/jira/browse/HBASE-28372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28372. -- Resolution: Duplicate > [hbase-thirdparty] Bump protobuf version to 3.25.2 > -- > > Key: HBASE-28372 > URL: https://issues.apache.org/jira/browse/HBASE-28372 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > There's an update to protobuf. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28373) [hbase-thirdparty] upgrade Guava to 33.0.0-jre
[ https://issues.apache.org/jira/browse/HBASE-28373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28373. -- Resolution: Duplicate > [hbase-thirdparty] upgrade Guava to 33.0.0-jre > -- > > Key: HBASE-28373 > URL: https://issues.apache.org/jira/browse/HBASE-28373 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28374) [hbase-thirdparty] bump remaining deps for 4.1.6 release
Nick Dimiduk created HBASE-28374: Summary: [hbase-thirdparty] bump remaining deps for 4.1.6 release Key: HBASE-28374 URL: https://issues.apache.org/jira/browse/HBASE-28374 Project: HBase Issue Type: Task Reporter: Nick Dimiduk Assignee: Nick Dimiduk -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28369) Add os-maven-plugin to hbase-thirdparty build
[ https://issues.apache.org/jira/browse/HBASE-28369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-28369. -- Resolution: Fixed > Add os-maven-plugin to hbase-thirdparty build > - > > Key: HBASE-28369 > URL: https://issues.apache.org/jira/browse/HBASE-28369 > Project: HBase > Issue Type: Task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-4.1.6 > > > A small nice thing we don't yet have in the hbase-thirdparty maven > configuration. This extension prints the current operating system environment > at the start of a build. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28373) [hbase-thirdparty] upgrade Guava to 33.0.0-jre
Nick Dimiduk created HBASE-28373: Summary: [hbase-thirdparty] upgrade Guava to 33.0.0-jre Key: HBASE-28373 URL: https://issues.apache.org/jira/browse/HBASE-28373 Project: HBase Issue Type: Task Reporter: Nick Dimiduk Assignee: Nick Dimiduk -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28372) [third-party] Bump protobuf version to 3.25.2
Nick Dimiduk created HBASE-28372: Summary: [third-party] Bump protobuf version to 3.25.2 Key: HBASE-28372 URL: https://issues.apache.org/jira/browse/HBASE-28372 Project: HBase Issue Type: Task Components: thirdparty Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: thirdparty-4.1.6 There's an update to protobuf. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28369) Add os-maven-plugin to hbase-thirdparty build
Nick Dimiduk created HBASE-28369: Summary: Add os-maven-plugin to hbase-thirdparty build Key: HBASE-28369 URL: https://issues.apache.org/jira/browse/HBASE-28369 Project: HBase Issue Type: Task Components: thirdparty Reporter: Nick Dimiduk Assignee: Nick Dimiduk A small nice thing we don't yet have in the hbase-thirdparty maven configuration. This extension prints the current operating system environment at the start of a build. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (HBASE-28325) Enable infra automation to comment on a Jira when a new PR is posted
[ https://issues.apache.org/jira/browse/HBASE-28325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reopened HBASE-28325: -- I don't want a comment for every PR comment, only a comment when the PR is detected/linked. Asking for clarification on INFRA-25382. > Enable infra automation to comment on a Jira when a new PR is posted > > > Key: HBASE-28325 > URL: https://issues.apache.org/jira/browse/HBASE-28325 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 4.0.0-alpha-1 > > > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions > Currently we make use of the "link" feature. This does not send a > notification to watchers, so I propose that we add the "comment" feature, so > that a comment will also be sent, and watchers can find out about the > availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28325) Enable infra automation to comment on a Jira when a new PR is posted.
Nick Dimiduk created HBASE-28325: Summary: Enable infra automation to comment on a Jira when a new PR is posted. Key: HBASE-28325 URL: https://issues.apache.org/jira/browse/HBASE-28325 Project: HBase Issue Type: Task Components: community Reporter: Nick Dimiduk Assignee: Nick Dimiduk https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Jiranotificationoptions Currently we make use of the "link" feature. This does not send a notification to watchers, so I propose that we add the "comment" feature, so that a comment will also be sent, and watchers can find out about the availability of the PR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (HBASE-27847) Introduce HBase image
[ https://issues.apache.org/jira/browse/HBASE-27847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reopened HBASE-27847: -- > Introduce HBase image > - > > Key: HBASE-27847 > URL: https://issues.apache.org/jira/browse/HBASE-27847 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: HBASE-27827 > > > As with the Hadoop image (HBASE-27846), we need a runtime image for the HBase > containers, and we need a place to define an API between the runtime image > and the orchestration layer. HBase project doesn't ship an image yet, so this > will provide double-duty. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28198) Fix broken link to replication documentation
Nick Dimiduk created HBASE-28198: Summary: Fix broken link to replication documentation Key: HBASE-28198 URL: https://issues.apache.org/jira/browse/HBASE-28198 Project: HBase Issue Type: Task Components: documentation Affects Versions: 3.0.0-alpha-4 Reporter: Nick Dimiduk The site link to the Replication section of the book has a broken link. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28068) Add hbase.normalizer.merge.merge_request_max_number_of_regions property to limit max number of regions in a merge request for merge normalization
[
https://issues.apache.org/jira/browse/HBASE-28068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-28068.
--
Resolution: Fixed
Addendums applied.
> Add hbase.normalizer.merge.merge_request_max_number_of_regions property to
> limit max number of regions in a merge request for merge normalization
> -
>
> Key: HBASE-28068
> URL: https://issues.apache.org/jira/browse/HBASE-28068
> Project: HBase
> Issue Type: Improvement
> Components: Normalizer
>Affects Versions: 2.4.0, 2.5.0, 2.6.0, 3.0.0-alpha-4, 4.0.0-alpha-1
>Reporter: Ravi Kishore Valeti
>Assignee: Rahul Kumar
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
>
>
> In our production environment, while investigating an issue, we observed that
> the Noramlizer had scheduled one single merge procedure to an RS providing
> 27K+ empty regions of a table (this was a result of a failed copy table job
> that left 27K+ empty regions of the table) to merge.
> This action led the procedure to go to stuck state and eventually the
> procedure framework bailed out after ~40mins. This was happening with each
> normalizer run until we deleted the table manually.
> Logs
> Normalizer triggers a merge procedure
> normalizer.RegionNormalizerWorker - NormalizationTarget[regionInfo=\{ENCODED
> => 6e8606335a62f6bafceb017dc7edfdf5, NAME => 'TEST.TEST_TABLE,.',
> STARTKEY => '', ENDKEY => ''},{*}regionSizeMb=0{*}],
> NormalizationTarget[regionInfo=\{ENCODED => 79607df308d7618e632abe8a12c1bf6b,
> NAME => 'TEST.TEST_TABLE,', STARTKEY => 'XXYY', ENDKEY =>
> 'YYZZ'},{*}regionSizeMb=0]{*}]] resulting in *pid 21968356*
> procedure immediately gets stuck
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time 12.4850 sec
> Finally fails after ~40 mins
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time *40 mins, 58.055 sec*
> Bails out with RuntimeException
> procedure2.ProcedureExecutor - force=false
> java.lang.UnsupportedOperationException: pid=21968356,
> state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META, locked=true,
> exception=java.lang.{*}RuntimeException via CODE-BUG: Uncaught runtime
> exception{*}: pid=21968356, state=RUNNABLE:MERGE_TABLE_REGIONS_UPDATE_META,
> locked=true; MergeTableRegionsProcedure table=TEST.TEST_TABLE,
> {*}regions={*}{*}[269a1b168af497cce9ba6d3d581568f2{*}
> .
> .
> .
> .
> *27K+ regions printed here]*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (HBASE-28068) Add hbase.normalizer.merge.merge_request_max_number_of_regions property to limit max number of regions in a merge request for merge normalization
[
https://issues.apache.org/jira/browse/HBASE-28068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-28068:
--
> Add hbase.normalizer.merge.merge_request_max_number_of_regions property to
> limit max number of regions in a merge request for merge normalization
> -
>
> Key: HBASE-28068
> URL: https://issues.apache.org/jira/browse/HBASE-28068
> Project: HBase
> Issue Type: Improvement
> Components: Normalizer
>Affects Versions: 2.4.0, 2.5.0, 2.6.0, 3.0.0-alpha-4, 4.0.0-alpha-1
>Reporter: Ravi Kishore Valeti
>Assignee: Rahul Kumar
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
>
>
> In our production environment, while investigating an issue, we observed that
> the Noramlizer had scheduled one single merge procedure to an RS providing
> 27K+ empty regions of a table (this was a result of a failed copy table job
> that left 27K+ empty regions of the table) to merge.
> This action led the procedure to go to stuck state and eventually the
> procedure framework bailed out after ~40mins. This was happening with each
> normalizer run until we deleted the table manually.
> Logs
> Normalizer triggers a merge procedure
> normalizer.RegionNormalizerWorker - NormalizationTarget[regionInfo=\{ENCODED
> => 6e8606335a62f6bafceb017dc7edfdf5, NAME => 'TEST.TEST_TABLE,.',
> STARTKEY => '', ENDKEY => ''},{*}regionSizeMb=0{*}],
> NormalizationTarget[regionInfo=\{ENCODED => 79607df308d7618e632abe8a12c1bf6b,
> NAME => 'TEST.TEST_TABLE,', STARTKEY => 'XXYY', ENDKEY =>
> 'YYZZ'},{*}regionSizeMb=0]{*}]] resulting in *pid 21968356*
> procedure immediately gets stuck
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time 12.4850 sec
> Finally fails after ~40 mins
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time *40 mins, 58.055 sec*
> Bails out with RuntimeException
> procedure2.ProcedureExecutor - force=false
> java.lang.UnsupportedOperationException: pid=21968356,
> state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META, locked=true,
> exception=java.lang.{*}RuntimeException via CODE-BUG: Uncaught runtime
> exception{*}: pid=21968356, state=RUNNABLE:MERGE_TABLE_REGIONS_UPDATE_META,
> locked=true; MergeTableRegionsProcedure table=TEST.TEST_TABLE,
> {*}regions={*}{*}[269a1b168af497cce9ba6d3d581568f2{*}
> .
> .
> .
> .
> *27K+ regions printed here]*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28068) Add hbase.normalizer.merge.merge_request_max_number_of_regions property to limit max number of regions in a merge request for merge normalization
[
https://issues.apache.org/jira/browse/HBASE-28068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-28068.
--
Resolution: Fixed
> Add hbase.normalizer.merge.merge_request_max_number_of_regions property to
> limit max number of regions in a merge request for merge normalization
> -
>
> Key: HBASE-28068
> URL: https://issues.apache.org/jira/browse/HBASE-28068
> Project: HBase
> Issue Type: Improvement
> Components: Normalizer
>Affects Versions: 2.4.0, 2.5.0, 2.6.0, 3.0.0-alpha-4, 4.0.0-alpha-1
>Reporter: Ravi Kishore Valeti
>Assignee: Rahul Kumar
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
>
>
> In our production environment, while investigating an issue, we observed that
> the Noramlizer had scheduled one single merge procedure to an RS providing
> 27K+ empty regions of a table (this was a result of a failed copy table job
> that left 27K+ empty regions of the table) to merge.
> This action led the procedure to go to stuck state and eventually the
> procedure framework bailed out after ~40mins. This was happening with each
> normalizer run until we deleted the table manually.
> Logs
> Normalizer triggers a merge procedure
> normalizer.RegionNormalizerWorker - NormalizationTarget[regionInfo=\{ENCODED
> => 6e8606335a62f6bafceb017dc7edfdf5, NAME => 'TEST.TEST_TABLE,.',
> STARTKEY => '', ENDKEY => ''},{*}regionSizeMb=0{*}],
> NormalizationTarget[regionInfo=\{ENCODED => 79607df308d7618e632abe8a12c1bf6b,
> NAME => 'TEST.TEST_TABLE,', STARTKEY => 'XXYY', ENDKEY =>
> 'YYZZ'},{*}regionSizeMb=0]{*}]] resulting in *pid 21968356*
> procedure immediately gets stuck
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time 12.4850 sec
> Finally fails after ~40 mins
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time *40 mins, 58.055 sec*
> Bails out with RuntimeException
> procedure2.ProcedureExecutor - force=false
> java.lang.UnsupportedOperationException: pid=21968356,
> state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META, locked=true,
> exception=java.lang.{*}RuntimeException via CODE-BUG: Uncaught runtime
> exception{*}: pid=21968356, state=RUNNABLE:MERGE_TABLE_REGIONS_UPDATE_META,
> locked=true; MergeTableRegionsProcedure table=TEST.TEST_TABLE,
> {*}regions={*}{*}[269a1b168af497cce9ba6d3d581568f2{*}
> .
> .
> .
> .
> *27K+ regions printed here]*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (HBASE-28068) Add hbase.normalizer.merge.merge_request_max_number_of_regions property to limit max number of regions in a merge request for merge normalization
[
https://issues.apache.org/jira/browse/HBASE-28068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-28068:
--
Re-opening for branch-2.4 backport.
> Add hbase.normalizer.merge.merge_request_max_number_of_regions property to
> limit max number of regions in a merge request for merge normalization
> -
>
> Key: HBASE-28068
> URL: https://issues.apache.org/jira/browse/HBASE-28068
> Project: HBase
> Issue Type: Improvement
> Components: Normalizer
>Affects Versions: 2.4.0, 2.5.0, 2.6.0, 3.0.0-alpha-4, 4.0.0-alpha-1
>Reporter: Ravi Kishore Valeti
>Assignee: Rahul Kumar
>Priority: Minor
> Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
>
>
> In our production environment, while investigating an issue, we observed that
> the Noramlizer had scheduled one single merge procedure to an RS providing
> 27K+ empty regions of a table (this was a result of a failed copy table job
> that left 27K+ empty regions of the table) to merge.
> This action led the procedure to go to stuck state and eventually the
> procedure framework bailed out after ~40mins. This was happening with each
> normalizer run until we deleted the table manually.
> Logs
> Normalizer triggers a merge procedure
> normalizer.RegionNormalizerWorker - NormalizationTarget[regionInfo=\{ENCODED
> => 6e8606335a62f6bafceb017dc7edfdf5, NAME => 'TEST.TEST_TABLE,.',
> STARTKEY => '', ENDKEY => ''},{*}regionSizeMb=0{*}],
> NormalizationTarget[regionInfo=\{ENCODED => 79607df308d7618e632abe8a12c1bf6b,
> NAME => 'TEST.TEST_TABLE,', STARTKEY => 'XXYY', ENDKEY =>
> 'YYZZ'},{*}regionSizeMb=0]{*}]] resulting in *pid 21968356*
> procedure immediately gets stuck
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time 12.4850 sec
> Finally fails after ~40 mins
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time *40 mins, 58.055 sec*
> Bails out with RuntimeException
> procedure2.ProcedureExecutor - force=false
> java.lang.UnsupportedOperationException: pid=21968356,
> state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META, locked=true,
> exception=java.lang.{*}RuntimeException via CODE-BUG: Uncaught runtime
> exception{*}: pid=21968356, state=RUNNABLE:MERGE_TABLE_REGIONS_UPDATE_META,
> locked=true; MergeTableRegionsProcedure table=TEST.TEST_TABLE,
> {*}regions={*}{*}[269a1b168af497cce9ba6d3d581568f2{*}
> .
> .
> .
> .
> *27K+ regions printed here]*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28077) Limit the maximum allocation when reading an HFileBlock
Nick Dimiduk created HBASE-28077:
Summary: Limit the maximum allocation when reading an HFileBlock
Key: HBASE-28077
URL: https://issues.apache.org/jira/browse/HBASE-28077
Project: HBase
Issue Type: Improvement
Components: regionserver
Reporter: Nick Dimiduk
During PR discussion on HBASE-28065 we observe that the value of
{{onDiskSizeWithoutHeader}} is read and used before its portion of an HFile has
had its checksum validated. A method parameter is also provided which is used
when the caller knows what size to expect based on some other source. While
there are guards in place that limit the range of values this field can take,
that range remains large, something like {{[33,Integer.MAX_VALUE)}}. We propose
further limiting the range of this value to safeguard the region server from an
excessively large allocation.
Conversation is in https://github.com/apache/hbase/pull/5384/files#r1322947549
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-27114) Upgrade scalatest maven plugin for thread-safety
[
https://issues.apache.org/jira/browse/HBASE-27114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-27114.
--
Fix Version/s: hbase-connectors-1.0.1
Resolution: Fixed
[~busbey] I think we need a new ticket for the issue you've described.
> Upgrade scalatest maven plugin for thread-safety
>
>
> Key: HBASE-27114
> URL: https://issues.apache.org/jira/browse/HBASE-27114
> Project: HBase
> Issue Type: Task
> Components: build, spark
>Affects Versions: hbase-connectors-1.0.1, hbase-connectors-1.1.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: hbase-connectors-1.0.1
>
>
> The {{master}} branch on the connectors repo warns when {{--threads}} is
> issued, the complaint being the scalatest-maven-plugin. Looks like the latest
> version resolves the complaint. Let's upgrade.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28065) Corrupt HFile data is mishandled in several cases
Nick Dimiduk created HBASE-28065:
Summary: Corrupt HFile data is mishandled in several cases
Key: HBASE-28065
URL: https://issues.apache.org/jira/browse/HBASE-28065
Project: HBase
Issue Type: Bug
Components: HFile
Affects Versions: 2.5.2
Reporter: Nick Dimiduk
While riding over a spat of HDFS data corruption issues, we've observed several
places in the read path that do not fall back to HDFS checksum appropriately.
These failures manifest during client reads and during compactions. Sometimes
failure is detected by the fallback {{verifyOnDiskSizeMatchesHeader}},
sometimes we attempt to allocate a buffer with a negative size, and sometimes
we read through to a failure from block decompression.
After code study, I think that all three cases arise from using a block header
that was read without checksum validation.
Will post up the stack traces in the comments. Now sure if we'll want a single
patch or multiple.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27946) Introduce HA hdfs+hbase colocated pod definition
Nick Dimiduk created HBASE-27946: Summary: Introduce HA hdfs+hbase colocated pod definition Key: HBASE-27946 URL: https://issues.apache.org/jira/browse/HBASE-27946 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Like the hdfs+hbase colocated pod definition but with an HA deployment strategy. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27945) Introduce hdfs+hbase colocated pod definition
Nick Dimiduk created HBASE-27945: Summary: Introduce hdfs+hbase colocated pod definition Key: HBASE-27945 URL: https://issues.apache.org/jira/browse/HBASE-27945 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Implement a deployment strategy that supports short-circuit reads by forcing the data node and region server processes to colocate by deployment them as sibling container within the same pod. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27943) Rework kuttl image as a yetus precommit base
Nick Dimiduk created HBASE-27943: Summary: Rework kuttl image as a yetus precommit base Key: HBASE-27943 URL: https://issues.apache.org/jira/browse/HBASE-27943 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Due to permissions issues on build workers (INFRA-24621), I think we'll benefit by shifting the perspective a little on how the kuttl image is used in CI. The current PRs establish the kuttl image as a (relatively) small, self-contained utility image. When it is invoked in CI, we run yetus with docker-in-docker support, and launch tests by calling `docker container run ... kuttl ...`. So, docker is invoked from within the Yetus pre-commit docker container. Rather, I want to implement the kuttl image as extending from the yetus image (or yetus-base, I'm not sure yet). That way we don't need to run pre-commit with docker-in-docker mode and the precommit process can invoke kuttl directly. The existing kuttl image build is too sophisticated to be invoked by Yetus as a provided docker image (it makes use of build-args). It may also be too sophisticated to be run in a Jenkins worker (it makes use of buildx, INFRA-24704). So I guess we need to publish the image someplace and use the published tag via the Yetus `--docker-tag` flag. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27935) Introduce Jenkins PR job for hbase-kustomize
[ https://issues.apache.org/jira/browse/HBASE-27935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-27935. -- Resolution: Fixed > Introduce Jenkins PR job for hbase-kustomize > > > Key: HBASE-27935 > URL: https://issues.apache.org/jira/browse/HBASE-27935 > Project: HBase > Issue Type: Task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > We need something to build off of. Let's start with a clone of what's on > hbase-operator-tools. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27935) Introduce Jenkins PR job for hbase-kustomize
Nick Dimiduk created HBASE-27935: Summary: Introduce Jenkins PR job for hbase-kustomize Key: HBASE-27935 URL: https://issues.apache.org/jira/browse/HBASE-27935 Project: HBase Issue Type: Task Components: build Reporter: Nick Dimiduk Assignee: Nick Dimiduk We need something to build off of. Let's start with a clone of what's on hbase-operator-tools. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27929) Add a LICENSE file to hbase-kustomize.git
[ https://issues.apache.org/jira/browse/HBASE-27929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-27929. -- Resolution: Fixed > Add a LICENSE file to hbase-kustomize.git > - > > Key: HBASE-27929 > URL: https://issues.apache.org/jira/browse/HBASE-27929 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Using this for an initial commit, something non-controversial so that we can > have an initial commit in place. Some commit history is required before PRs > can be opened against a repository, hence doing this a little outside of our > normal process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27930) Add .asf.yaml to hbase-kustomize.git
Nick Dimiduk created HBASE-27930: Summary: Add .asf.yaml to hbase-kustomize.git Key: HBASE-27930 URL: https://issues.apache.org/jira/browse/HBASE-27930 Project: HBase Issue Type: Task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Establish some basic configurations for this new git repository. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27929) Add a LICENSE file to hbase-kustomize.git
Nick Dimiduk created HBASE-27929: Summary: Add a LICENSE file to hbase-kustomize.git Key: HBASE-27929 URL: https://issues.apache.org/jira/browse/HBASE-27929 Project: HBase Issue Type: Task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Using this for an initial commit, something non-controversial so that we can have an initial commit in place. Some commit history is required before PRs can be opened against a repository, hence doing this a little outside of our normal process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27856) Add hadolint binary to operator-tools yetus environment
Nick Dimiduk created HBASE-27856: Summary: Add hadolint binary to operator-tools yetus environment Key: HBASE-27856 URL: https://issues.apache.org/jira/browse/HBASE-27856 Project: HBase Issue Type: Task Components: build Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: hbase-operator-tools-1.3.0 Since we're adding dockerfiles via HBASE-27827, let's also have a pre-commit check for them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27847) Introduce HBase image
Nick Dimiduk created HBASE-27847: Summary: Introduce HBase image Key: HBASE-27847 URL: https://issues.apache.org/jira/browse/HBASE-27847 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: HBASE-27827 As with the Hadoop image (HBASE-27846), we need a runtime image for the HBase containers, and we need a place to define an API between the runtime image and the orchestration layer. HBase project doesn't ship an image yet, so this will provide double-duty. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27846) Introduce hadoop image
Nick Dimiduk created HBASE-27846: Summary: Introduce hadoop image Key: HBASE-27846 URL: https://issues.apache.org/jira/browse/HBASE-27846 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk The image shipped by upstream requires some tweaks. Extend it to suit our needs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27837) CI integration for the integration test suite
Nick Dimiduk created HBASE-27837: Summary: CI integration for the integration test suite Key: HBASE-27837 URL: https://issues.apache.org/jira/browse/HBASE-27837 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Fix For: HBASE-27827 Following HBASE-27836 , we should work out how to get CI running the integration tests, and a cluster to run them against. KinD or minikube maybe? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27836) CI Integration for unit test suite
Nick Dimiduk created HBASE-27836: Summary: CI Integration for unit test suite Key: HBASE-27836 URL: https://issues.apache.org/jira/browse/HBASE-27836 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Fix For: HBASE-27827 We should figure out how to tie at least the unit tests into CI, either via the normal maven lifecycle or as an external thing yetus pre-commit knows about. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27835) introduce ha-hbase overlay
Nick Dimiduk created HBASE-27835: Summary: introduce ha-hbase overlay Key: HBASE-27835 URL: https://issues.apache.org/jira/browse/HBASE-27835 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Extend the HBase deployment for HA. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27831) introduce zookeeper-single-instance component
Nick Dimiduk created HBASE-27831: Summary: introduce zookeeper-single-instance component Key: HBASE-27831 URL: https://issues.apache.org/jira/browse/HBASE-27831 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Provide a basic zookeeper deployment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27833) introduce zookeeper-ha_ensemble component
Nick Dimiduk created HBASE-27833: Summary: introduce zookeeper-ha_ensemble component Key: HBASE-27833 URL: https://issues.apache.org/jira/browse/HBASE-27833 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Extend the zookeeper deployment for HA. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27834) introduce ha-hdfs overlay
Nick Dimiduk created HBASE-27834: Summary: introduce ha-hdfs overlay Key: HBASE-27834 URL: https://issues.apache.org/jira/browse/HBASE-27834 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Extend the HDFS deployment for HA. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27830) introduce hdfs overlay
Nick Dimiduk created HBASE-27830: Summary: introduce hdfs overlay Key: HBASE-27830 URL: https://issues.apache.org/jira/browse/HBASE-27830 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Provide a basic implementation HDFS deployment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27832) introduce hbase overlay
Nick Dimiduk created HBASE-27832: Summary: introduce hbase overlay Key: HBASE-27832 URL: https://issues.apache.org/jira/browse/HBASE-27832 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Provide a basic HBase deployment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27828) introduce hbase-kubernetes-deployment module
Nick Dimiduk created HBASE-27828: Summary: introduce hbase-kubernetes-deployment module Key: HBASE-27828 URL: https://issues.apache.org/jira/browse/HBASE-27828 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Add a maven module under which the orchestration tools can reside. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27829) introduce build for `kuttl` image, basis for dev/test environment
Nick Dimiduk created HBASE-27829: Summary: introduce build for `kuttl` image, basis for dev/test environment Key: HBASE-27829 URL: https://issues.apache.org/jira/browse/HBASE-27829 Project: HBase Issue Type: Sub-task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Define a starting image for dev and test. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27827) Introduce kubernetes deployment
Nick Dimiduk created HBASE-27827: Summary: Introduce kubernetes deployment Key: HBASE-27827 URL: https://issues.apache.org/jira/browse/HBASE-27827 Project: HBase Issue Type: New Feature Reporter: Nick Dimiduk Assignee: Nick Dimiduk As per the [discussion|https://lists.apache.org/thread/fgxyk4y32xnhzr5prdmhfjkfpk15g5jx] on the dev list, introduce a basic harness for deploying ZooKeeper, HDFS, and HBase on Kubernetes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27777) bin/hbase --help does not list "omnibus_tatball" options
Nick Dimiduk created HBASE-2:
Summary: bin/hbase --help does not list "omnibus_tatball" options
Key: HBASE-2
URL: https://issues.apache.org/jira/browse/HBASE-2
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 2.4.17
Reporter: Nick Dimiduk
Launching {{bin/hbase --help}} from the 2.4.17RC0 full distribution tarball, I
see a limited set of options. It looks like we do not source hbase-config.sh
before printing the help message, which means {{HBASE_HOME}} is not set, and we
don't get the extended output.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27774) Move Dockerfile and python-requirements.txt used only by flaky-tests job
Nick Dimiduk created HBASE-27774: Summary: Move Dockerfile and python-requirements.txt used only by flaky-tests job Key: HBASE-27774 URL: https://issues.apache.org/jira/browse/HBASE-27774 Project: HBase Issue Type: Task Components: build, community, documentation Affects Versions: 3.0.0-alpha-4 Reporter: Nick Dimiduk Assignee: Nick Dimiduk We have a Dockerfile floating around in dev-support. It looks like it's only used by the flaky-test Jenkins job, so move it under that directory. I think that it used to be used in multiple places, as way to package up the dev-support/python-requirements.txt file. However, that too seems to only be used by the flaky-tests job, so move it as well. Update the job for the new invocation form, and remove mentions of python-requirements.txt from the book -- an old, out-dated section about how to format patches for attachment to JIRA. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27762) Include EventType and ProcedureV2 pid in logging via MDC
Nick Dimiduk created HBASE-27762:
Summary: Include EventType and ProcedureV2 pid in logging via MDC
Key: HBASE-27762
URL: https://issues.apache.org/jira/browse/HBASE-27762
Project: HBase
Issue Type: Task
Affects Versions: 2.6.0, 3.0.0-alpha-4
Reporter: Nick Dimiduk
Tracing the distributed actions of ProcedureV2 {{RemoteProcedure}} execution is
painful. We are pretty good about logging the {{proc_id}} most of the time, but
we don't catch them all. Now that we're up on log4j2, let's use the MDC feature
to include some basic information universally.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27761) Replication threads not attached to their parent process
Nick Dimiduk created HBASE-27761: Summary: Replication threads not attached to their parent process Key: HBASE-27761 URL: https://issues.apache.org/jira/browse/HBASE-27761 Project: HBase Issue Type: Task Components: read replicas, regionserver, Replication Affects Versions: 2.5.4 Reporter: Nick Dimiduk While debugging HBASE-27707 in a unit test, I see behaviour that I cannot explain. My test uses a minicluster, enables read replica replication, writes some data, concurrently kills a region server thread hosting a primary region, and then verifies that all replicas eventually show all data. Inspecting logs, noticed that replication source threads seem to continue working even after their associated region server is killed. Interspersing some thread dumps and sleeps, I can see that replication threads associated with the condemned region server are not being removed after it is killed. I think that this behaviour will render unreliably any replication test that relies on killing a source or sink region server. It also implies to me that the minicluster leaks replication threads and cannot be reliably recycled within a single jvm process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27741) Fall back to protoc osx-x86_64 on Apple Silicon
Nick Dimiduk created HBASE-27741: Summary: Fall back to protoc osx-x86_64 on Apple Silicon Key: HBASE-27741 URL: https://issues.apache.org/jira/browse/HBASE-27741 Project: HBase Issue Type: Task Components: build Affects Versions: 2.5.0, 2.6.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Building non-master branches on an Apple Silicon machine fails because there's no protoc binary available. Use a profile to fall back to the x86 version of the binary, as per https://cwiki.apache.org/confluence/display/HADOOP/Develop+on+Apple+Silicon+%28M1%29+macOS -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27737) Add supplemental model for com.aayushatharva.brotli4j:native-osx-aarch64
Nick Dimiduk created HBASE-27737: Summary: Add supplemental model for com.aayushatharva.brotli4j:native-osx-aarch64 Key: HBASE-27737 URL: https://issues.apache.org/jira/browse/HBASE-27737 Project: HBase Issue Type: Task Components: build, community Reporter: Nick Dimiduk Assignee: Nick Dimiduk License aggregation fails on Apple Silicon because we're missing the supplemental model entry for this architecture's brotli4j implementation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27720) TestClusterRestartFailover is flakey
Nick Dimiduk created HBASE-27720:
Summary: TestClusterRestartFailover is flakey
Key: HBASE-27720
URL: https://issues.apache.org/jira/browse/HBASE-27720
Project: HBase
Issue Type: Task
Components: test
Affects Versions: 2.5.4
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
I'm seeing failures like this in PR,
{noformat}
[ERROR] Failures:
[ERROR]
org.apache.hadoop.hbase.master.TestClusterRestartFailoverSplitWithoutZk.test
[ERROR] Run 1:
TestClusterRestartFailoverSplitWithoutZk>TestClusterRestartFailover.test:143
serverNode should be deleted after SCP finished expected null, but
was:
[ERROR] Run 2:
TestClusterRestartFailoverSplitWithoutZk>TestClusterRestartFailover.test:147
serverCrashSubmittedCount(8) should be equal expected:<4> but was:<8>
[ERROR] Run 3:
TestClusterRestartFailoverSplitWithoutZk>TestClusterRestartFailover.test:147
serverCrashSubmittedCount(12) should be equal expected:<4> but was:<12>
{noformat}
Looks like subsequent runs would have passed, but for the firm metric count
assertion.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27719) Fix surefire corrupted channel
Nick Dimiduk created HBASE-27719:
Summary: Fix surefire corrupted channel
Key: HBASE-27719
URL: https://issues.apache.org/jira/browse/HBASE-27719
Project: HBase
Issue Type: Task
Components: build, test
Reporter: Nick Dimiduk
Testing, at least with hbase-server, emits a warning,
{noformat}
[WARNING] Corrupted channel by directly writing to native stream in forked JVM
1. See FAQ web page and the dump file
/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5036/yetus-jdk8-hadoop2-check/src/hbase-server/target/surefire-reports/2023-03-14T13-20-30_932-jvmRun1.dumpstream
{noformat}
I cannot tell if this is causing maven to recognize the test run as a failure,
though in this case, the test run was marked as failing, with two tests rerun
and passing on their second attempt.
Surefire apparently uses stdio to communicate with maven, so when our code
under test produces output on the channel, it corrupts that communication.
https://maven.apache.org/surefire/maven-surefire-plugin/examples/process-communication.html#the-tcp-ip-communication-channel
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27708) Thread contention resolving User subject with tracing disabled
Nick Dimiduk created HBASE-27708: Summary: Thread contention resolving User subject with tracing disabled Key: HBASE-27708 URL: https://issues.apache.org/jira/browse/HBASE-27708 Project: HBase Issue Type: Bug Components: Client, tracing Affects Versions: 2.5.0 Reporter: Nick Dimiduk Even with OpenTelemetry tracing disabled, we see contention related to populating the string representation of the User principle on the client side. Can HBase connection cache this? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27707) Region replica replication sometimes orphans WAL queue entries during recovery
Nick Dimiduk created HBASE-27707:
Summary: Region replica replication sometimes orphans WAL queue
entries during recovery
Key: HBASE-27707
URL: https://issues.apache.org/jira/browse/HBASE-27707
Project: HBase
Issue Type: Bug
Components: read replicas, Replication
Affects Versions: 2.5.0
Reporter: Nick Dimiduk
Running with timeline-consistent read replicas and
{{hbase.region.replica.replication.enabled=true}}, we're seeing some region
servers have WAL queue entires that never clear. This appears to correlate with
SCP and recovery of replication queues. The result is WALs that build up,
consuming dangerous amounts of space on HDFS. Remediation requires disabling
and removing the {{region_replica_replication}} peer, which forces an impacted
region server to abort with the message "Failed to operate on replication
queue". We then delete the zk entry, which unlocks the WAL and the cleaner
chore can sweep them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27696) [hbase-operator-tools] Use $revision as placeholder for maven version
Nick Dimiduk created HBASE-27696: Summary: [hbase-operator-tools] Use $revision as placeholder for maven version Key: HBASE-27696 URL: https://issues.apache.org/jira/browse/HBASE-27696 Project: HBase Issue Type: Task Components: build, pom Reporter: Nick Dimiduk Assignee: Nick Dimiduk To align with our main repo. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27695) Standby masters appear to not purge their old write-ahead logs
Nick Dimiduk created HBASE-27695: Summary: Standby masters appear to not purge their old write-ahead logs Key: HBASE-27695 URL: https://issues.apache.org/jira/browse/HBASE-27695 Project: HBase Issue Type: Bug Components: master Affects Versions: 2.5.0 Reporter: Nick Dimiduk Dropping this one here, haven't investigated yet. We noticed that on a cluster who's elected master had been active for quite a long time that the master store wal directory steadily grew in size. Failing over master cause the directory to be cleaned up. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-23983) Spotbugs warning complain on master build
[
https://issues.apache.org/jira/browse/HBASE-23983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-23983.
--
Resolution: Fixed
> Spotbugs warning complain on master build
> -
>
> Key: HBASE-23983
> URL: https://issues.apache.org/jira/browse/HBASE-23983
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Major
> Fix For: 2.6.0, 2.5.4, 3.0.0-alpha-1
>
>
> Spotbugs currently complains on every master build with an extant warning
> related:
> {code}
> Return value of putIfAbsent is ignored, but node is reused in
> org.apache.hadoop.hbase.master.assignment.RegionStates.createRegionStateNode(RegionInfo)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-23561) Look up of Region in Master by encoded region name is O(n)
[
https://issues.apache.org/jira/browse/HBASE-23561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-23561.
--
Resolution: Fixed
> Look up of Region in Master by encoded region name is O(n)
> --
>
> Key: HBASE-23561
> URL: https://issues.apache.org/jira/browse/HBASE-23561
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1
>Reporter: Michael Stack
>Assignee: Minwoo Kang
>Priority: Trivial
> Fix For: 2.6.0, 2.5.4, 3.0.0-alpha-1
>
>
> {{ public RegionState getRegionState(final String encodedRegionName) {
> // TODO: Need a map but it is just dispatch merge...
> for (RegionStateNode node: regionsMap.values()) {
> if (node.getRegionInfo().getEncodedName().equals(encodedRegionName)) {
> return node.toRegionState();
> }
> }
> return null;
> }}}
> It is not used much so making it trivial.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (HBASE-23983) Spotbugs warning complain on master build
[
https://issues.apache.org/jira/browse/HBASE-23983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-23983:
--
Reopening for backport.
> Spotbugs warning complain on master build
> -
>
> Key: HBASE-23983
> URL: https://issues.apache.org/jira/browse/HBASE-23983
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> Spotbugs currently complains on every master build with an extant warning
> related:
> {code}
> Return value of putIfAbsent is ignored, but node is reused in
> org.apache.hadoop.hbase.master.assignment.RegionStates.createRegionStateNode(RegionInfo)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-23102) Improper Usage of Map putIfAbsent
[
https://issues.apache.org/jira/browse/HBASE-23102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-23102.
--
Resolution: Fixed
Backported to 2.6 and 2.5.
> Improper Usage of Map putIfAbsent
> -
>
> Key: HBASE-23102
> URL: https://issues.apache.org/jira/browse/HBASE-23102
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.3.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: newbie, noob
> Fix For: 2.6.0, 2.5.4, 3.0.0-alpha-1
>
> Attachments: HBASE-23102.1.patch
>
>
> When using {{Map#putIfAbsent}}, the argument should not be a {{new}} object.
> Otherwise, if the item is present, the object that was instantiated is
> immediately thrown away. Instead, use {{Map#computeIfAbsent}} so that the
> object is only instantiated if it is needed.
> There exists a good example in the {{Map}} JavaDoc:
> https://docs.oracle.com/javase/8/docs/api/java/util/Map.html#computeIfAbsent-K-java.util.function.Function-
>
> h2. Locations
> https://github.com/apache/hbase/blob/9370347efea5b09e2fa8f4e5d82fa32491e1181b/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java#L227-L236
> https://github.com/apache/hbase/blob/025ddce868eb06b4072b5152c5ffae5a01e7ae30/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/throttle/StoreHotnessProtector.java#L124-L129
> https://github.com/apache/hbase/blob/1170f28122d9d36e511ba504a5263ec62e11ef6a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java#L555
> https://github.com/apache/hbase/blob/4ca760fe9dd373b8d8a4c48db15e42424920653c/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java#L584-L586
> https://github.com/apache/hbase/blob/4ca760fe9dd373b8d8a4c48db15e42424920653c/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java#L585
> https://github.com/apache/hbase/blob/5b01e613fbbb92e243e99a1d199b4ffbb21ed2d9/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java#L834
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (HBASE-23561) Look up of Region in Master by encoded region name is O(n)
[
https://issues.apache.org/jira/browse/HBASE-23561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-23561:
--
Reopen for back-ports.
> Look up of Region in Master by encoded region name is O(n)
> --
>
> Key: HBASE-23561
> URL: https://issues.apache.org/jira/browse/HBASE-23561
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1
>Reporter: Michael Stack
>Assignee: Minwoo Kang
>Priority: Trivial
> Fix For: 3.0.0-alpha-1
>
>
> {{ public RegionState getRegionState(final String encodedRegionName) {
> // TODO: Need a map but it is just dispatch merge...
> for (RegionStateNode node: regionsMap.values()) {
> if (node.getRegionInfo().getEncodedName().equals(encodedRegionName)) {
> return node.toRegionState();
> }
> }
> return null;
> }}}
> It is not used much so making it trivial.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (HBASE-23102) Improper Usage of Map putIfAbsent
[
https://issues.apache.org/jira/browse/HBASE-23102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-23102:
--
Reopen for back-ports.
> Improper Usage of Map putIfAbsent
> -
>
> Key: HBASE-23102
> URL: https://issues.apache.org/jira/browse/HBASE-23102
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.3.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: newbie, noob
> Fix For: 3.0.0-alpha-1
>
> Attachments: HBASE-23102.1.patch
>
>
> When using {{Map#putIfAbsent}}, the argument should not be a {{new}} object.
> Otherwise, if the item is present, the object that was instantiated is
> immediately thrown away. Instead, use {{Map#computeIfAbsent}} so that the
> object is only instantiated if it is needed.
> There exists a good example in the {{Map}} JavaDoc:
> https://docs.oracle.com/javase/8/docs/api/java/util/Map.html#computeIfAbsent-K-java.util.function.Function-
>
> h2. Locations
> https://github.com/apache/hbase/blob/9370347efea5b09e2fa8f4e5d82fa32491e1181b/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java#L227-L236
> https://github.com/apache/hbase/blob/025ddce868eb06b4072b5152c5ffae5a01e7ae30/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/throttle/StoreHotnessProtector.java#L124-L129
> https://github.com/apache/hbase/blob/1170f28122d9d36e511ba504a5263ec62e11ef6a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java#L555
> https://github.com/apache/hbase/blob/4ca760fe9dd373b8d8a4c48db15e42424920653c/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java#L584-L586
> https://github.com/apache/hbase/blob/4ca760fe9dd373b8d8a4c48db15e42424920653c/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java#L585
> https://github.com/apache/hbase/blob/5b01e613fbbb92e243e99a1d199b4ffbb21ed2d9/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java#L834
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27652) Client-side lock contention around Configuration when using read replica regions
Nick Dimiduk created HBASE-27652: Summary: Client-side lock contention around Configuration when using read replica regions Key: HBASE-27652 URL: https://issues.apache.org/jira/browse/HBASE-27652 Project: HBase Issue Type: Bug Components: Client, read replicas Affects Versions: 2.5.1 Reporter: Nick Dimiduk Since upgrading to 2.5.1 our client-side application has noticed lock contention. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27651) hbase-daemon.sh foreground_start does not propagate SIGHUP or SIGTERM properly
Nick Dimiduk created HBASE-27651:
Summary: hbase-daemon.sh foreground_start does not propagate
SIGHUP or SIGTERM properly
Key: HBASE-27651
URL: https://issues.apache.org/jira/browse/HBASE-27651
Project: HBase
Issue Type: Bug
Components: scripts
Reporter: Nick Dimiduk
Running HBase in a container environment, launched using
{{{}bin/hbase-daemon.sh foreground_start ...{}}}. Signaling the container for
config refresh or for graceful termination doesn't make it through to the JVM
process. Rather, the {{trap}} function is unconditionally sending {{{}kill
-9{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27634) Builds emit errors related to SBOM parsing
Nick Dimiduk created HBASE-27634:
Summary: Builds emit errors related to SBOM parsing
Key: HBASE-27634
URL: https://issues.apache.org/jira/browse/HBASE-27634
Project: HBase
Issue Type: Task
Components: build
Affects Versions: 3.0.0-alpha-4
Reporter: Nick Dimiduk
My build is littered with noise of the form of,
{noformat}
[INFO] CycloneDX: Resolving Dependencies
[ERROR] An error occurred attempting to read POM
org.codehaus.plexus.util.xml.pull.XmlPullParserException: UTF-8 BOM plus xml
decl of ISO-8859-1 is incompatible (position: START_DOCUMENT seen
[jira] [Created] (HBASE-27616) Document upgrade to latest 2.4/5 in the Book
Nick Dimiduk created HBASE-27616: Summary: Document upgrade to latest 2.4/5 in the Book Key: HBASE-27616 URL: https://issues.apache.org/jira/browse/HBASE-27616 Project: HBase Issue Type: Task Components: community, documentation Reporter: Nick Dimiduk Let's review the upgrade process from 2.3 to 2.4 and 2.5, document any operator actions in The Book. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27574) Implement ClusterManage interface for Kubernetes
Nick Dimiduk created HBASE-27574:
Summary: Implement ClusterManage interface for Kubernetes
Key: HBASE-27574
URL: https://issues.apache.org/jira/browse/HBASE-27574
Project: HBase
Issue Type: Improvement
Components: integration tests
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Implement the {{ClusterManager}} interface enough such that we can run the
ChaosMonkey against a deployment on Kubernetes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27568) ChaosMonkey add support for JournalNodes
Nick Dimiduk created HBASE-27568: Summary: ChaosMonkey add support for JournalNodes Key: HBASE-27568 URL: https://issues.apache.org/jira/browse/HBASE-27568 Project: HBase Issue Type: Task Components: integration tests Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.16, 2.5.3 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Looks like our ChaosMonkey knowledge of HDFS topology is stale. Add the concept of a Journal Node to ClusterManager et al. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27567) Introduce ChaosMonkey Action to print HDFS Cluster status
Nick Dimiduk created HBASE-27567: Summary: Introduce ChaosMonkey Action to print HDFS Cluster status Key: HBASE-27567 URL: https://issues.apache.org/jira/browse/HBASE-27567 Project: HBase Issue Type: Task Components: integration tests Reporter: Nick Dimiduk Assignee: Nick Dimiduk We have ChaosMonkey actions for perturbing HDFS components, but we lack a means to periodically display status/health. Introduce a "dump" action similar to the one we have for HBase. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27563) ChaosMonkey sometimes generated invalid boundaries for random item selection
Nick Dimiduk created HBASE-27563: Summary: ChaosMonkey sometimes generated invalid boundaries for random item selection Key: HBASE-27563 URL: https://issues.apache.org/jira/browse/HBASE-27563 Project: HBase Issue Type: Bug Components: integration tests Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.16, 2.5.3 Reporter: Nick Dimiduk Assignee: Nick Dimiduk While running chaos monkey, I occasionally see `IndexOutOfBoundsException` originating from `PolicyBasedChaosMonkey#selectRandomItems`. Looks like we make some false assumptions about the arguments passes to the function. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27485) HBaseTestingUtility minicluster requires log4j2
Nick Dimiduk created HBASE-27485: Summary: HBaseTestingUtility minicluster requires log4j2 Key: HBASE-27485 URL: https://issues.apache.org/jira/browse/HBASE-27485 Project: HBase Issue Type: Bug Components: test Reporter: Nick Dimiduk Using the HBaseMiniCluster now requires addition of log4j-api and log4j-core to the classpath. This can be demonstrated by updating hbase-downstreamer for 2.5.1. https://github.com/saintstack/hbase-downstreamer -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27477) Tweak hbase-vote.sh to support hbase-thirdparty
Nick Dimiduk created HBASE-27477:
Summary: Tweak hbase-vote.sh to support hbase-thirdparty
Key: HBASE-27477
URL: https://issues.apache.org/jira/browse/HBASE-27477
Project: HBase
Issue Type: Task
Components: community
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
With a small change, the {{hbase-vote.sh}} script can be used to verify release
candidates produced from the hbase-thirdparty repository.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-18585) Update release instructions involving gpg-agent
[
https://issues.apache.org/jira/browse/HBASE-18585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-18585.
--
Resolution: Won't Fix
The release process is now automated via the `do-release` family of scripts.
Instead of manual intervention and documentation, we can now rely on updates
those scripts to handle any such remaining edge cases.
> Update release instructions involving gpg-agent
> ---
>
> Key: HBASE-18585
> URL: https://issues.apache.org/jira/browse/HBASE-18585
> Project: HBase
> Issue Type: Task
> Components: build, documentation
>Reporter: Nick Dimiduk
>Priority: Minor
>
> Could be what I ran into is hyper-local, but making a note here none the
> less. For the build of 1.1.12RC0, I stumbled into MGPG-59. Presumably this is
> because of an graded version of {{gpg-agent}}. The release build takes some
> time, such that even if you purse the advice of invoking gpg before starting
> the build, the default config expires before the build finished on my
> machine. I haven't tested this out yet, but a solution appears to look
> something like
> # configure {{gpg-agent}} with a longer value for {{default-cache-ttl}} in
> {{$HOME/.gnupg/gpg-agent.conf}}.
> # invoke {{gpg}} such that the agent will cache the passphrase. MGPG-59
> suggests signing some random file.
> # run {{./dev-support/make_release.sh}} as normal.
> My context:
> {noformat}
> diocles:~ ndimiduk$ sw_vers
> ProductName:Mac OS X
> ProductVersion: 10.11.6
> BuildVersion: 15G1611
> diocles:~ ndimiduk$ gpg-agent --version
> gpg-agent (GnuPG) 2.1.23
> libgcrypt 1.8.0
> Copyright (C) 2017 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27411) Dependency on org.bouncycastle:bcprov-jdk15on:jar define multiple times
Nick Dimiduk created HBASE-27411:
Summary: Dependency on org.bouncycastle:bcprov-jdk15on:jar define
multiple times
Key: HBASE-27411
URL: https://issues.apache.org/jira/browse/HBASE-27411
Project: HBase
Issue Type: Task
Components: build
Affects Versions: 3.0.0-alpha-4
Reporter: Nick Dimiduk
{noformat}
$ env
JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
mvn -T0.5C clean install -DskipTests
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for
org.apache.hbase:hbase-server:jar:3.0.0-alpha-4-SNAPSHOT
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
be unique: org.bouncycastle:bcprov-jdk15on:jar -> duplicate declaration of
version (?) @ line 331, column 17
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten
the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
[WARNING]
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-23248) hbase openjdk11 compile error
[ https://issues.apache.org/jira/browse/HBASE-23248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-23248. -- Resolution: Abandoned > hbase openjdk11 compile error > -- > > Key: HBASE-23248 > URL: https://issues.apache.org/jira/browse/HBASE-23248 > Project: HBase > Issue Type: Bug > Components: build, java >Reporter: jackylau >Priority: Major > Attachments: log > > > i find this > [https://stackoverflow.com/questions/49398894/unable-to-compile-simple-java-10-java-11-project-with-maven/55047110#55047110], > but it still can not solve this problem > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-shade-plugin:3.0.0:shade (default) on project > hbase-protocol-shaded: Error creating shaded jar: null: > IllegalArgumentException -> [Help 1][ERROR] Failed to execute goal > org.apache.maven.plugins:maven-shade-plugin:3.0.0:shade (default) on project > hbase-protocol-shaded: Error creating shaded jar: null: > IllegalArgumentException -> [Help > 1]org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal org.apache.maven.plugins:maven-shade-plugin:3.0.0:shade (default) on > project hbase-protocol-shaded: Error creating shaded jar: null at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) > at > org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) at > org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) at > org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) at > org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) at > org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) at > org.apache.maven.cli.MavenCli.main(MavenCli.java:199) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) at > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) > at > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) > at > org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)Caused > by: org.apache.maven.plugin.MojoExecutionException: Error creating shaded > jar: null at > org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:546) at > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) > ... 20 moreCaused by: java.lang.IllegalArgumentException at > org.objectweb.asm.ClassReader.(Unknown Source) at > org.objectweb.asm.ClassReader.(Unknown Source) at > org.objectweb.asm.ClassReader.(Unknown Source) at > org.vafer.jdependency.Clazzpath.addClazzpathUnit(Clazzpath.java:201) at > org.vafer.jdependency.Clazzpath.addClazzpathUnit(Clazzpath.java:132) at > org.apache.maven.plugins.shade.filter.MinijarFilter.(MinijarFilter.java:95) > at > org.apache.maven.plugins.shade.mojo.ShadeMojo.getFilters(ShadeMojo.java:826) > at org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:434) > ... 22 more -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27352) Quoted string argument with spaces passed from command line are propagated wrongly to the underlying java class
[ https://issues.apache.org/jira/browse/HBASE-27352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-27352. -- Resolution: Fixed Committed to branch-2.5+. Thanks [~ukumar] for the contribution! > Quoted string argument with spaces passed from command line are propagated > wrongly to the underlying java class > --- > > Key: HBASE-27352 > URL: https://issues.apache.org/jira/browse/HBASE-27352 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.5.0 >Reporter: Ujjawal Kumar >Assignee: Ujjawal Kumar >Priority: Minor > Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4 > > Attachments: Screenshot 2022-08-31 at 10.52.11 PM.png > > > Running the following command : > /hbase/bin/hbase org.apache.hadoop.hbase.mapreduce.Import > -Dmapreduce.map.java.opts="-XX:+HeapDumpOnOutOfMemoryError > -XX:HeapDumpPath=$HADOOP_HOME/logs/@taskid@_map.hprof" SYSTEM.CATALOG /tmp > fails with error. It is due to the fact that [read -r -a CMD_ARGS <<< "$@" > |https://github.com/apache/hbase/blob/master/bin/hbase#L878] in the hbase > script would split the command line args (i.e. "$@"). Instead a > straightforward array copy can be used. > !Screenshot 2022-08-31 at 10.52.11 PM.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (HBASE-25563) Add "2.4 Documentation" to the website
[ https://issues.apache.org/jira/browse/HBASE-25563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reopened HBASE-25563: -- Reopening to add the 2.4 menu entry to the main site. > Add "2.4 Documentation" to the website > -- > > Key: HBASE-25563 > URL: https://issues.apache.org/jira/browse/HBASE-25563 > Project: HBase > Issue Type: Task > Components: community, documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Nick Dimiduk >Assignee: Andrew Kyle Purtell >Priority: Major > > Docs from the 2.4.0 build should be slotted into the website. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27351) Add 2.5 Documentation to the website
[
https://issues.apache.org/jira/browse/HBASE-27351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-27351.
--
Resolution: Fixed
> Add 2.5 Documentation to the website
>
>
> Key: HBASE-27351
> URL: https://issues.apache.org/jira/browse/HBASE-27351
> Project: HBase
> Issue Type: Task
> Components: community
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> Following the example from HBASE-24487, will create PR for both the main repo
> and the hbase-site repo.
> Generating the site documentation from main repo, copy to site, as follows:
> {noformat}
> hbase $ git checkout rel/2.5.0
> hbase $ mvn clean site site:stage -DskipTests
> hbase-site $ git checkout asf-site
> hbase-site $ cp -r ../hbase/target/staging ./2.5
> hbase-site $ git add 2.5
> {noformat}
> Then, update the site structure to include a reference to the new content,
> following https://github.com/apache/hbase/pull/2060
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-27350) Add 2.5.0 to the downloads page
[ https://issues.apache.org/jira/browse/HBASE-27350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-27350. -- Resolution: Fixed > Add 2.5.0 to the downloads page > --- > > Key: HBASE-27350 > URL: https://issues.apache.org/jira/browse/HBASE-27350 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 3.0.0-alpha-4 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27351) Add 2.5 Documentation to the website
Nick Dimiduk created HBASE-27351:
Summary: Add 2.5 Documentation to the website
Key: HBASE-27351
URL: https://issues.apache.org/jira/browse/HBASE-27351
Project: HBase
Issue Type: Task
Components: community
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Following the example from HBASE-24487, will create PR for both the main repo
and the hbase-site repo.
Generating the site documentation from main repo, copy to site, as follows:
{noformat}
hbase $ git checkout rel/2.5.0
hbase $ mvn clean site site:stage -DskipTests
hbase-site $ git checkout asf-site
hbase-site $ cp -r ../hbase/target/staging ./2.5
hbase-site $ git add 2.5
{noformat}
Then, update the site structure to include a reference to the new content,
following https://github.com/apache/hbase/pull/2060
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27350) Add 2.5.0 to the downloads page
Nick Dimiduk created HBASE-27350: Summary: Add 2.5.0 to the downloads page Key: HBASE-27350 URL: https://issues.apache.org/jira/browse/HBASE-27350 Project: HBase Issue Type: Task Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 3.0.0-alpha-4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
