[VOTE] Fourth release candidate for HBase 2.3.0 (RC3) is available
Please vote on this Apache hbase release candidate, hbase-2.3.0RC3 The VOTE will remain open for at least 72 hours. [ ] +1 Release this package as Apache hbase 2.3.0 [ ] -1 Do not release this package because ... The tag to be voted on is 2.3.0RC3: https://github.com/apache/hbase/tree/2.3.0RC3 The release files, including signatures, digests, as well as CHANGES.md and RELEASENOTES.md included in this RC can be found at: https://dist.apache.org/repos/dist/dev/hbase/2.3.0RC3/ Maven artifacts are available in a staging repository at: https://repository.apache.org/content/repositories/orgapachehbase-1396/ Artifacts were signed with the ndimi...@apache.org key which can be found in: https://dist.apache.org/repos/dist/release/hbase/KEYS To learn more about Apache hbase, please see http://hbase.apache.org/ Thanks, Your HBase Release Manager
[jira] [Created] (HBASE-24691) Fix flaky TestWALEntryStream
Guanghao Zhang created HBASE-24691: -- Summary: Fix flaky TestWALEntryStream Key: HBASE-24691 URL: https://issues.apache.org/jira/browse/HBASE-24691 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/lastSuccessfulBuild/artifact/dashboard.html] Failed 100.0% (13 / 13) recently. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24689) Generate CHANGES.md and RELEASENOTES.md for 2.2.6
Guanghao Zhang created HBASE-24689: -- Summary: Generate CHANGES.md and RELEASENOTES.md for 2.2.6 Key: HBASE-24689 URL: https://issues.apache.org/jira/browse/HBASE-24689 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24688) AssignRegionHandler uses EventType.M_RS_CLOSE_META instead of EventType.M_RS_OPEN_META
Huaxiang Sun created HBASE-24688: Summary: AssignRegionHandler uses EventType.M_RS_CLOSE_META instead of EventType.M_RS_OPEN_META Key: HBASE-24688 URL: https://issues.apache.org/jira/browse/HBASE-24688 Project: HBase Issue Type: Bug Reporter: Huaxiang Sun This results in openMetaRegion always be executed in closeMetaExecutor. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Third release candidate for HBase 2.3.0 (RC2) is available
The patch as posted has test problems. Since this is not a new regression for 2.3.0, I'll go ahead with another RC. If HBASE-24625 is fixed and there's an RC4, we'll catch it then, otherwise we can pick it up in 2.3.1. That'll be two critical known issues with 2.3.0. On Mon, Jul 6, 2020 at 9:39 AM Nick Dimiduk wrote: > On Sat, Jul 4, 2020 at 7:53 AM 张铎(Duo Zhang) > wrote: > >> Sorry but you missed HBASE-24635... >> > > But how... 郎 > > And I think we'd better also include HBASE-24625? It is a nasty bug... >> > > Okay, yes, let's get this one in as well. > > Viraj Jasani 于2020年7月3日周五 下午10:27写道： >> >> > +1 >> > >> > * Signature: ok >> > * Checksum : ok >> > * Rat check (1.8.0_251): ok >> > - mvn clean apache-rat:check >> > * Built from source (1.8.0_251): ok >> > - mvn clean install -DskipTests >> > * CRUD: ok >> > * Flush/Compaction: ok >> > * WebUI: ok >> > * Load 1M rows using LTT: ok >> > * Unit tests pass (1.8.0_251): failed >> > - mvn package -P runAllTests >> > >> > >> > [ERROR] Failures: >> > [ERROR] >> > >> TestVerifyReplication.testVerifyRepJob:132->TestReplicationBase.runSmallBatchTest:340->TestReplicationBase.waitForReplication:136->TestReplicationBase.waitForReplication:145 >> > Waited too much time for normal batch replication >> > [ERROR] >> > >> TestVerifyReplicationAdjunct.testVerifyReplicationWithSnapshotSupport:284->TestReplicationBase.runSmallBatchTest:340->TestReplicationBase.waitForReplication:136->TestReplicationBase.waitForReplication:145 >> > Waited too much time for normal batch replication >> > >> > Subsequent runs were successful for both tests. >> > Filed HBASE-24677 with sunfire report xmls. >> > On high level, I see some hanging threads for ReadOnlyZKClient and >> > "Replica not found for a datablock" related Exceptions in both reports. >> > >> > >> > On 2020/07/03 00:09:31, Nick Dimiduk wrote: >> > > Please vote on this Apache hbase release candidate, >> > > hbase-2.3.0RC2 >> > > >> > > The VOTE will remain open for at least 72 hours. >> > > >> > > [ ] +1 Release this package as Apache hbase 2.3.0 >> > > [ ] -1 Do not release this package because ... >> > > >> > > The tag to be voted on is 2.3.0RC2: >> > > >> > > https://github.com/apache/hbase/tree/2.3.0RC2 >> > > >> > > The release files, including signatures, digests, as well as >> CHANGES.md >> > > and RELEASENOTES.md included in this RC can be found at: >> > > >> > > https://dist.apache.org/repos/dist/dev/hbase/2.3.0RC2/ >> > > >> > > Maven artifacts are available in a staging repository at: >> > > >> > > >> > https://repository.apache.org/content/repositories/orgapachehbase-1395/ >> > > >> > > Artifacts were signed with the ndimi...@apache.org key which can be >> > found >> > > in: >> > > >> > > https://dist.apache.org/repos/dist/release/hbase/KEYS >> > > >> > > To learn more about Apache hbase, please see >> > > >> > > http://hbase.apache.org/ >> > > >> > > Thanks, >> > > Your HBase Release Manager >> > > >> > >> >
[jira] [Created] (HBASE-24687) New Connection being created for each table
Manas created HBASE-24687: - Summary: New Connection being created for each table Key: HBASE-24687 URL: https://issues.apache.org/jira/browse/HBASE-24687 Project: HBase Issue Type: Bug Components: mob Affects Versions: 2.2.3 Reporter: Manas Attachments: Screen Shot 2020-07-06 at 6.06.43 PM.png Currently creating a new connection for every table under MobFileCleanerChore.java where we should theoretically just using the connection from HBase masterservices. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Please pause git and jira for branch-2.3
I've been having difficulties keeping Jira and git in sync. Until we have a 2.3.0 release, committers please do not push patches to branch-2.3. Instead, please stage a backport pull request and mention me as a reviewer. I'll handle merging and Jira updates from there. Much appreciated. -n On Thu, Jul 2, 2020 at 5:10 PM Nick Dimiduk wrote: > Release is posted. Thanks. > > On Thu, Jul 2, 2020 at 10:55 AM Nick Dimiduk wrote: > >> Heya, >> >> I'd like to build another RC. Please hold off on making changes to >> branch-2.3 in git and jira while I synchronize those two systems. >> >> Thanks, >> Nick >> >
[jira] [Reopened] (HBASE-24625) AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length.
[
https://issues.apache.org/jira/browse/HBASE-24625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-24625:
--
Reopening to apply patch to branch-2.3
> AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced
> file length.
>
>
> Key: HBASE-24625
> URL: https://issues.apache.org/jira/browse/HBASE-24625
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 2.1.0, 2.0.0, 2.2.0, 2.3.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0, 2.2.6
>
>
> By HBASE-14004, we introduce {{WALFileLengthProvider}} interface to keep the
> current writing wal file length by ourselves, {{WALEntryStream}} used by
> {{ReplicationSourceWALReader}} could only read WAL file byte size <=
> {{WALFileLengthProvider.getLogFileSizeIfBeingWritten}} if the WAL file is
> current been writing on the same RegionServer .
> {{AsyncFSWAL}} implements {{WALFileLengthProvider}} by
> {{AbstractFSWAL.getLogFileSizeIfBeingWritten}}, just as folllows :
> {code:java}
>public OptionalLong getLogFileSizeIfBeingWritten(Path path) {
> rollWriterLock.lock();
> try {
> Path currentPath = getOldPath();
> if (path.equals(currentPath)) {
> W writer = this.writer;
> return writer != null ? OptionalLong.of(writer.getLength()) :
> OptionalLong.empty();
> } else {
> return OptionalLong.empty();
> }
> } finally {
> rollWriterLock.unlock();
> }
> }
> {code}
> For {{AsyncFSWAL}}, above {{AsyncFSWAL.writer}} is
> {{AsyncProtobufLogWriter}} ，and {{AsyncProtobufLogWriter.getLength}} is as
> follows:
> {code:java}
> public long getLength() {
> return length.get();
> }
> {code}
> But for {{AsyncProtobufLogWriter}}, any append method may increase the above
> {{AsyncProtobufLogWriter.length}}, especially for following
> {{AsyncFSWAL.append}}
> method just appending the {{WALEntry}} to
> {{FanOutOneBlockAsyncDFSOutput.buf}}:
> {code:java}
> public void append(Entry entry) {
> int buffered = output.buffered();
> try {
> entry.getKey().
>
> getBuilder(compressor).setFollowingKvCount(entry.getEdit().size()).build()
> .writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> throw new AssertionError("should not happen", e);
> }
>
> try {
>for (Cell cell : entry.getEdit().getCells()) {
> cellEncoder.write(cell);
>}
> } catch (IOException e) {
>throw new AssertionError("should not happen", e);
> }
> length.addAndGet(output.buffered() - buffered);
> }
> {code}
> That is to say, {{AsyncFSWAL.getLogFileSizeIfBeingWritten}} could not reflect
> the file length which successfully synced to underlying HDFS, which is not
> as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [VOTE] Third release candidate for HBase 2.3.0 (RC2) is available
On Sat, Jul 4, 2020 at 7:53 AM 张铎(Duo Zhang) wrote: > Sorry but you missed HBASE-24635... > But how... 郎 And I think we'd better also include HBASE-24625? It is a nasty bug... > Okay, yes, let's get this one in as well. Viraj Jasani 于2020年7月3日周五 下午10:27写道： > > > +1 > > > > * Signature: ok > > * Checksum : ok > > * Rat check (1.8.0_251): ok > > - mvn clean apache-rat:check > > * Built from source (1.8.0_251): ok > > - mvn clean install -DskipTests > > * CRUD: ok > > * Flush/Compaction: ok > > * WebUI: ok > > * Load 1M rows using LTT: ok > > * Unit tests pass (1.8.0_251): failed > > - mvn package -P runAllTests > > > > > > [ERROR] Failures: > > [ERROR] > > > TestVerifyReplication.testVerifyRepJob:132->TestReplicationBase.runSmallBatchTest:340->TestReplicationBase.waitForReplication:136->TestReplicationBase.waitForReplication:145 > > Waited too much time for normal batch replication > > [ERROR] > > > TestVerifyReplicationAdjunct.testVerifyReplicationWithSnapshotSupport:284->TestReplicationBase.runSmallBatchTest:340->TestReplicationBase.waitForReplication:136->TestReplicationBase.waitForReplication:145 > > Waited too much time for normal batch replication > > > > Subsequent runs were successful for both tests. > > Filed HBASE-24677 with sunfire report xmls. > > On high level, I see some hanging threads for ReadOnlyZKClient and > > "Replica not found for a datablock" related Exceptions in both reports. > > > > > > On 2020/07/03 00:09:31, Nick Dimiduk wrote: > > > Please vote on this Apache hbase release candidate, > > > hbase-2.3.0RC2 > > > > > > The VOTE will remain open for at least 72 hours. > > > > > > [ ] +1 Release this package as Apache hbase 2.3.0 > > > [ ] -1 Do not release this package because ... > > > > > > The tag to be voted on is 2.3.0RC2: > > > > > > https://github.com/apache/hbase/tree/2.3.0RC2 > > > > > > The release files, including signatures, digests, as well as CHANGES.md > > > and RELEASENOTES.md included in this RC can be found at: > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.3.0RC2/ > > > > > > Maven artifacts are available in a staging repository at: > > > > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1395/ > > > > > > Artifacts were signed with the ndimi...@apache.org key which can be > > found > > > in: > > > > > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > > > To learn more about Apache hbase, please see > > > > > > http://hbase.apache.org/ > > > > > > Thanks, > > > Your HBase Release Manager > > > > > >
[jira] [Created] (HBASE-24686) [LOG] Log improvement in Connection#close
mokai created HBASE-24686: - Summary: [LOG] Log improvement in Connection#close Key: HBASE-24686 URL: https://issues.apache.org/jira/browse/HBASE-24686 Project: HBase Issue Type: Improvement Components: Client, logging Affects Versions: 2.2.3 Reporter: mokai We met some customers used hbase connection improperly, some threads call failed since the shared connection closed by one of the threads. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24685) MultiAction and FailureInfo should be removed
[ https://issues.apache.org/jira/browse/HBASE-24685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HBASE-24685. -- Fix Version/s: 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed > MultiAction and FailureInfo should be removed > - > > Key: HBASE-24685 > URL: https://issues.apache.org/jira/browse/HBASE-24685 > Project: HBase > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Just came across MultiAction and FailureInfo which are IA.Private and not > being used anywhere on trunk. Both of them are being used on branch-2 though. > We should remove them on trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24685) MultiAction and FailureInfo should be removed
Viraj Jasani created HBASE-24685: Summary: MultiAction and FailureInfo should be removed Key: HBASE-24685 URL: https://issues.apache.org/jira/browse/HBASE-24685 Project: HBase Issue Type: Task Reporter: Viraj Jasani Just came across MultiAction and FailureInfo which are IA.Private and not being used anywhere on trunk. Both of them are being used on branch-2 though. We should remove them on trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24684) Fetch ReplicationSink servers list from HMaster instead of ZooKeeper
Guanghao Zhang created HBASE-24684: -- Summary: Fetch ReplicationSink servers list from HMaster instead of ZooKeeper Key: HBASE-24684 URL: https://issues.apache.org/jira/browse/HBASE-24684 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24683) Add a basic ReplicationServer which only implement ReplicationSink Service
Guanghao Zhang created HBASE-24683: -- Summary: Add a basic ReplicationServer which only implement ReplicationSink Service Key: HBASE-24683 URL: https://issues.apache.org/jira/browse/HBASE-24683 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24593) [branch-2.2] Fix the maven compilation failure for nightly build
[ https://issues.apache.org/jira/browse/HBASE-24593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24593. Fix Version/s: 2.2.6 Resolution: Fixed > [branch-2.2] Fix the maven compilation failure for nightly build > > > Key: HBASE-24593 > URL: https://issues.apache.org/jira/browse/HBASE-24593 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > > [https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/896/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24546) CloneSnapshotProcedure unlimited retry
[
https://issues.apache.org/jira/browse/HBASE-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-24546.
Resolution: Fixed
Pushed to branch-2.2+. Thanks [~wenfeiyi666] for contributing.
> CloneSnapshotProcedure unlimited retry
> --
>
> Key: HBASE-24546
> URL: https://issues.apache.org/jira/browse/HBASE-24546
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 2.3.0, master, 2.2.5
>Reporter: wenfeiyi666
>Assignee: wenfeiyi666
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.2.6
>
>
> since regions dir was not remove in the previous execution created, need to
> be remove when retrying, resulting in exception, unlimited retry
> {code:java}
> procedure.CloneSnapshotProcedure: Retriable error trying to clone
> snapshot=snapshot_test to table=test:backup
> state=CLONE_SNAPSHOT_WRITE_FS_LAYOUT
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotException: clone snapshot={
> ss=snapshot_test table=test:backup type=FLUSH } failed because A clone should
> not have regions to remove
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:434)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFsLayout(CloneSnapshotProcedure.java:465)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFilesystemLayout(CloneSnapshotProcedure.java:392)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:67)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:194)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1662)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1409)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1979)
> Caused by: java.lang.IllegalArgumentException: A clone should not have
> regions to remove
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:418)
> ... 10 more
> {code}
> and the cloned regions name are unchanged, resulting in new created regions
> be removed when retrying
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
