[jira] [Commented] (HBASE-27110) Move replication peer storage from zookeeper to other storage systems
[ https://issues.apache.org/jira/browse/HBASE-27110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702976#comment-17702976 ] Guanghao Zhang commented on HBASE-27110: HBase used hbase:meta or ZK as meta info storage. Then HBase 2.x introduced a new master local region to store some meta info. Need a discussion about the long-term road map about meta info storage? It's not just a problem of hbase replication peer storage. > Move replication peer storage from zookeeper to other storage systems > - > > Key: HBASE-27110 > URL: https://issues.apache.org/jira/browse/HBASE-27110 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Duo Zhang >Priority: Major > > This is a more specific issue based on the works which are already done in > HBASE-15867. > There are several candidates for storing replication peer. > 1. A new family of master local region > 2. On FileSystem > 3. A new family of hbase:meta > Either choice has pros and cons. We need to decide which way to go first. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-25780) Add 2.2.7 to download page
[ https://issues.apache.org/jira/browse/HBASE-25780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25780. Assignee: Guanghao Zhang Resolution: Fixed > Add 2.2.7 to download page > -- > > Key: HBASE-25780 > URL: https://issues.apache.org/jira/browse/HBASE-25780 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25780) Add 2.2.7 to download page
Guanghao Zhang created HBASE-25780: -- Summary: Add 2.2.7 to download page Key: HBASE-25780 URL: https://issues.apache.org/jira/browse/HBASE-25780 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25764) Generate CHANGES.md and RELEASENOTES.md for 2.2.7
[ https://issues.apache.org/jira/browse/HBASE-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25764: -- Assignee: Guanghao Zhang > Generate CHANGES.md and RELEASENOTES.md for 2.2.7 > - > > Key: HBASE-25764 > URL: https://issues.apache.org/jira/browse/HBASE-25764 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25764) Generate CHANGES.md and RELEASENOTES.md for 2.2.7
[ https://issues.apache.org/jira/browse/HBASE-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25764. Resolution: Fixed > Generate CHANGES.md and RELEASENOTES.md for 2.2.7 > - > > Key: HBASE-25764 > URL: https://issues.apache.org/jira/browse/HBASE-25764 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25765) Set version as 2.2.7 in branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25765. Fix Version/s: 2.2.7 Resolution: Fixed > Set version as 2.2.7 in branch-2.2 > -- > > Key: HBASE-25765 > URL: https://issues.apache.org/jira/browse/HBASE-25765 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25765) Set version as 2.2.7 in branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25765: -- Assignee: Guanghao Zhang > Set version as 2.2.7 in branch-2.2 > -- > > Key: HBASE-25765 > URL: https://issues.apache.org/jira/browse/HBASE-25765 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25765) Set version as 2.2.7 in branch-2.2
Guanghao Zhang created HBASE-25765: -- Summary: Set version as 2.2.7 in branch-2.2 Key: HBASE-25765 URL: https://issues.apache.org/jira/browse/HBASE-25765 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25764) Generate CHANGES.md and RELEASENOTES.md for 2.2.7
Guanghao Zhang created HBASE-25764: -- Summary: Generate CHANGES.md and RELEASENOTES.md for 2.2.7 Key: HBASE-25764 URL: https://issues.apache.org/jira/browse/HBASE-25764 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25544) Release 2.2.7
[ https://issues.apache.org/jira/browse/HBASE-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318710#comment-17318710 ] Guanghao Zhang commented on HBASE-25544: These ut should be not important and not easy to fix them. Will finish this release but don't fix these ut... > Release 2.2.7 > - > > Key: HBASE-25544 > URL: https://issues.apache.org/jira/browse/HBASE-25544 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > As discussed in [https://s.apache.org/7rqow] , the stable pointer had been > moved to 2.3.x and the final release for 2.2.x will be 2.2.7. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25234) [Upgrade]Incompatibility in reading RS report from 2.1 RS when Master is upgraded to a version containing HBASE-21406
[
https://issues.apache.org/jira/browse/HBASE-25234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-25234:
---
Fix Version/s: (was: 2.3.4)
(was: 2.2.7)
(was: 2.4.0)
(was: 3.0.0-alpha-1)
> [Upgrade]Incompatibility in reading RS report from 2.1 RS when Master is
> upgraded to a version containing HBASE-21406
> -
>
> Key: HBASE-25234
> URL: https://issues.apache.org/jira/browse/HBASE-25234
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Sanjeet Nishad
>Assignee: Michael Stack
>Priority: Major
>
> While upgrading to a version having HBASE-21406 and following the upgrade
> process suggested in HBASE-21075, after Master is upgraded, the following
> exception is observed while reading the rs report from old region servers :
> {code:java}
> 2020-11-02 18:25:30,303 WARN [RS-EventLoopGroup-1-2] ipc.RpcServer:
> /x.x.x.x:16000 is unable to read call parameter from client x.x.x.x
> org.apache.hbase.thirdparty.com.google.protobuf.UninitializedMessageException:
> Message missing required fields: load.replLoadSink.timestampStarted,
> load.replLoadSink.totalOpsProcessed
> at
> org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:477)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerReportRequest$Builder.build(RegionServerStatusProtos.java:2411)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerReportRequest$Builder.build(RegionServerStatusProtos.java:2349)
> at
> org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:654)
> at
> org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:458)
> at
> org.apache.hadoop.hbase.ipc.ServerRpcConnection.saslReadAndProcess(ServerRpcConnection.java:351)
> at
> org.apache.hadoop.hbase.ipc.NettyServerRpcConnection.process(NettyServerRpcConnection.java:92)
> at
> org.apache.hadoop.hbase.ipc.NettyServerRpcConnection.process(NettyServerRpcConnection.java:68)
> at
> org.apache.hadoop.hbase.ipc.NettyRpcServerRequestDecoder.channelRead(NettyRpcServerRequestDecoder.java:62)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at
[jira] [Updated] (HBASE-25556) Frequent replication "Encountered a malformed edit" warnings
[
https://issues.apache.org/jira/browse/HBASE-25556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-25556:
---
Fix Version/s: 2.2.7
> Frequent replication "Encountered a malformed edit" warnings
>
>
> Key: HBASE-25556
> URL: https://issues.apache.org/jira/browse/HBASE-25556
> Project: HBase
> Issue Type: Bug
> Components: Operability, Replication
>Affects Versions: 2.4.1
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.3.5, 2.4.2
>
>
> A sequence of two log lines are frequently printed at WARN level when
> replication is active. Can be several times per minute per regionserver when
> taking writes with replication active. These can appear after a WAL roll or
> after the log archiver moves a WAL file.
> Is this an off-by one error? (Will have to look at the code.)
> If this is expected and harmless, WARN level logging is the wrong thing to
> do, it will trigger operator concern.
> The first:
> {noformat}
> 2021-02-05 16:42:27,269 WARN
> [RS_REFRESH_PEER-regionserver/HOST:8120-0.replicationSource,1.replicationSource.wal-reader.HOST%2C8120%2C1612570567906,1]
> wal.ProtobufLogReader: Encountered a malformed edit, seeking back to last
> good position in file, from 134220988 to 134220987
> java.io.EOFException: Partial PB while reading WAL, probably an unexpected
> EOF, ignoring. current offset=134220988
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:370)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:97)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:85)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.readNextEntryAndRecordReaderPosition(WALEntryStream.java:263)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:176)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:101)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.readWALEntries(ReplicationSourceWALReader.java:220)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:137)
> {noformat}
> The second:
> {noformat}
> 2021-02-05 16:42:27,271 WARN
> [RS_REFRESH_PEER-regionserver/HOST:8120-0.replicationSource,1.replicationSource.wal-reader.HOST%2C8120%2C1612570567906,1]
>
> wal.ProtobufLogReader: Encountered a malformed edit, seeking back to last
> good position in file, from 134220988 to 134220987
> java.io.EOFException: Partial PB while reading WAL, probably an unexpected
> EOF, ignoring. current offset=134220988
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:370)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:97)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:85)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.readNextEntryAndRecordReaderPosition(WALEntryStream.java:263)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:184)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:101)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.readWALEntries(ReplicationSourceWALReader.java:220)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:137)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25427) [branch-2.2] Fix the ruby problem for pre-commit
[ https://issues.apache.org/jira/browse/HBASE-25427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25427. Fix Version/s: 2.2.7 Assignee: Guanghao Zhang Resolution: Fixed > [branch-2.2] Fix the ruby problem for pre-commit > > > Key: HBASE-25427 > URL: https://issues.apache.org/jira/browse/HBASE-25427 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > > *00:33:07* [91mERROR: Error installing rubocop:*00:33:07* > parallel requires Ruby version >= 2.4.*00:33:08* [0mSuccessfully installed > jaro_winkler-1.5.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25187) Improve SizeCachedKV variants initialization
[ https://issues.apache.org/jira/browse/HBASE-25187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25187: --- Fix Version/s: 2.2.7 > Improve SizeCachedKV variants initialization > > > Key: HBASE-25187 > URL: https://issues.apache.org/jira/browse/HBASE-25187 > Project: HBase > Issue Type: Improvement >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0 > > > Currently in SizeCachedKV we get the rowlength and Key length from the > buffers. This can be optimized because we can pass the keylen and row len > while actually creating the cell while reading the cell from the block. Some > times we see that the SizeCachedKV takes the max width in a flame graph - > considering the fact we also do a sanity check on the created KV. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25425) Some notes on RawCell
[ https://issues.apache.org/jira/browse/HBASE-25425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25425: --- Fix Version/s: 2.2.7 > Some notes on RawCell > - > > Key: HBASE-25425 > URL: https://issues.apache.org/jira/browse/HBASE-25425 > Project: HBase > Issue Type: Improvement >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Trivial > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > > Add a few notes to RawCell on what it is. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25083) make sure the next hbase 1.y release has Hadoop 2.10 as a minimum version
[ https://issues.apache.org/jira/browse/HBASE-25083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25083: --- Fix Version/s: 2.2.7 > make sure the next hbase 1.y release has Hadoop 2.10 as a minimum version > - > > Key: HBASE-25083 > URL: https://issues.apache.org/jira/browse/HBASE-25083 > Project: HBase > Issue Type: Task > Components: documentation, hadoop2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0, 2.2.7, 2.3.4, 2.5.0 > > > Our reference guide list of prerequisites still has Hadoop 2.8 and 2.9 listed > for HBase 1 releases. > * [hadoop 2.8 is > EOM|https://lists.apache.org/thread.html/r348f7bc93a522f05b7cce78a911854d128a6b1b8bd8124bad4d06ce6%40%3Cuser.hadoop.apache.org%3E] > * [hadoop 2.9 is > EOM|https://lists.apache.org/thread.html/r16b14cce9504f7a9d228612c6b808e72d8dd20863c78be51a7e04ed5%40%3Cuser.hadoop.apache.org%3E] > The current list in the reference guide for HBase 1.6 is just the 1.5 list > copied. we should update it to remove 2.8 and 2.9 and make sure we're no > longer doing build/test based on those versions for branch-1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24885) STUCK RIT by hbck2 assigns
[
https://issues.apache.org/jira/browse/HBASE-24885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-24885:
---
Fix Version/s: 2.2.7
> STUCK RIT by hbck2 assigns
> --
>
> Key: HBASE-24885
> URL: https://issues.apache.org/jira/browse/HBASE-24885
> Project: HBase
> Issue Type: Bug
> Components: hbck2, Region Assignment
>Affects Versions: 3.0.0-alpha-1, 2.2.3
>Reporter: Bo Cui
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.2
>
>
> If a region has been assign to rs1 and then client assigns region again by
> "hbck2 assigns"
> 1、if regionPlan is region to be assign to rs2，the region will be opened on
> rs1 and rs2.
> master log:
> {quote}WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager:
> rit=OPEN, location=rs2, table=tableName, region=reionName reported OPEN on
> server=rs1 but state has otherwise
> {quote}
> 2、if regionPlan is region to be assign to rs1， the
> TransitRegionStateProcedure and OpenRegionProcedure will stuck. because rs1
> is not responding to master
> rslog:
> {quote}Receiving OPEN for the region:{}, which we are already trying to OPEN
> - ignoring this new request for this region.
> {quote}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24305) Handle deprecations in ServerName
[
https://issues.apache.org/jira/browse/HBASE-24305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-24305:
---
Fix Version/s: 2.3.6
2.4.3
2.2.7
> Handle deprecations in ServerName
> -
>
> Key: HBASE-24305
> URL: https://issues.apache.org/jira/browse/HBASE-24305
> Project: HBase
> Issue Type: Task
>Affects Versions: 3.0.0-alpha-1
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.4.3, 2.3.6
>
>
> Some functions in {{ServerName}} were deprecated in 2.0.0 and should be
> removed for version 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24492) ProtobufLogReader.readNext does not need looping
[ https://issues.apache.org/jira/browse/HBASE-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-24492: --- Fix Version/s: 2.3.6 2.2.7 > ProtobufLogReader.readNext does not need looping > > > Key: HBASE-24492 > URL: https://issues.apache.org/jira/browse/HBASE-24492 > Project: HBase > Issue Type: Task > Components: Replication, wal >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0, 2.2.7, 2.3.6 > > > ProtobufLogReader.readNext is using infinite while loop for the whole method, > however it returns value or throws errors in single execution. while(true) > was part of initial code and with better improvements in the method, this > loop has not been removed yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25544) Release 2.2.7
[ https://issues.apache.org/jira/browse/HBASE-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318669#comment-17318669 ] Guanghao Zhang commented on HBASE-25544: There are some ut (ExportSnapshot related, MOB related, MR related) which always timeout. The UT can pass if ingore them... > Release 2.2.7 > - > > Key: HBASE-25544 > URL: https://issues.apache.org/jira/browse/HBASE-25544 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > As discussed in [https://s.apache.org/7rqow] , the stable pointer had been > moved to 2.3.x and the final release for 2.2.x will be 2.2.7. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25605) Try ignore the ExportSnapshot related unit tests for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25605. Assignee: Guanghao Zhang Resolution: Fixed > Try ignore the ExportSnapshot related unit tests for branch-2.2 > --- > > Key: HBASE-25605 > URL: https://issues.apache.org/jira/browse/HBASE-25605 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25605) Try ignore the ExportSnapshot related unit tests for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25605: --- Fix Version/s: 2.2.7 > Try ignore the ExportSnapshot related unit tests for branch-2.2 > --- > > Key: HBASE-25605 > URL: https://issues.apache.org/jira/browse/HBASE-25605 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25605) Try ignore the ExportSnapshot related unit tests for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25605: --- Summary: Try ignore the ExportSnapshot related unit tests for branch-2.2 (was: Try ignore the ExportSnapshot related unit tests) > Try ignore the ExportSnapshot related unit tests for branch-2.2 > --- > > Key: HBASE-25605 > URL: https://issues.apache.org/jira/browse/HBASE-25605 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25605) Try ignore the ExportSnapshot related unit tests
Guanghao Zhang created HBASE-25605: -- Summary: Try ignore the ExportSnapshot related unit tests Key: HBASE-25605 URL: https://issues.apache.org/jira/browse/HBASE-25605 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25371) When openRegion fails during initial verification(before initializing and setting seq num), exception is observed during region close.
[ https://issues.apache.org/jira/browse/HBASE-25371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25371: --- Fix Version/s: (was: 2.2.7) > When openRegion fails during initial verification(before initializing and > setting seq num), exception is observed during region close. > -- > > Key: HBASE-25371 > URL: https://issues.apache.org/jira/browse/HBASE-25371 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.2.3 >Reporter: Ajeet Rai >Assignee: Mohammad Arshad >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.5, 2.4.2 > > > When openRegion fails during initial verification(before initializing and > setting seq num), exception is observed during region close: > > 2020-12-03 16:34:47,133 ERROR > [RS_OPEN_REGION-regionserver/AA:16040-0] handler.OpenRegionHandler: > Failed open of > region=ns2:testtable4,15,1606912406234.cd386135276b7d3c57416df3666e4aea.2020-12-03 > 16:34:47,133 ERROR [RS_OPEN_REGION-regionserver/blrphispra01054:16040-0] > handler.OpenRegionHandler: Failed open of > region=ns2:testtable4,15,1606912406234.cd386135276b7d3c57416df3666e4aea.java.io.IOException: > The new max sequence id 1 is less than the old max sequence id 7134 at > org.apache.hadoop.hbase.wal.WALSplitUtil.writeRegionSequenceIdFile(WALSplitUtil.java:418) > at > org.apache.hadoop.hbase.regionserver.HRegion.writeRegionCloseMarker(HRegion.java:1253) > at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1793) > at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1606) at > org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1552) at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7522) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7467) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7439) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7397) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7348) > at > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:286) > at > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:111) > at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22348) allow one to actually disable replication svc
[ https://issues.apache.org/jira/browse/HBASE-22348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22348: --- Fix Version/s: (was: 2.2.7) > allow one to actually disable replication svc > - > > Key: HBASE-22348 > URL: https://issues.apache.org/jira/browse/HBASE-22348 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.2.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Labels: replication > Attachments: HBASE-22348.patch > > > Minor, but it does create extra ZK traffic for no reason and there's no way > to disable that it appears. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24792) return value of LogRoller.walRollFinished may be wrong
[ https://issues.apache.org/jira/browse/HBASE-24792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-24792: --- Fix Version/s: (was: 2.2.7) > return value of LogRoller.walRollFinished may be wrong > -- > > Key: HBASE-24792 > URL: https://issues.apache.org/jira/browse/HBASE-24792 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.3.1, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.5 > > > The LogRoller.walRollFinished impl is already buggy. This just checks the > status of the boolean. Once we start a roll on a WAL, we reset the boolean > (Even before patch HBASE-24665). So it is not clearly telling anything abt > the roll status. This can return true even while an active wal roll is going > on. > We might need another boolean in Controller which clearly tracks whether we > are ongoing a roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23591) Negative memStoreSizing
[
https://issues.apache.org/jira/browse/HBASE-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-23591:
---
Fix Version/s: (was: 2.2.7)
> Negative memStoreSizing
> ---
>
> Key: HBASE-23591
> URL: https://issues.apache.org/jira/browse/HBASE-23591
> Project: HBase
> Issue Type: Bug
> Components: read replicas
>Reporter: Szabolcs Bukros
>Priority: Major
>
> After a flush on the replica region the memStoreSizing becomes negative:
> {code:java}
> 2019-12-17 08:31:59,983 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> 0beaae111b0f6e98bfde31ba35be5408 : Replaying flush marker action:
> COMMIT_FLUSH table_name: "IntegrationTestRegionReplicaReplicati
> on" encoded_region_name: "544affde3e027454f67c8ea46c8f69ee"
> flush_sequence_number: 41392 store_flushes { family_name: "f1"
> store_home_dir: "f1" flush_output: "3c48a23eac784a348a18e10e337d80a2" }
> store_flushes { family_name: "f2" store_home_dir: "f2" flush_output:
> "9a5283ec95694667b4ead2398af5f01e" } store_flushes { family_name: "f3"
> store_home_dir: "f3" flush_output: "e6f25e6b0eca4d22af15d0626d0f8759" }
> region_name:
> "IntegrationTestRegionReplicaReplication,,1576599911697.544affde3e027454f67c8ea46c8f69ee."
> 2019-12-17 08:31:59,984 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> 0beaae111b0f6e98bfde31ba35be5408 : Received a flush commit marker with
> seqId:41392 and a previous prepared snapshot was found
> 2019-12-17 08:31:59,993 INFO org.apache.hadoop.hbase.regionserver.HStore:
> Region: 0beaae111b0f6e98bfde31ba35be5408 added
> hdfs://replica-1:8020/hbase/data/default/IntegrationTestRegionReplicaReplication/544affde3e027454f67c8ea46c8f69ee/f1/3c48a23eac784a348a18e10e337d80a2,
> entries=32445, sequenceid=41392, filesize=27.6 M
> 2019-12-17 08:32:00,016 INFO org.apache.hadoop.hbase.regionserver.HStore:
> Region: 0beaae111b0f6e98bfde31ba35be5408 added
> hdfs://replica-1:8020/hbase/data/default/IntegrationTestRegionReplicaReplication/544affde3e027454f67c8ea46c8f69ee/f2/9a5283ec95694667b4ead2398af5f01e,
> entries=12264, sequenceid=41392, filesize=10.9 M
> 2019-12-17 08:32:00,121 INFO org.apache.hadoop.hbase.regionserver.HStore:
> Region: 0beaae111b0f6e98bfde31ba35be5408 added
> hdfs://replica-1:8020/hbase/data/default/IntegrationTestRegionReplicaReplication/544affde3e027454f67c8ea46c8f69ee/f3/e6f25e6b0eca4d22af15d0626d0f8759,
> entries=32379, sequenceid=41392, filesize=27.5 M
> 2019-12-17 08:32:00,122 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> CustomLog decrMemStoreSize. Current: dataSize=135810071,
> getHeapSize=148400960, getOffHeapSize=0, getCellsCount=167243 delta:
> dataSizeDelta=155923644, heapSizeDelta=170112320, offHeapSizeDelta=0,
> cellsCountDelta=188399
> 2019-12-17 08:32:00,122 ERROR org.apache.hadoop.hbase.regionserver.HRegion:
> Asked to modify this region's
> (IntegrationTestRegionReplicaReplication,,1576599911697_0001.0beaae111b0f6e98bfde31ba35be54
> 08.) memStoreSizing to a negative value which is incorrect. Current
> memStoreSizing=135810071, delta=-155923644
> java.lang.Exception
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkNegativeMemStoreDataSize(HRegion.java:1323)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.decrMemStoreSize(HRegion.java:1316)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.decrMemStoreSize(HRegion.java:1303)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayWALFlushCommitMarker(HRegion.java:5194)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayWALFlushMarker(HRegion.java:5025)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doReplayBatchOp(RSRpcServices.java:1143)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.replay(RSRpcServices.java:2232)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:29754)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> {code}
> I added some custom logging to the snapshot logic to be able to see snapshot
> sizes:
> {code:java}
> 2019-12-17 08:31:56,900 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> 0beaae111b0f6e98bfde31ba35be5408 : Replaying flush marker action: START_FLUSH
> table_name: "IntegrationTestRegionReplicaReplication" encoded_region_name:
> "544affde3e027454f67c8ea46c8f69ee" flush_sequence_number: 41392 store_flushes
> { family_name: "f1" store_home_dir: "f1" } store_flushes { family_name: "f2"
> store_home_dir: "f2"
[jira] [Updated] (HBASE-25141) ref guide for 2.2.z builds needs to be updated to include 2.2 specific details
[ https://issues.apache.org/jira/browse/HBASE-25141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25141: --- Fix Version/s: (was: 2.2.7) > ref guide for 2.2.z builds needs to be updated to include 2.2 specific details > -- > > Key: HBASE-25141 > URL: https://issues.apache.org/jira/browse/HBASE-25141 > Project: HBase > Issue Type: Task > Components: documentation, website >Reporter: Sean Busbey >Priority: Minor > > for example, it's currently missing a column for 2.2 in the hadoop compat > matrix. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22917) Proc-WAL roll fails always saying someone else has already created log
[
https://issues.apache.org/jira/browse/HBASE-22917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-22917:
---
Fix Version/s: (was: 2.2.7)
> Proc-WAL roll fails always saying someone else has already created log
> --
>
> Key: HBASE-22917
> URL: https://issues.apache.org/jira/browse/HBASE-22917
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, wal
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
>
> Recently we met a weird scenario where Procedure WAL roll fails as it is
> already created by someone else.
> Later while going through the logs and code, observed that during Proc-WAL
> roll it failed to write the header. On failure file stream is just closed,
> {code}
> try {
> ProcedureWALFormat.writeHeader(newStream, header);
> startPos = newStream.getPos();
> } catch (IOException ioe) {
> LOG.warn("Encountered exception writing header", ioe);
> newStream.close();
> return false;
> }
> {code}
> Since we don't delete the corrupted file or increment the *flushLogId*, so on
> each retry it is trying to create the same *flushLogId* file. However Hmaster
> failover will resolve this issue, but we should handle it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23044) CatalogJanitor#cleanMergeQualifier may clean wrong parent regions
[
https://issues.apache.org/jira/browse/HBASE-23044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17278456#comment-17278456
]
Guanghao Zhang commented on HBASE-23044:
Sorry to hear that..
> CatalogJanitor#cleanMergeQualifier may clean wrong parent regions
> -
>
> Key: HBASE-23044
> URL: https://issues.apache.org/jira/browse/HBASE-23044
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.6, 2.2.1, 2.1.6
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.7, 2.2.2
>
>
> 2019-09-17,19:42:40,539 INFO [PEWorker-1]
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished pid=1223589,
> state=SUCCESS; GCMultipleMergedRegionsProcedure
> child={color:red}647600d28633bb2fe06b40682bab0593{color},
> parents:[81b6fc3c560a00692bc7c3cd266a626a],
> [472500358997b0dc8f0002ec86593dcf] in 2.6470sec
> 2019-09-17,19:59:54,179 INFO [PEWorker-6]
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished pid=1223651,
> state=SUCCESS; GCMultipleMergedRegionsProcedure
> child={color:red}647600d28633bb2fe06b40682bab0593{color},
> parents:[9c52f24e0a9cc9b4959c1ebdfea29d64],
> [a623f298870df5581bcfae7f83311b33] in 1.0340sec
> The child is same region {color:red}647600d28633bb2fe06b40682bab0593{color}
> but the parent regions are different.
> MergeTableRegionProcedure#prepareMergeRegion will try to cleanMergeQualifier
> for the regions to merge.
> {code:java}
> for (RegionInfo ri: this.regionsToMerge) {
> if (!catalogJanitor.cleanMergeQualifier(ri)) {
> String msg = "Skip merging " +
> RegionInfo.getShortNameToLog(regionsToMerge) +
> ", because parent " + RegionInfo.getShortNameToLog(ri) + " has a
> merge qualifier";
> LOG.warn(msg);
> throw new MergeRegionException(msg);
> }
> {code}
> If region A and B merge to C, region D and E merge to F. When merge C and F,
> it will try to cleanMergeQualifier for C and F.
> catalogJanitor.cleanMergeQualifier for region C succeed but
> catalogJanitor.cleanMergeQualifier for region F failed as there are
> references in region F.
> When merge C and F again, it will try to cleanMergeQualifier for C and F
> again. But MetaTableAccessor.getMergeRegions will get wrong parents now. It
> use scan with filter to scan result. But region C's MergeQualifier already
> was deleted before. Then the scan will return a wrong result, may be anther
> region..
> {code:java}
> public boolean cleanMergeQualifier(final RegionInfo region) throws
> IOException {
> // Get merge regions if it is a merged region and already has merge
> qualifier
> List parents =
> MetaTableAccessor.getMergeRegions(this.services.getConnection(),
> region.getRegionName());
> if (parents == null || parents.isEmpty()) {
> // It doesn't have merge qualifier, no need to clean
> return true;
> }
> return cleanMergeRegion(region, parents);
> }
> public static List getMergeRegions(Connection connection, byte[]
> regionName)
> throws IOException {
> return getMergeRegions(getMergeRegionsRaw(connection, regionName));
> }
> private static Cell [] getMergeRegionsRaw(Connection connection, byte []
> regionName)
> throws IOException {
> Scan scan = new Scan().withStartRow(regionName).
> setOneRowLimit().
> readVersions(1).
> addFamily(HConstants.CATALOG_FAMILY).
> setFilter(new QualifierFilter(CompareOperator.EQUAL,
> new RegexStringComparator(HConstants.MERGE_QUALIFIER_PREFIX_STR+
> ".*")));
> try (Table m = getMetaHTable(connection); ResultScanner scanner =
> m.getScanner(scan)) {
> // Should be only one result in this scanner if any.
> Result result = scanner.next();
> if (result == null) {
> return null;
> }
> // Should be safe to just return all Cells found since we had filter in
> place.
> // All values should be RegionInfos or something wrong.
> return result.rawCells();
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25545) Fix the build problem for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25545: --- Fix Version/s: 2.2.7 > Fix the build problem for branch-2.2 > > > Key: HBASE-25545 > URL: https://issues.apache.org/jira/browse/HBASE-25545 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.2.7 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25545) Fix the build problem for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25545. Resolution: Fixed Pushed to branch-2.2. Thanks [~meiyi] for reviewing. > Fix the build problem for branch-2.2 > > > Key: HBASE-25545 > URL: https://issues.apache.org/jira/browse/HBASE-25545 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.2.7 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25545) Fix the build problem for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25545: -- Assignee: Guanghao Zhang > Fix the build problem for branch-2.2 > > > Key: HBASE-25545 > URL: https://issues.apache.org/jira/browse/HBASE-25545 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.2.7 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25544) Release 2.2.7
[ https://issues.apache.org/jira/browse/HBASE-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25544: -- Assignee: Guanghao Zhang > Release 2.2.7 > - > > Key: HBASE-25544 > URL: https://issues.apache.org/jira/browse/HBASE-25544 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > As discussed in [https://s.apache.org/7rqow] , the stable pointer had been > moved to 2.3.x and the final release for 2.2.x will be 2.2.7. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25545) Fix the build problem for branch-2.2
Guanghao Zhang created HBASE-25545: -- Summary: Fix the build problem for branch-2.2 Key: HBASE-25545 URL: https://issues.apache.org/jira/browse/HBASE-25545 Project: HBase Issue Type: Sub-task Affects Versions: 2.2.7 Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25544) Release 2.2.7
Guanghao Zhang created HBASE-25544: -- Summary: Release 2.2.7 Key: HBASE-25544 URL: https://issues.apache.org/jira/browse/HBASE-25544 Project: HBase Issue Type: Umbrella Reporter: Guanghao Zhang As discussed in [https://s.apache.org/7rqow] , the stable pointer had been moved to 2.3.x and the final release for 2.2.x will be 2.2.7. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25277) postScannerFilterRow impacts Scan performance a lot in HBase 2.x
[ https://issues.apache.org/jira/browse/HBASE-25277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253189#comment-17253189 ] Guanghao Zhang commented on HBASE-25277: Merged to branch-2.2 too. And set 2.2.7 as one of the resolved version. > postScannerFilterRow impacts Scan performance a lot in HBase 2.x > > > Key: HBASE-25277 > URL: https://issues.apache.org/jira/browse/HBASE-25277 > Project: HBase > Issue Type: Bug > Components: Coprocessors, scan >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Labels: perfomance, scanning > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4, 2.5.0 > > Attachments: test_report.png > > > In our test we observed Scan performance is degraded by more than 60% in > HBase-2.x as compared to 1.3.x. As per the flamegraph report, RS spent 31% > of the time in postScannerFilterRow, however the coprocessors > (AccessController, VisibilityController & ConstraintProcessor) does nothing > in that hook. > HBASE-14489 added the logic to avoid the call to postScannerFilterRow when > not needed which is not working as expected in HBase 2.x. AccessController, > VisibilityController & ConstraintProcessor override the postScannerFilterRow > with dummy (same as RegionObserver) implementation, so > RegionCoprocessorHost.hasCustomPostScannerFilterRow will be TRUE and call the > hook for all configured CPs while processing each row . Suppose we have > configured 5 region CPs and there are 1 M rows in table, then there will be 1 > * 5 M dummy call to postScannerFilterRow during whole table scan. > We need to remove postScannerFilterRow hook from these CPs as these are not > doing anything. > Another problem is in RegionCoprocessorHost.hasCustomPostScannerFilterRow > init logic, currently it is always TRUE even though we remove > postScannerFilterRow hook from AccessController, VisibilityController & > ConstraintProcessor, because we are finding postScannerFilterRow until it > is found (we look in configured CP's super class also) or clazz is NULL. > https://github.com/apache/hbase/blob/035c192eb665469ce0c071db86c78f4a873c123b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L301 > Supper class of Object (super class of AccessController) will be NULL, so > RegionCoprocessorHost.hasCustomPostScannerFilterRow will be set as TRUE > https://github.com/apache/hbase/blob/035c192eb665469ce0c071db86c78f4a873c123b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L279 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25277) postScannerFilterRow impacts Scan performance a lot in HBase 2.x
[ https://issues.apache.org/jira/browse/HBASE-25277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25277: --- Fix Version/s: 2.2.7 > postScannerFilterRow impacts Scan performance a lot in HBase 2.x > > > Key: HBASE-25277 > URL: https://issues.apache.org/jira/browse/HBASE-25277 > Project: HBase > Issue Type: Bug > Components: Coprocessors, scan >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Labels: perfomance, scanning > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4, 2.5.0 > > Attachments: test_report.png > > > In our test we observed Scan performance is degraded by more than 60% in > HBase-2.x as compared to 1.3.x. As per the flamegraph report, RS spent 31% > of the time in postScannerFilterRow, however the coprocessors > (AccessController, VisibilityController & ConstraintProcessor) does nothing > in that hook. > HBASE-14489 added the logic to avoid the call to postScannerFilterRow when > not needed which is not working as expected in HBase 2.x. AccessController, > VisibilityController & ConstraintProcessor override the postScannerFilterRow > with dummy (same as RegionObserver) implementation, so > RegionCoprocessorHost.hasCustomPostScannerFilterRow will be TRUE and call the > hook for all configured CPs while processing each row . Suppose we have > configured 5 region CPs and there are 1 M rows in table, then there will be 1 > * 5 M dummy call to postScannerFilterRow during whole table scan. > We need to remove postScannerFilterRow hook from these CPs as these are not > doing anything. > Another problem is in RegionCoprocessorHost.hasCustomPostScannerFilterRow > init logic, currently it is always TRUE even though we remove > postScannerFilterRow hook from AccessController, VisibilityController & > ConstraintProcessor, because we are finding postScannerFilterRow until it > is found (we look in configured CP's super class also) or clazz is NULL. > https://github.com/apache/hbase/blob/035c192eb665469ce0c071db86c78f4a873c123b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L301 > Supper class of Object (super class of AccessController) will be NULL, so > RegionCoprocessorHost.hasCustomPostScannerFilterRow will be set as TRUE > https://github.com/apache/hbase/blob/035c192eb665469ce0c071db86c78f4a873c123b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L279 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25426) [branch-2.2] Fix the ruby problem for pre-commit
[ https://issues.apache.org/jira/browse/HBASE-25426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25426. Resolution: Duplicate Duplicate with HBASE-25426. > [branch-2.2] Fix the ruby problem for pre-commit > > > Key: HBASE-25426 > URL: https://issues.apache.org/jira/browse/HBASE-25426 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Priority: Major > > *00:33:07* [91mERROR: Error installing rubocop:*00:33:07* > parallel requires Ruby version >= 2.4.*00:33:08* [0mSuccessfully installed > jaro_winkler-1.5.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25426) [branch-2.2] Fix the ruby problem for pre-commit
[ https://issues.apache.org/jira/browse/HBASE-25426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252656#comment-17252656 ] Guanghao Zhang edited comment on HBASE-25426 at 12/21/20, 7:26 AM: --- Duplicate with HBASE-25427 was (Author: zghaobac): Duplicate with HBASE-25426. > [branch-2.2] Fix the ruby problem for pre-commit > > > Key: HBASE-25426 > URL: https://issues.apache.org/jira/browse/HBASE-25426 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Priority: Major > > *00:33:07* [91mERROR: Error installing rubocop:*00:33:07* > parallel requires Ruby version >= 2.4.*00:33:08* [0mSuccessfully installed > jaro_winkler-1.5.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25427) [branch-2.2] Fix the ruby problem for pre-commit
Guanghao Zhang created HBASE-25427: -- Summary: [branch-2.2] Fix the ruby problem for pre-commit Key: HBASE-25427 URL: https://issues.apache.org/jira/browse/HBASE-25427 Project: HBase Issue Type: Bug Affects Versions: 2.2.6 Reporter: Guanghao Zhang *00:33:07* [91mERROR: Error installing rubocop:*00:33:07*parallel requires Ruby version >= 2.4.*00:33:08* [0mSuccessfully installed jaro_winkler-1.5.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25426) [branch-2.2] Fix the ruby problem for pre-commit
Guanghao Zhang created HBASE-25426: -- Summary: [branch-2.2] Fix the ruby problem for pre-commit Key: HBASE-25426 URL: https://issues.apache.org/jira/browse/HBASE-25426 Project: HBase Issue Type: Bug Affects Versions: 2.2.6 Reporter: Guanghao Zhang *00:33:07* [91mERROR: Error installing rubocop:*00:33:07*parallel requires Ruby version >= 2.4.*00:33:08* [0mSuccessfully installed jaro_winkler-1.5.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24637) Reseek regression related to filter SKIP hinting
[ https://issues.apache.org/jira/browse/HBASE-24637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245804#comment-17245804 ] Guanghao Zhang commented on HBASE-24637: Sorry for reply late. Take a look about the PPT and I understand what you want. It is great. I will take a look about the patch later. Thanks [~ram_krish] for ping me. > Reseek regression related to filter SKIP hinting > > > Key: HBASE-24637 > URL: https://issues.apache.org/jira/browse/HBASE-24637 > Project: HBase > Issue Type: Bug > Components: Filters, Performance, Scanners >Affects Versions: 2.2.5 >Reporter: Andrew Kyle Purtell >Priority: Major > Attachments: W-7665966-FAST_DIFF-FILTER_ALL.pdf, > W-7665966-Instrument-low-level-scan-details-branch-1.patch, > W-7665966-Instrument-low-level-scan-details-branch-2.2.patch, > parse_call_trace.pl, seeksVsSkip_ppt.pptx > > > I have been looking into reported performance regressions in HBase 2 relative > to HBase 1. Depending on the test scenario, HBase 2 can demonstrate > significantly better microbenchmarks in a number of cases, and usually shows > improvement in whole cluster benchmarks like YCSB. > To assist in debugging I added methods to RpcServer for updating per-call > metrics that leverage the fact it puts a reference to the current Call into a > thread local and that all activity for a given RPC is processed by a single > thread context. I then instrumented ScanQueryMatcher (in branch-1) and its > various friends (in branch-2.2), StoreScanner, HFileReaderV2 and > HFileReaderV3 (in branch-1) and HFileReaderImpl (in branch-2.2), HFileBlock, > and DefaultMemStore (branch-1) and SegmentScanner (branch-2.2). Test tables > with one family and 1, 5, 10, 20, 50, and 100 distinct column-qualifiers per > row were created, snapshot, dropped, and cloned from the snapshot. Both 1.6 > and 2.2 versions under test operated on identical data files in HDFS. For > tests with 1.6 and 2.2 on the server side the same 1.6 PE client was used, to > ensure only the server side differed. > The results for pe --filterAll were revealing. See attached. > It appears a refactor to ScanQueryMatcher and friends has disabled the > ability of filters to provide meaningful SKIP hints, which disables an > optimization that avoids reseeking, leading to a serious and proportional > regression in reseek activity and time spent in that code path. So for > queries that use filters, there can be a substantial regression. > Other test cases that did not use filters did not show this regression. If > filters are not used the behavior of ScanQueryMatcher between 1.6 and 2.2 was > almost identical, as measured by counts of the hint types returned, whether > or not column or version trackers are called, and counts of store seeks or > reseeks. Regarding micro-timings, there was a 10% variance in my testing and > results generally fell within this range, except for the filter all case of > course. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25324) Remove unnecessary array to list conversion in SplitLogManager
[
https://issues.apache.org/jira/browse/HBASE-25324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25324.
Fix Version/s: 2.4.0
3.0.0-alpha-1
Resolution: Fixed
Pushed to branch-2 and master. Thanks [~yuqi] for contributing.
> Remove unnecessary array to list conversion in SplitLogManager
> --
>
> Key: HBASE-25324
> URL: https://issues.apache.org/jira/browse/HBASE-25324
> Project: HBase
> Issue Type: Improvement
>Reporter: yuqi
>Assignee: yuqi
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> {code:java}
> public static List getFileList(final Configuration conf, final
> List logDirs,
> final PathFilter filter)
> throws IOException {
> List fileStatus = new ArrayList<>();
>
> FileStatus[] a = new FileStatus[fileStatus.size()];
> //list to array
> return fileStatus.toArray(a);
> }
> public List getWALsToSplit(ServerName serverName, boolean
> splitMeta)
> throws IOException {
> List logDirs =
> master.getMasterWalManager().getLogDirs(Collections.singleton(serverName));
> FileStatus[] fileStatuses =
> SplitLogManager.getFileList(this.conf, logDirs, splitMeta ?
> META_FILTER : NON_META_FILTER);
> LOG.info("{} WAL count={}, meta={}", serverName, fileStatuses.length,
> splitMeta);
> //array convert to list which is just contrary to the logic of
> getFileList
> return Lists.newArrayList(fileStatuses);
>
> {code}
> After judge the logic, there seems no need to make such a conversion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25323) Fix potential NPE when the zookeeper path of RegionServerTracker does not exist when start
[ https://issues.apache.org/jira/browse/HBASE-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25323. Fix Version/s: 2.3.4 2.2.7 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.2+. Thanks [~yuqi] for contributing. > Fix potential NPE when the zookeeper path of RegionServerTracker does not > exist when start > -- > > Key: HBASE-25323 > URL: https://issues.apache.org/jira/browse/HBASE-25323 > Project: HBase > Issue Type: Improvement >Reporter: yuqi >Assignee: yuqi >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4 > > Attachments: image-2020-11-24-16-56-04-411.png, > image-2020-11-24-16-58-04-621.png > > > !image-2020-11-24-16-56-04-411.png! > !image-2020-11-24-16-58-04-621.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25281) Bulkload split hfile too many times due to unreasonable split point
[
https://issues.apache.org/jira/browse/HBASE-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25281.
Fix Version/s: 2.4.0
3.0.0-alpha-1
Resolution: Fixed
Pushed to branch-2 and master. Thanks [~niuyulin] for contributing.
> Bulkload split hfile too many times due to unreasonable split point
> ---
>
> Key: HBASE-25281
> URL: https://issues.apache.org/jira/browse/HBASE-25281
> Project: HBase
> Issue Type: Improvement
> Components: tooling
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
> Attachments: bulkload.png
>
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
> if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
> regions are in ascending order), we should use region C endkey to split, not
> region A. In this way, we can get equal .top and .bottom hfiles, reduce time
> complexity of split from O( n ) to O(logn),decrease invoke of {{bulkLoad}}
> rpc to regionserver ,and also avoid write amplification during
> {{copyHFileHalf}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25325) Remove unused class ClusterSchemaException
[ https://issues.apache.org/jira/browse/HBASE-25325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25325. Fix Version/s: 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2 and master. Thanks [~yuqi] for contributing. > Remove unused class ClusterSchemaException > -- > > Key: HBASE-25325 > URL: https://issues.apache.org/jira/browse/HBASE-25325 > Project: HBase > Issue Type: Improvement >Reporter: yuqi >Assignee: yuqi >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently, ClusterSchemaException is useless, so we can freely delete it -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25282) Remove processingServers in DeadServer as we can get this information by Procedure of master
[ https://issues.apache.org/jira/browse/HBASE-25282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25282. Fix Version/s: 3.0.0-alpha-1 Resolution: Fixed Pushed to master. Thanks [~yuqi] for contributing. > Remove processingServers in DeadServer as we can get this information by > Procedure of master > > > Key: HBASE-25282 > URL: https://issues.apache.org/jira/browse/HBASE-25282 > Project: HBase > Issue Type: Improvement >Reporter: yuqi >Assignee: yuqi >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > We can directly get the servers in processing through proceduces in master as > long as proceduce is a instance of `ServerCrashProcedure` > *Our final goal is to remove class `DeadServer`*, the next step is to remove > the deadServers attributes in DeadServer and then we can freely delete the > class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25213) Should request Compaction when bulkLoadHFiles is done
[ https://issues.apache.org/jira/browse/HBASE-25213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25213. Resolution: Fixed Pushed the addendum patch to branch-2 and master. > Should request Compaction when bulkLoadHFiles is done > - > > Key: HBASE-25213 > URL: https://issues.apache.org/jira/browse/HBASE-25213 > Project: HBase > Issue Type: Improvement >Reporter: niuyulin >Assignee: niuyulin >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24654) Allow unset table's rsgroup
[ https://issues.apache.org/jira/browse/HBASE-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24654. Resolution: Fixed Pushed to master. Thanks [~Ddupg] for contributing. > Allow unset table's rsgroup > --- > > Key: HBASE-24654 > URL: https://issues.apache.org/jira/browse/HBASE-24654 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1 > > > In TableDescriptorBuilder, we have only one method to set rsgroup, but have > no one to unset it. this unset method is necessary In some cases. > If the table had rsgroup config before, but now I want to use the namespace > config. It doesn't work that I set table rsgroup config to default rsgroup, > must remove rsgroup config. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25311) ui throws NPE
[ https://issues.apache.org/jira/browse/HBASE-25311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25311: --- Fix Version/s: 2.3.4 2.2.7 2.4.0 3.0.0-alpha-1 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to branch-2.2+. Thanks [~Bo Cui] for contributing. > ui throws NPE > - > > Key: HBASE-25311 > URL: https://issues.apache.org/jira/browse/HBASE-25311 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.2.3 >Reporter: Bo Cui >Assignee: Bo Cui >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4 > > > https://github.com/apache/hbase/blob/eca904e0fb438461a8da3f37cea3eaf496988be9/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L3624 > if rs has invalid znode, and restart master, ui will throw NPE. > i encountered this problem during the upgrade. > workaround: restart HBase. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25113) [testing] HBaseCluster support ReplicationServer for UTs
[ https://issues.apache.org/jira/browse/HBASE-25113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25113. Resolution: Fixed Merged. Thanks [~Ddupg] for contributing. > [testing] HBaseCluster support ReplicationServer for UTs > > > Key: HBASE-25113 > URL: https://issues.apache.org/jira/browse/HBASE-25113 > Project: HBase > Issue Type: Sub-task > Components: Replication >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25213) Should request Compaction when bulkLoadHFiles is done
[ https://issues.apache.org/jira/browse/HBASE-25213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25213. Fix Version/s: 2.4.0 3.0.0-alpha-1 Resolution: Fixed Merged. Thanks [~niuyulin] for contributing. > Should request Compaction when bulkLoadHFiles is done > - > > Key: HBASE-25213 > URL: https://issues.apache.org/jira/browse/HBASE-25213 > Project: HBase > Issue Type: Improvement >Reporter: niuyulin >Assignee: niuyulin >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25213) Should request Compaction when bulkLoadHFiles is done
[ https://issues.apache.org/jira/browse/HBASE-25213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235949#comment-17235949 ] Guanghao Zhang commented on HBASE-25213: [~niuyulin] Can you add a PR for branch-2. There are conflicts when cherry-pick to branch-2. Thanks. > Should request Compaction when bulkLoadHFiles is done > - > > Key: HBASE-25213 > URL: https://issues.apache.org/jira/browse/HBASE-25213 > Project: HBase > Issue Type: Improvement >Reporter: niuyulin >Assignee: niuyulin >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25300) 'Unknown table hbase:quota' happens when desc table in shell if quota disabled
[ https://issues.apache.org/jira/browse/HBASE-25300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25300. Fix Version/s: 2.3.4 2.2.7 2.4.0 Resolution: Fixed Pushed to branch-2.2+. Thanks [~Ddupg] for contributing. > 'Unknown table hbase:quota' happens when desc table in shell if quota disabled > -- > > Key: HBASE-25300 > URL: https://issues.apache.org/jira/browse/HBASE-25300 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25187) Improve SizeCachedKV variants initialization
[ https://issues.apache.org/jira/browse/HBASE-25187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235078#comment-17235078 ] Guanghao Zhang commented on HBASE-25187: +1 for branch-2.2, too. Thanks. > Improve SizeCachedKV variants initialization > > > Key: HBASE-25187 > URL: https://issues.apache.org/jira/browse/HBASE-25187 > Project: HBase > Issue Type: Improvement >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > Currently in SizeCachedKV we get the rowlength and Key length from the > buffers. This can be optimized because we can pass the keylen and row len > while actually creating the cell while reading the cell from the block. Some > times we see that the SizeCachedKV takes the max width in a flame graph - > considering the fact we also do a sanity check on the created KV. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25289) [testing] Clean up resources after tests in rsgroup_shell_test.rb
[ https://issues.apache.org/jira/browse/HBASE-25289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25289. Fix Version/s: 2.3.4 2.4.0 Resolution: Fixed Pushed to branch-2.3 and branch-2. Thanks [~Ddupg] for contributing. > [testing] Clean up resources after tests in rsgroup_shell_test.rb > - > > Key: HBASE-25289 > URL: https://issues.apache.org/jira/browse/HBASE-25289 > Project: HBase > Issue Type: Improvement > Components: rsgroup, test >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > In rsgroup_shell_test.rb, some tests don't remove rsgroups and drop tables, > messing up adding new tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25289) [testing] Clean up resources after tests in rsgroup_shell_test.rb
[ https://issues.apache.org/jira/browse/HBASE-25289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234166#comment-17234166 ] Guanghao Zhang commented on HBASE-25289: Merged to master branch. But there are many conflict when cherry-pick to branch-2. [~Ddupg] Can you help to submit a new PR for branch-2/branch-2.3/branch-2.2? Thanks. > [testing] Clean up resources after tests in rsgroup_shell_test.rb > - > > Key: HBASE-25289 > URL: https://issues.apache.org/jira/browse/HBASE-25289 > Project: HBase > Issue Type: Improvement > Components: rsgroup, test >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1 > > > In rsgroup_shell_test.rb, some tests don't remove rsgroups and drop tables, > messing up adding new tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25296) [Documentation] fix duplicate conf entry
[
https://issues.apache.org/jira/browse/HBASE-25296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25296.
Resolution: Fixed
Pushed to master. Thanks [~tangtianhang] for contributing.
> [Documentation] fix duplicate conf entry
>
>
> Key: HBASE-25296
> URL: https://issues.apache.org/jira/browse/HBASE-25296
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: tianhang tang
>Assignee: tianhang tang
>Priority: Trivial
>
> [hbase.rolling.restart|https://hbase.apache.org/book.html#hbase.rolling.restart]
> {panel:title=HBase 2.0+ can no longer read Sequence File based WAL file.}
> HBase can no longer read the deprecated WAL files written in the Apache
> Hadoop Sequence File format. The hbase.regionserver.hlog.reader.impl and
> hbase.regionserver.hlog.reader.impl configuration entries should be set to
> use the Protobuf based WAL reader / writer classes. This implementation has
> been the default since HBase 0.96, so legacy WAL files should not be a
> concern for most downstream users.
> {panel}
> It should be:
> "The _hbase.regionserver.hlog.reader.impl_ and
> _hbase.regionserver.hlog.writer.impl_ "...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25295) Refactor the locate WAL logic in ReplicationSource
Guanghao Zhang created HBASE-25295: -- Summary: Refactor the locate WAL logic in ReplicationSource Key: HBASE-25295 URL: https://issues.apache.org/jira/browse/HBASE-25295 Project: HBase Issue Type: Bug Components: Replication Reporter: Guanghao Zhang When cluster replication enabled and one RegionServer crashed, its WALs will be move from WALs dir to oldWALs dir and its replication queue will moved to other RegionServer's replication queue. HDFS layout (WAL Storage) /hbase/WALs/RS1/1.log /hbase/WALs/RS1/2.log /hbase/WALs/RS1/3.log ZooKeeper layout (Replication queue storage) /hbase/replication/rs/RS1/peerId/1.log /hbase/replication/rs/RS1/peerId/2.log /hbase/replication/rs/RS1/peerId/3.log Failover finished: HDFS layout (WAL Storage) /hbase/.oldWALs/1.log /hbase/.oldWALs/2.log /hbase/.oldWALs/3.log ZooKeeper layout (Replication queue storage) /hbase/replication/rs/RS2/peerId-RS1/1.log /hbase/replication/rs/RS2/peerId-RS1/2.log /hbase/replication/rs/RS2/peerId-RS1/3.log And if enabled hbase.separate.oldlogdir.by.regionserver, the HDFS layout may be: HDFS layout (WAL Storage) /hbase/.oldWALs/RS1/1.log /hbase/.oldWALs/RS1/2.log /hbase/.oldWALs/RS1/3.log Then if RS2 crashed, the HDFS layout will not change but ZooKeeper layout may changed. ZooKeeper layout (Replication queue storage) /hbase/replication/rs/RS3/peerId-RS1-RS2/1.log /hbase/replication/rs/RS3/peerId-RS1-RS2/2.log /hbase/replication/rs/RS3/peerId-RS1-RS2/3.log So even the replication queue was transfered many times, the HDFS layout never change. Another case is master-cluster disaster, the failover work not finished. Then ReplicationSyncUp tool can start replication source to replicate the WAL data. The HDFS layout need to consider two more case: /hbase/WALs/RS1/1.log /hbase/WALs/RS1/2.log /hbase/WALs/RS1/3.log or /hbase/WALs/RS1-splitting/1.log /hbase/WALs/RS1-splitting/2.log /hbase/WALs/RS1-splitting/3.log -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25255) Master fails to initialize when creating rs group table
[
https://issues.apache.org/jira/browse/HBASE-25255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231059#comment-17231059
]
Guanghao Zhang commented on HBASE-25255:
{quote} it is possible that before meta loaded is finished, we could add the
region state node for rsgroup table to AssignmentManager with OFFLINE state,
and then we will call processOfflineRegions to assign the offline regions, but
at the same time, the CreateTableProcedure will assign it too, thus we get this
assertion error.
{quote}
Other system tables have same problem?
> Master fails to initialize when creating rs group table
> ---
>
> Key: HBASE-25255
> URL: https://issues.apache.org/jira/browse/HBASE-25255
> Project: HBase
> Issue Type: Bug
> Components: master, rsgroup
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Attachments:
> TEST-org.apache.hadoop.hbase.rsgroup.TestRSGroupsKillRS.xml
>
>
> Saw this when setup TestRSGroupsKillRS
> {noformat}
> 2020-11-07 16:29:54,565 ERROR [master/e476f4f509a7:0:becomeActiveMaster]
> helpers.MarkerIgnoringBase(159): Failed to become active master
> java.lang.AssertionError
> at
> org.apache.hadoop.hbase.master.assignment.RegionStateNode.setProcedure(RegionStateNode.java:198)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createAssignProcedure(AssignmentManager.java:647)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.lambda$null$6(AssignmentManager.java:878)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at
> java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
> at
> java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1699)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:546)
> at
> java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
> at
> java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:505)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createAssignProcedures(AssignmentManager.java:879)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:759)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:775)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.processOfflineRegions(AssignmentManager.java:1513)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1012)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2116)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:515)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24632) Enable procedure-based log splitting as default in hbase3
[ https://issues.apache.org/jira/browse/HBASE-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17230493#comment-17230493 ] Guanghao Zhang commented on HBASE-24632: Ok. Reconsider this after 2.5.x released. :) > Enable procedure-based log splitting as default in hbase3 > - > > Key: HBASE-24632 > URL: https://issues.apache.org/jira/browse/HBASE-24632 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Means changing this value in HConstants to false: >public static final boolean DEFAULT_HBASE_SPLIT_COORDINATED_BY_ZK = true; > Should probably also deprecate the current zk distributed split too so we can > clear out those classes to. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25276) Need to throw the original exception in HRegion#openHRegion
[
https://issues.apache.org/jira/browse/HBASE-25276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25276.
Fix Version/s: 2.3.4
2.2.7
2.4.0
3.0.0-alpha-1
Resolution: Fixed
Pushed to branch-2.2+. Thanks [~niuyulin] for contributing.
> Need to throw the original exception in HRegion#openHRegion
> ---
>
> Key: HBASE-25276
> URL: https://issues.apache.org/jira/browse/HBASE-25276
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: niuyulin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4
>
>
> {code:java}
> protected HRegion openHRegion(final CancelableProgressable reporter)
> throws IOException {
> try {
> ..
> } catch(Throwable t) {
> // By coprocessor path wrong region will open failed,
> // MetricsRegionWrapperImpl is already init and not close,
> // add region close when open failed
> this.close(); // This may throw IOException too.
> throw t;
> }
> return this;
> }
> {code}
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L8144
> this.close() may throw IOException too. The original exception will not be
> recorded.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25276) Need to throw the original exception in HRegion#openHRegion
Guanghao Zhang created HBASE-25276:
--
Summary: Need to throw the original exception in
HRegion#openHRegion
Key: HBASE-25276
URL: https://issues.apache.org/jira/browse/HBASE-25276
Project: HBase
Issue Type: Bug
Reporter: Guanghao Zhang
{code:java}
protected HRegion openHRegion(final CancelableProgressable reporter)
throws IOException {
try {
..
} catch(Throwable t) {
// By coprocessor path wrong region will open failed,
// MetricsRegionWrapperImpl is already init and not close,
// add region close when open failed
this.close(); // This may throw IOException too.
throw t;
}
return this;
}
{code}
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L8144
this.close() may throw IOException too. The original exception will not be
recorded.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24632) Enable procedure-based log splitting as default in hbase3
[ https://issues.apache.org/jira/browse/HBASE-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229625#comment-17229625 ] Guanghao Zhang commented on HBASE-24632: [~stack] [~anoop.hbase] The zk based log splitting is only a internal implenation. Can we purge them out in master branch and no need wait to 4.0.0? > Enable procedure-based log splitting as default in hbase3 > - > > Key: HBASE-24632 > URL: https://issues.apache.org/jira/browse/HBASE-24632 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Means changing this value in HConstants to false: >public static final boolean DEFAULT_HBASE_SPLIT_COORDINATED_BY_ZK = true; > Should probably also deprecate the current zk distributed split too so we can > clear out those classes to. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25071) ReplicationServer support start ReplicationSource internal
[ https://issues.apache.org/jira/browse/HBASE-25071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25071. Resolution: Fixed Merged. Thanks all for reviewing. > ReplicationServer support start ReplicationSource internal > -- > > Key: HBASE-25071 > URL: https://issues.apache.org/jira/browse/HBASE-25071 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24999) Master manages ReplicationServers
[ https://issues.apache.org/jira/browse/HBASE-24999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24999. Resolution: Fixed Merged. Thanks [~Ddupg] for contributing. > Master manages ReplicationServers > - > > Key: HBASE-24999 > URL: https://issues.apache.org/jira/browse/HBASE-24999 > Project: HBase > Issue Type: Sub-task > Components: Replication >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > > In [HBASE-24683|https://issues.apache.org/jira/browse/HBASE-24683] add an > isolated ReplicationServer. > What this issue is to do: > # ReplicationServer reports to Master periodically. > # Add a basic ReplicationServerManager in Master to manage ReplicationServer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25225) Create table very slowly if there are multi regions
[
https://issues.apache.org/jira/browse/HBASE-25225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-25225:
---
Description:
Run the same UT TestRegionReplicaFailover on my local PC, mvn clean test
-Dtest=TestRegionReplicaFailover, branch-2.2 takes 8 mins but branch-2.3 only
needs 2 mins.
I found the problem is related to procedure schedule. See the below log:
2020-10-21 13:52:28,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1427):
Finished pid=296, ppid=45, state=SUCCESS;
org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure in 1.6250sec
2020-10-21 13:52:28,538 INFO [PEWorker-3] procedure2.ProcedureExecutor(1427):
Finished pid=45, ppid=20, state=SUCCESS; TransitRegionStateProcedure
table=testLotsOfRegionRepli2, region=50703895da3cb8c942d3197600d549bc, ASSIGN
in 59.4330sec
The real assign procedure only cost 1.6 seconds but the
TransitRegionStateProcedure cost 59.4 seconds. The pid=45 procedure was
initialized at 2020-10-21 13:51:28,666. It was added to TableQueue at
2020-10-21 13:51:28,789. But took xlock to run at 2020-10-21 13:52:24,761. See
the below log:
{color:#ff}2020-10-21 13:51:28,789{color} DEBUG [PEWorker-4]
procedure.MasterProcedureScheduler(352): Add TableQueue(testLotsOfRegionRepli2,
xlock=true (20) sharedLock=0 size=25) to run queue because: pid=45, ppid=20,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=testLotsOfRegionRepli2,
region=50703895da3cb8c942d3197600d549bc, ASSIGN has the excusive lock access
{color:#ff}2020-10-21 13:52:24,761{color} INFO [PEWorker-2]
procedure.MasterProcedureScheduler(737): Took xlock for pid=45, ppid=20,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=testLotsOfRegionRepli2,
region=50703895da3cb8c942d3197600d549bc, ASSIGN
But when I tried this UT on another PC, it only cost 2 mins, which is the same
with branch-2.3. It is weird.
Marked this as blocker for release 2.2.7.
If you are interested for this, please run "mvn clean test
-Dtest=TestRegionReplicaFailover" and comment the cost time here. Thanks.
was:
Run the same UT TestRegionReplicaFailover on my local PC, mvn clean test
-Dtest=TestRegionReplicaFailover, branch-2.2 takes 8 mins but branch-2.3 only
needs 2 mins.
I found the problem is related to procedure schedule. See the below log:
2020-10-21 13:52:28,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1427):
Finished pid=296, ppid=45, state=SUCCESS;
org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure in 1.6250sec
2020-10-21 13:52:28,538 INFO [PEWorker-3] procedure2.ProcedureExecutor(1427):
Finished pid=45, ppid=20, state=SUCCESS; TransitRegionStateProcedure
table=testLotsOfRegionRepli2, region=50703895da3cb8c942d3197600d549bc, ASSIGN
in 59.4330sec
The real assign procedure only cost 1.6 seconds but the
TransitRegionStateProcedure cost 59.4 seconds. The pid=45 procedure was
initialized at 2020-10-21 13:51:28,666. It was added to TableQueue at
2020-10-21 13:51:28,789. But took xlock to run at 2020-10-21 13:52:24,761. See
the below log:
{color:#ff}2020-10-21 13:51:28,789{color} DEBUG [PEWorker-4]
procedure.MasterProcedureScheduler(352): Add TableQueue(testLotsOfRegionRepli2,
xlock=true (20) sharedLock=0 size=25) to run queue because: pid=45, ppid=20,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=testLotsOfRegionRepli2,
region=50703895da3cb8c942d3197600d549bc, ASSIGN has the excusive lock access
{color:#ff}2020-10-21 13:52:24,761{color} INFO [PEWorker-2]
procedure.MasterProcedureScheduler(737): Took xlock for pid=45, ppid=20,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=testLotsOfRegionRepli2,
region=50703895da3cb8c942d3197600d549bc, ASSIGN
But when I tried this UT on another PC, it only cost 2 mins, which is the same
with branch-2.3. It is weird.
Marked this as blocker for release 2.2.7.
> Create table very slowly if there are multi regions
> ---
>
> Key: HBASE-25225
> URL: https://issues.apache.org/jira/browse/HBASE-25225
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.6
>Reporter: Guanghao Zhang
>Priority: Blocker
>
> Run the same UT TestRegionReplicaFailover on my local PC, mvn clean test
> -Dtest=TestRegionReplicaFailover, branch-2.2 takes 8 mins but branch-2.3 only
> needs 2 mins.
>
> I found the problem is related to procedure schedule. See the below log:
> 2020-10-21 13:52:28,097 INFO [PEWorker-1]
> procedure2.ProcedureExecutor(1427): Finished pid=296, ppid=45, state=SUCCESS;
> org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure in 1.6250sec
> 2020-10-21 13:52:28,538 INFO
[jira] [Created] (HBASE-25225) Create table very slowly if there are multi regions
Guanghao Zhang created HBASE-25225:
--
Summary: Create table very slowly if there are multi regions
Key: HBASE-25225
URL: https://issues.apache.org/jira/browse/HBASE-25225
Project: HBase
Issue Type: Bug
Affects Versions: 2.2.6
Reporter: Guanghao Zhang
Run the same UT TestRegionReplicaFailover on my local PC, mvn clean test
-Dtest=TestRegionReplicaFailover, branch-2.2 takes 8 mins but branch-2.3 only
needs 2 mins.
I found the problem is related to procedure schedule. See the below log:
2020-10-21 13:52:28,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1427):
Finished pid=296, ppid=45, state=SUCCESS;
org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure in 1.6250sec
2020-10-21 13:52:28,538 INFO [PEWorker-3] procedure2.ProcedureExecutor(1427):
Finished pid=45, ppid=20, state=SUCCESS; TransitRegionStateProcedure
table=testLotsOfRegionRepli2, region=50703895da3cb8c942d3197600d549bc, ASSIGN
in 59.4330sec
The real assign procedure only cost 1.6 seconds but the
TransitRegionStateProcedure cost 59.4 seconds. The pid=45 procedure was
initialized at 2020-10-21 13:51:28,666. It was added to TableQueue at
2020-10-21 13:51:28,789. But took xlock to run at 2020-10-21 13:52:24,761. See
the below log:
{color:#ff}2020-10-21 13:51:28,789{color} DEBUG [PEWorker-4]
procedure.MasterProcedureScheduler(352): Add TableQueue(testLotsOfRegionRepli2,
xlock=true (20) sharedLock=0 size=25) to run queue because: pid=45, ppid=20,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=testLotsOfRegionRepli2,
region=50703895da3cb8c942d3197600d549bc, ASSIGN has the excusive lock access
{color:#ff}2020-10-21 13:52:24,761{color} INFO [PEWorker-2]
procedure.MasterProcedureScheduler(737): Took xlock for pid=45, ppid=20,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=testLotsOfRegionRepli2,
region=50703895da3cb8c942d3197600d549bc, ASSIGN
But when I tried this UT on another PC, it only cost 2 mins, which is the same
with branch-2.3. It is weird.
Marked this as blocker for release 2.2.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-25110) Add heartbeat for ReplicationServer and dispatch replication sources to ReplicationServer
[ https://issues.apache.org/jira/browse/HBASE-25110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25110: -- Assignee: Guanghao Zhang > Add heartbeat for ReplicationServer and dispatch replication sources to > ReplicationServer > - > > Key: HBASE-25110 > URL: https://issues.apache.org/jira/browse/HBASE-25110 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25128) RSGroupInfo's toString() and hashCode() does not take into account configuration map.
[ https://issues.apache.org/jira/browse/HBASE-25128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25128. Fix Version/s: 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2 and master. Thanks [~sanjeetnishad] for contributing. > RSGroupInfo's toString() and hashCode() does not take into account > configuration map. > - > > Key: HBASE-25128 > URL: https://issues.apache.org/jira/browse/HBASE-25128 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 2.2.3 >Reporter: Sanjeet Nishad >Assignee: Sanjeet Nishad >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RSGroupInfo's toString() and hashcode() methods should include Configurations > as well which is added as a part of HBASE-24431. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25186) TestMasterRegionOnTwoFileSystems is failing after HBASE-25065
[ https://issues.apache.org/jira/browse/HBASE-25186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218130#comment-17218130 ] Guanghao Zhang commented on HBASE-25186: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile (default-testCompile) on project hbase-server: Compilation failure [ERROR] /home/zhangguanghao1/code/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/master/region/TestMasterRegionWALCleaner.java:[89,5] cannot find symbol [ERROR] symbol: class FileStatus [ERROR] location: class org.apache.hadoop.hbase.master.region.TestMasterRegionWALCleaner > TestMasterRegionOnTwoFileSystems is failing after HBASE-25065 > - > > Key: HBASE-25186 > URL: https://issues.apache.org/jira/browse/HBASE-25186 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: ramkrishna.s.vasudevan >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0-alpha-1, 2.4.0 > > > After HBASE-25065, we are having a test case failure with > TestMasterRegionOnTwoFileSystems. > The reason is that we manually trigger a WAL roll on the master region. As > part of the WAL roll we expect the Master region's WAL will also be moved > from region oldWAL dir to the global oldWAL directory. This happens after > afterRoll() method in AbstractWALRoller. > Since now the WAL archival is asynchronous the afterRoll() method does not > find any WAL file to be moved in the local region oldWAL dir. So the movement > to global oldWAL dir does not happen. > The test case checks for the file in the oldWAL dir and since it is not found > the test timesout. WE need a way to fix this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-25186) TestMasterRegionOnTwoFileSystems is failing after HBASE-25065
[ https://issues.apache.org/jira/browse/HBASE-25186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reopened HBASE-25186: Reopen for branch-2. Failed to mvn package. > TestMasterRegionOnTwoFileSystems is failing after HBASE-25065 > - > > Key: HBASE-25186 > URL: https://issues.apache.org/jira/browse/HBASE-25186 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: ramkrishna.s.vasudevan >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0-alpha-1, 2.4.0 > > > After HBASE-25065, we are having a test case failure with > TestMasterRegionOnTwoFileSystems. > The reason is that we manually trigger a WAL roll on the master region. As > part of the WAL roll we expect the Master region's WAL will also be moved > from region oldWAL dir to the global oldWAL directory. This happens after > afterRoll() method in AbstractWALRoller. > Since now the WAL archival is asynchronous the afterRoll() method does not > find any WAL file to be moved in the local region oldWAL dir. So the movement > to global oldWAL dir does not happen. > The test case checks for the file in the oldWAL dir and since it is not found > the test timesout. WE need a way to fix this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25204) Nightly job failed as the name of jdk and maven changed
[ https://issues.apache.org/jira/browse/HBASE-25204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25204. Fix Version/s: 2.2.7 1.4.14 2.4.0 1.7.0 2.3.3 3.0.0-alpha-1 Resolution: Fixed Pushed to all active branchs. Thanks [~zhangduo] for reviewing. > Nightly job failed as the name of jdk and maven changed > > > Key: HBASE-25204 > URL: https://issues.apache.org/jira/browse/HBASE-25204 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7 > > > See > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/85/console] > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/103/console] > > org.codehaus.groovy.control.MultipleCompilationErrorsException: startup > failed: WorkflowScript: 508: Tool type "maven" does not have an install of > "Maven (latest)" configured - did you mean "maven_latest"? @ line 508, column > 19. maven 'Maven (latest)' ^ WorkflowScript: 510: Tool type "jdk" does not > have an install of "JDK 1.8 (latest)" configured - did you mean > "jdk_1.8_latest"? @ line 510, column 17. jdk "JDK 1.8 (latest)" > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25204) Nightly job failed as the name of jdk and maven changed
[ https://issues.apache.org/jira/browse/HBASE-25204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25204: -- Assignee: Guanghao Zhang > Nightly job failed as the name of jdk and maven changed > > > Key: HBASE-25204 > URL: https://issues.apache.org/jira/browse/HBASE-25204 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > See > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/85/console] > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/103/console] > > org.codehaus.groovy.control.MultipleCompilationErrorsException: startup > failed: WorkflowScript: 508: Tool type "maven" does not have an install of > "Maven (latest)" configured - did you mean "maven_latest"? @ line 508, column > 19. maven 'Maven (latest)' ^ WorkflowScript: 510: Tool type "jdk" does not > have an install of "JDK 1.8 (latest)" configured - did you mean > "jdk_1.8_latest"? @ line 510, column 17. jdk "JDK 1.8 (latest)" > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25204) Nightly job failed as the name of jdk and maven changed
Guanghao Zhang created HBASE-25204: -- Summary: Nightly job failed as the name of jdk and maven changed Key: HBASE-25204 URL: https://issues.apache.org/jira/browse/HBASE-25204 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang See [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/85/console] [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/103/console] org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: WorkflowScript: 508: Tool type "maven" does not have an install of "Maven (latest)" configured - did you mean "maven_latest"? @ line 508, column 19. maven 'Maven (latest)' ^ WorkflowScript: 510: Tool type "jdk" does not have an install of "JDK 1.8 (latest)" configured - did you mean "jdk_1.8_latest"? @ line 510, column 17. jdk "JDK 1.8 (latest)" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25200) Try enlarge the flaky test timeout for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216432#comment-17216432 ] Guanghao Zhang commented on HBASE-25200: I thought the flakey test's reason is that: # Too many tests failed in one nightly job. # The flaky test job run thest tests but cannot finished in 2 hours. # Thest tests marked as flaky again. # Go back to 2. And cannot ok anymore. > Try enlarge the flaky test timeout for branch-2.2 > - > > Key: HBASE-25200 > URL: https://issues.apache.org/jira/browse/HBASE-25200 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > Now there are too many flaky tests to run. And the flaky test job cannot > finished in 2 hours. Then these tests will be marked flaky again. > > See > [https://ci-hadoop.apache.org/job/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/124/artifact/dashboard.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25200) Try enlarge the flaky test timeout for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25200: --- Description: Now there are too many flaky tests to run. And the flaky test job cannot finished in 2 hours. Then these tests will be marked to flaky again. See https://ci-hadoop.apache.org/job/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/124/artifact/dashboard.html was:Now there are too many flaky tests to run. And the flaky test job cannot finished. Then these tests will be marked to flaky again. > Try enlarge the flaky test timeout for branch-2.2 > - > > Key: HBASE-25200 > URL: https://issues.apache.org/jira/browse/HBASE-25200 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > > Now there are too many flaky tests to run. And the flaky test job cannot > finished in 2 hours. Then these tests will be marked to flaky again. > > See > https://ci-hadoop.apache.org/job/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/124/artifact/dashboard.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25200) Try enlarge the flaky test timeout for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25200: -- Assignee: Guanghao Zhang > Try enlarge the flaky test timeout for branch-2.2 > - > > Key: HBASE-25200 > URL: https://issues.apache.org/jira/browse/HBASE-25200 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > Now there are too many flaky tests to run. And the flaky test job cannot > finished in 2 hours. Then these tests will be marked flaky again. > > See > [https://ci-hadoop.apache.org/job/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/124/artifact/dashboard.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25200) Try enlarge the flaky test timeout for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25200: --- Description: Now there are too many flaky tests to run. And the flaky test job cannot finished in 2 hours. Then these tests will be marked flaky again. See [https://ci-hadoop.apache.org/job/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/124/artifact/dashboard.html] was: Now there are too many flaky tests to run. And the flaky test job cannot finished in 2 hours. Then these tests will be marked to flaky again. See https://ci-hadoop.apache.org/job/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/124/artifact/dashboard.html > Try enlarge the flaky test timeout for branch-2.2 > - > > Key: HBASE-25200 > URL: https://issues.apache.org/jira/browse/HBASE-25200 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > > Now there are too many flaky tests to run. And the flaky test job cannot > finished in 2 hours. Then these tests will be marked flaky again. > > See > [https://ci-hadoop.apache.org/job/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/124/artifact/dashboard.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25200) Try enlarge the flaky test timeout for branch-2.2
Guanghao Zhang created HBASE-25200: -- Summary: Try enlarge the flaky test timeout for branch-2.2 Key: HBASE-25200 URL: https://issues.apache.org/jira/browse/HBASE-25200 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang Now there are too many flaky tests to run. And the flaky test job cannot finished. Then these tests will be marked to flaky again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25177) Try create table with 100 regions for branch-2.2 nightly job's hadoop integration test
[ https://issues.apache.org/jira/browse/HBASE-25177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25177. Resolution: Won't Fix > Try create table with 100 regions for branch-2.2 nightly job's hadoop > integration test > -- > > Key: HBASE-25177 > URL: https://issues.apache.org/jira/browse/HBASE-25177 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > It still failed now. > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/171/log/] > > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88//artifact/output-integration/hadoop-2.log] > > It failed when create table with 1000 regions. And not import the example TSV > to HDFS. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25178) Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3
[ https://issues.apache.org/jira/browse/HBASE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25178. Resolution: Duplicate Already fixed by HBASE-25144. > Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3 > - > > Key: HBASE-25178 > URL: https://issues.apache.org/jira/browse/HBASE-25178 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > For branch-2.2, see > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] > It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess > -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". > > For branch-2.3, see HBASE-23834. HBase failed to start on hadoop 3.3.0 > because the jetty problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25177) Try create table with 100 regions for branch-2.2 nightly job's hadoop integration test
[ https://issues.apache.org/jira/browse/HBASE-25177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25177: -- Assignee: Guanghao Zhang > Try create table with 100 regions for branch-2.2 nightly job's hadoop > integration test > -- > > Key: HBASE-25177 > URL: https://issues.apache.org/jira/browse/HBASE-25177 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > It still failed now. > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/171/log/] > > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88//artifact/output-integration/hadoop-2.log] > > It failed when create table with 1000 regions. And not import the example TSV > to HDFS. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25178) Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3
[ https://issues.apache.org/jira/browse/HBASE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25178: -- Assignee: Guanghao Zhang > Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3 > - > > Key: HBASE-25178 > URL: https://issues.apache.org/jira/browse/HBASE-25178 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > For branch-2.2, see > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] > It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess > -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". > > For branch-2.3, see HBASE-23834. HBase failed to start on hadoop 3.3.0 > because the jetty problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25144) Add Hadoop-3.3.0 to personality hadoopcheck
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212132#comment-17212132 ] Guanghao Zhang edited comment on HBASE-25144 at 10/12/20, 6:07 AM: --- I do not think we need to try 3.3.0 build for 2.3.x and 2.2.x. Please see HBASE-23834, it is a big change and is only committed to master and branch-2. was (Author: apache9): I do not think we need to try 3.3.0 build for 2.3.x and 2.2.x. Please see HBASE-23824, it is a big change and is only committed to master and branch-2. > Add Hadoop-3.3.0 to personality hadoopcheck > --- > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25178) Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3
[ https://issues.apache.org/jira/browse/HBASE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25178: --- Description: For branch-2.2, see [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". For branch-2.3, see HBASE-23834. HBase failed to start on hadoop 3.3.0 because the jetty problem. was: See [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". > Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3 > - > > Key: HBASE-25178 > URL: https://issues.apache.org/jira/browse/HBASE-25178 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Priority: Major > > For branch-2.2, see > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] > It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess > -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". > > For branch-2.3, see HBASE-23834. HBase failed to start on hadoop 3.3.0 > because the jetty problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25178) Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3
[ https://issues.apache.org/jira/browse/HBASE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25178: --- Summary: Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3 (was: Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2) > Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3 > - > > Key: HBASE-25178 > URL: https://issues.apache.org/jira/browse/HBASE-25178 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Priority: Major > > See > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] > > It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess > -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25178) Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25178: --- Summary: Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2 (was: Fix the LICENSE error when branch-2.2 build with hadoop 3.3.0 ) > Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2 > -- > > Key: HBASE-25178 > URL: https://issues.apache.org/jira/browse/HBASE-25178 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Priority: Major > > See > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] > > It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess > -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25178) Fix the LICENSE error when branch-2.2 build with hadoop 3.3.0
Guanghao Zhang created HBASE-25178: -- Summary: Fix the LICENSE error when branch-2.2 build with hadoop 3.3.0 Key: HBASE-25178 URL: https://issues.apache.org/jira/browse/HBASE-25178 Project: HBase Issue Type: Bug Affects Versions: 2.2.6 Reporter: Guanghao Zhang See [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25177) Try create table with 100 regions for branch-2.2 nightly job's hadoop integration test
Guanghao Zhang created HBASE-25177: -- Summary: Try create table with 100 regions for branch-2.2 nightly job's hadoop integration test Key: HBASE-25177 URL: https://issues.apache.org/jira/browse/HBASE-25177 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang It still failed now. [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/171/log/] [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88//artifact/output-integration/hadoop-2.log] It failed when create table with 1000 regions. And not import the example TSV to HDFS. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
[
https://issues.apache.org/jira/browse/HBASE-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25172.
Resolution: Fixed
> No need timelineservice for branch-2.2 nightly job's hadoop integration test
>
>
> Key: HBASE-25172
> URL: https://issues.apache.org/jira/browse/HBASE-25172
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.2.7
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
>
>
> /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
> --single-process --working-dir output-integration/hadoop-2
> --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
> {color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
> hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
> hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
> hadoop-2/bin/mapred
>
> branch-2.2 still use hadoop 2.8.5 and hadoop 2.8.5 doesn't have
> timelineservice. The dev-support/hbase_nightly_pseudo-distributed-test.sh not
> consider this timelineservice and only consider 5 paramerters. But
> branch-2.3+ use 2.10.x hadoop, so they consider 6 parameters.
>
> And for hadoop-3, the timelineservice is not used, too. See
> [https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
[
https://issues.apache.org/jira/browse/HBASE-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-25172:
---
Description:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
{color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and hadoop 2.8.5 doesn't have
timelineservice. The dev-support/hbase_nightly_pseudo-distributed-test.sh not
consider this timelineservice and only consider 5 paramerters. But branch-2.3+
use 2.10.x hadoop, so they consider 6 parameters.
And for hadoop-3, the timelineservice is not used, too. See
[https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286]
was:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
{color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this
timelineservice and only consider 5 paramerters. But branch-2.3+ use 2.10.x
hadoop, so they consider 6 parameters.
And for hadoop-3, the timelineservice is not used, too. See
[https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286]
> No need timelineservice for branch-2.2 nightly job's hadoop integration test
>
>
> Key: HBASE-25172
> URL: https://issues.apache.org/jira/browse/HBASE-25172
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.2.7
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
>
>
> /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
> --single-process --working-dir output-integration/hadoop-2
> --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
> {color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
> hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
> hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
> hadoop-2/bin/mapred
>
> branch-2.2 still use hadoop 2.8.5 and hadoop 2.8.5 doesn't have
> timelineservice. The dev-support/hbase_nightly_pseudo-distributed-test.sh not
> consider this timelineservice and only consider 5 paramerters. But
> branch-2.3+ use 2.10.x hadoop, so they consider 6 parameters.
>
> And for hadoop-3, the timelineservice is not used, too. See
> [https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
[
https://issues.apache.org/jira/browse/HBASE-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-25172:
---
Description:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
{color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this
timelineservice and only consider 5 paramerters. But branch-2.3+ use 2.10.x
hadoop, so they consider 6 parameters.
And for hadoop-3, the timelineservice is not used, too. See
[https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286]
was:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
{color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this
timelineservice and only consider 5 paramerter.
And for hadoop-3, the timelineservice is not used, too. See
https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286
> No need timelineservice for branch-2.2 nightly job's hadoop integration test
>
>
> Key: HBASE-25172
> URL: https://issues.apache.org/jira/browse/HBASE-25172
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.2.7
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
>
>
> /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
> --single-process --working-dir output-integration/hadoop-2
> --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
> {color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
> hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
> hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
> hadoop-2/bin/mapred
>
> branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
> dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this
> timelineservice and only consider 5 paramerters. But branch-2.3+ use 2.10.x
> hadoop, so they consider 6 parameters.
>
> And for hadoop-3, the timelineservice is not used, too. See
> [https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
[
https://issues.apache.org/jira/browse/HBASE-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-25172:
---
Description:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
{color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this
timelineservice and only consider 5 paramerter.
And for hadoop-3, the timelineservice is not used, too. See
https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286
was:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
{color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this
timelineservice and only consider 5 paramerter.
> No need timelineservice for branch-2.2 nightly job's hadoop integration test
>
>
> Key: HBASE-25172
> URL: https://issues.apache.org/jira/browse/HBASE-25172
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.2.7
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
>
>
> /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
> --single-process --working-dir output-integration/hadoop-2
> --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
> {color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
> hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
> hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
> hadoop-2/bin/mapred
>
> branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
> dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this
> timelineservice and only consider 5 paramerter.
>
> And for hadoop-3, the timelineservice is not used, too. See
> https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
[
https://issues.apache.org/jira/browse/HBASE-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-25172:
---
Description:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
{color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this
timelineservice and only consider 5 paramerter.
was:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
{color:#FF}hadoop-2/share/hadoop/yarn/timelineservice{color}
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this.
> No need timelineservice for branch-2.2 nightly job's hadoop integration test
>
>
> Key: HBASE-25172
> URL: https://issues.apache.org/jira/browse/HBASE-25172
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.2.7
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
>
>
> /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
> --single-process --working-dir output-integration/hadoop-2
> --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
> {color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
> hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
> hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
> hadoop-2/bin/mapred
>
> branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
> dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this
> timelineservice and only consider 5 paramerter.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
[
https://issues.apache.org/jira/browse/HBASE-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-25172:
---
Description:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
{color:#FF}hadoop-2/share/hadoop/yarn/timelineservice{color}
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this.
was:
[https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
/home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
hadoop-2/share/hadoop/yarn/timelineservice
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this.
> No need timelineservice for branch-2.2 nightly job's hadoop integration test
>
>
> Key: HBASE-25172
> URL: https://issues.apache.org/jira/browse/HBASE-25172
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.2.7
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
>
>
> /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
> --single-process --working-dir output-integration/hadoop-2
> --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
> {color:#FF}hadoop-2/share/hadoop/yarn/timelineservice{color}
> hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
> hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
> hadoop-2/bin/mapred
>
> branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The
> dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
[ https://issues.apache.org/jira/browse/HBASE-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25172: --- Affects Version/s: 2.2.6 > No need timelineservice for branch-2.2 nightly job's hadoop integration test > > > Key: HBASE-25172 > URL: https://issues.apache.org/jira/browse/HBASE-25172 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/] > > /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh > --single-process --working-dir output-integration/hadoop-2 > --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop > hadoop-2/share/hadoop/yarn/timelineservice > hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar > hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar > hadoop-2/bin/mapred > > branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The > dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
[ https://issues.apache.org/jira/browse/HBASE-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-25172: --- Fix Version/s: 2.2.7 > No need timelineservice for branch-2.2 nightly job's hadoop integration test > > > Key: HBASE-25172 > URL: https://issues.apache.org/jira/browse/HBASE-25172 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/] > > /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh > --single-process --working-dir output-integration/hadoop-2 > --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop > hadoop-2/share/hadoop/yarn/timelineservice > hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar > hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar > hadoop-2/bin/mapred > > branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The > dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
