[jira] [Updated] (HBASE-21165) During ProcedureStore load, there is no listing of progress...
[
https://issues.apache.org/jira/browse/HBASE-21165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-21165:
--
Description:
I have a Master that crashed on a large cluster with hundreds of outstanding
Procedure WALs and ~1M of Procedures to load. It is taking a long time (two
hours) to load...
There were STUCK procedures that were preventing clean-up of the old WALs.
I can tell we are making progress by enabling TRACE on the Procedure Store.
Better would be an emission as we made progress through the files with an
emission after every so many procedures loaded.
Seems like post-load, there is a long time spent sorting-out the Procedure
image... We are in here for ages:
{code}
"master/vc0207:22001:becomeActiveMaster" #98 daemon prio=5 os_prio=0
tid=0x00d31800 nid=0x1efc0 runnable [0x7f0a3c17d000]
java.lang.Thread.State: RUNNABLE
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader$WalProcedureMap.removeFromMap(ProcedureWALFormatReader.java:837)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader$WalProcedureMap.fetchReady(ProcedureWALFormatReader.java:614)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader.finish(ProcedureWALFormatReader.java:201)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.load(ProcedureWALFormat.java:94)
at
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(WALProcedureStore.java:426)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:382)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:663)
at
org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1335)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:878)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2119)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:567)
at
org.apache.hadoop.hbase.master.HMaster$$Lambda$42/1930759883.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
{code}
was:
I have a Master that crashed on a large cluster with hundreds of ourstanding
Procedure WALs and millions of Procedures to load. It is taking a long time
(hours) to load... There were STUCK procedures that were preventing clean-up of
the old WALs.
I can tell we are making progress by enabling TRACE on the Procedure Store.
Better would be an emission as we made progress through the files with an
emission after every so many procedures loaded.
> During ProcedureStore load, there is no listing of progress...
> --
>
> Key: HBASE-21165
> URL: https://issues.apache.org/jira/browse/HBASE-21165
> Project: HBase
> Issue Type: Bug
> Components: amv2, Operability
>Reporter: stack
>Assignee: stack
>Priority: Major
>
> I have a Master that crashed on a large cluster with hundreds of outstanding
> Procedure WALs and ~1M of Procedures to load. It is taking a long time (two
> hours) to load...
> There were STUCK procedures that were preventing clean-up of the old WALs.
> I can tell we are making progress by enabling TRACE on the Procedure Store.
> Better would be an emission as we made progress through the files with an
> emission after every so many procedures loaded.
> Seems like post-load, there is a long time spent sorting-out the Procedure
> image... We are in here for ages:
> {code}
> "master/vc0207:22001:becomeActiveMaster" #98 daemon prio=5 os_prio=0
> tid=0x00d31800 nid=0x1efc0 runnable [0x7f0a3c17d000]
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader$WalProcedureMap.removeFromMap(ProcedureWALFormatReader.java:837)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader$WalProcedureMap.fetchReady(ProcedureWALFormatReader.java:614)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader.finish(ProcedureWALFormatReader.java:201)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.load(ProcedureWALFormat.java:94)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(WALProcedureStore.java:426)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:382)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:663)
> at
> org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1335)
> at
>
[jira] [Created] (HBASE-21167) Master killed after IOE in FanOutOneBlockAsyncDFSOutput on log roll
stack created HBASE-21167:
-
Summary: Master killed after IOE in FanOutOneBlockAsyncDFSOutput
on log roll
Key: HBASE-21167
URL: https://issues.apache.org/jira/browse/HBASE-21167
Project: HBase
Issue Type: Bug
Components: wal
Reporter: stack
Logging this in case we see it again. I had a Master working furiously. It had
assigned over 400k regions on startup. Then this happened which knocked the
hard-working server out:
{code}
2018-09-06 07:50:18,983 ERROR org.apache.hadoop.hbase.master.HMaster: Master
server abort: loaded coprocessors are:
[org.apache.hadoop.hbase.security.access.AccessController,
com.cloudera.navigator.audit.hbase.MasterAuditCoProcessor]
2018-09-06 07:50:18,983 ERROR org.apache.hadoop.hbase.master.HMaster: *
ABORTING master vc0207.halxg.cloudera.com,22001,1536173228913: IOE in log
roller *
java.io.IOException: Connection to 10.17.208.34/10.17.208.34:20002 closed
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$channelInactive$2(FanOutOneBlockAsyncDFSOutput.java:289)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.channelInactive(FanOutOneBlockAsyncDFSOutput.java:288)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
at
org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
at
org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
at
org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
at
org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
at
org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
at
org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
at
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1354)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at
[jira] [Updated] (HBASE-21165) During ProcedureStore load, there is no listing of progress...
[
https://issues.apache.org/jira/browse/HBASE-21165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-21165:
--
Description:
I have a Master that crashed on a large cluster with hundreds of outstanding
Procedure WALs and ~1M of Procedures to load. It is taking a long time (two
hours) to load...
There were STUCK procedures that were preventing clean-up of the old WALs.
I can tell we are making progress by enabling TRACE on the Procedure Store.
Better would be an emission as we made progress through the files with an
emission after every so many procedures loaded.
Seems like post-load, there is a long time spent sorting-out the Procedure
image... We are in finish for ages doing stuff like:
{code}
"master/vc0207:22001:becomeActiveMaster" #98 daemon prio=5 os_prio=0
tid=0x00d31800 nid=0x1efc0 runnable [0x7f0a3c17d000]
java.lang.Thread.State: RUNNABLE
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader$WalProcedureMap.removeFromMap(ProcedureWALFormatReader.java:837)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader$WalProcedureMap.fetchReady(ProcedureWALFormatReader.java:614)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader.finish(ProcedureWALFormatReader.java:201)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.load(ProcedureWALFormat.java:94)
at
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(WALProcedureStore.java:426)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:382)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:663)
at
org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1335)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:878)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2119)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:567)
at
org.apache.hadoop.hbase.master.HMaster$$Lambda$42/1930759883.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
{code}
and
{code}
"master/vc0207:22001:becomeActiveMaster" #98 daemon prio=5 os_prio=0
tid=0x00d31800 nid=0x1efc0 runnable [0x7f0a3c17d000]
java.lang.Thread.State: RUNNABLE
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.security.Provider$Service.newInstance(Provider.java:1595)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
at java.security.Security.getImpl(Security.java:695)
at java.security.MessageDigest.getInstance(MessageDigest.java:167)
at org.apache.hadoop.hbase.util.MD5Hash.getMD5AsHex(MD5Hash.java:59)
at
org.apache.hadoop.hbase.client.RegionInfo.createRegionName(RegionInfo.java:560)
at
org.apache.hadoop.hbase.client.RegionInfo.createRegionName(RegionInfo.java:490)
at
org.apache.hadoop.hbase.client.RegionInfoBuilder$MutableRegionInfo.(RegionInfoBuilder.java:243)
at
org.apache.hadoop.hbase.client.RegionInfoBuilder.build(RegionInfoBuilder.java:120)
at
org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toRegionInfo(ProtobufUtil.java:3132)
at
org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.deserializeStateData(ServerCrashProcedure.java:335)
at
org.apache.hadoop.hbase.procedure2.ProcedureUtil.convertToProcedure(ProcedureUtil.java:283)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader$Entry.convert(ProcedureWALFormatReader.java:359)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader$EntryIterator.next(ProcedureWALFormatReader.java:410)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.loadProcedures(ProcedureExecutor.java:460)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$200(ProcedureExecutor.java:76)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$1.load(ProcedureExecutor.java:391)
at
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore$2.load(WALProcedureStore.java:441)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader.finish(ProcedureWALFormatReader.java:202)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.load(ProcedureWALFormat.java:94)
at
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(WALProcedureStore.java:426)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:382)
at
[jira] [Commented] (HBASE-21167) Master killed after IOE in FanOutOneBlockAsyncDFSOutput on log roll
[
https://issues.apache.org/jira/browse/HBASE-21167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606676#comment-16606676
]
Duo Zhang commented on HBASE-21167:
---
In general I think we will retry many times when rolling a WAL?
> Master killed after IOE in FanOutOneBlockAsyncDFSOutput on log roll
> ---
>
> Key: HBASE-21167
> URL: https://issues.apache.org/jira/browse/HBASE-21167
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: stack
>Priority: Major
>
> Logging this in case we see it again. I had a Master working furiously. It
> had assigned over 400k regions on startup. Then this happened which knocked
> the hard-working server out:
> {code}
> 2018-09-06 07:50:18,983 ERROR org.apache.hadoop.hbase.master.HMaster: Master
> server abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.security.access.AccessController,
> com.cloudera.navigator.audit.hbase.MasterAuditCoProcessor]
> 2018-09-06 07:50:18,983 ERROR org.apache.hadoop.hbase.master.HMaster: *
> ABORTING master vc0207.halxg.cloudera.com,22001,1536173228913: IOE in log
> roller *
> java.io.IOException: Connection to 10.17.208.34/10.17.208.34:20002 closed
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$channelInactive$2(FanOutOneBlockAsyncDFSOutput.java:289)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.channelInactive(FanOutOneBlockAsyncDFSOutput.java:288)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
>
[jira] [Comment Edited] (HBASE-21167) Master killed after IOE in FanOutOneBlockAsyncDFSOutput on log roll
[
https://issues.apache.org/jira/browse/HBASE-21167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606695#comment-16606695
]
stack edited comment on HBASE-21167 at 9/7/18 4:28 AM:
---
[~Apache9] Thanks for taking a look. This is MasterProcWal. Maybe we don't.
I'll take a look.
was (Author: stack):
[~Apache9] Thanks for taking a look. This is MasterProcWal. Maybe we don't.
I'll take a look.
> Master killed after IOE in FanOutOneBlockAsyncDFSOutput on log roll
> ---
>
> Key: HBASE-21167
> URL: https://issues.apache.org/jira/browse/HBASE-21167
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: stack
>Priority: Major
>
> Logging this in case we see it again. I had a Master working furiously. It
> had assigned over 400k regions on startup. Then this happened which knocked
> the hard-working server out:
> {code}
> 2018-09-06 07:50:18,983 ERROR org.apache.hadoop.hbase.master.HMaster: Master
> server abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.security.access.AccessController,
> com.cloudera.navigator.audit.hbase.MasterAuditCoProcessor]
> 2018-09-06 07:50:18,983 ERROR org.apache.hadoop.hbase.master.HMaster: *
> ABORTING master vc0207.halxg.cloudera.com,22001,1536173228913: IOE in log
> roller *
> java.io.IOException: Connection to 10.17.208.34/10.17.208.34:20002 closed
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$channelInactive$2(FanOutOneBlockAsyncDFSOutput.java:289)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.channelInactive(FanOutOneBlockAsyncDFSOutput.java:288)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
>
[jira] [Commented] (HBASE-21167) Master killed after IOE in FanOutOneBlockAsyncDFSOutput on log roll
[
https://issues.apache.org/jira/browse/HBASE-21167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606695#comment-16606695
]
stack commented on HBASE-21167:
---
[~Apache9] Thanks for taking a look. This is MasterProcWal. Maybe we don't.
I'll take a look.
> Master killed after IOE in FanOutOneBlockAsyncDFSOutput on log roll
> ---
>
> Key: HBASE-21167
> URL: https://issues.apache.org/jira/browse/HBASE-21167
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: stack
>Priority: Major
>
> Logging this in case we see it again. I had a Master working furiously. It
> had assigned over 400k regions on startup. Then this happened which knocked
> the hard-working server out:
> {code}
> 2018-09-06 07:50:18,983 ERROR org.apache.hadoop.hbase.master.HMaster: Master
> server abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.security.access.AccessController,
> com.cloudera.navigator.audit.hbase.MasterAuditCoProcessor]
> 2018-09-06 07:50:18,983 ERROR org.apache.hadoop.hbase.master.HMaster: *
> ABORTING master vc0207.halxg.cloudera.com,22001,1536173228913: IOE in log
> roller *
> java.io.IOException: Connection to 10.17.208.34/10.17.208.34:20002 closed
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$channelInactive$2(FanOutOneBlockAsyncDFSOutput.java:289)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.channelInactive(FanOutOneBlockAsyncDFSOutput.java:288)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
> at
> org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
> at
>
[jira] [Updated] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21001:
--
Attachment: HBASE-21001.master.003.patch
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch,
> HBASE-21001.master.003.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21161) Enable the test added in HBASE-20741 that was removed accidentally
[
https://issues.apache.org/jira/browse/HBASE-21161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606048#comment-16606048
]
Hudson commented on HBASE-21161:
Results for branch branch-2
[build #1210 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1210/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1210//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1210//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1210//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Enable the test added in HBASE-20741 that was removed accidentally
> --
>
> Key: HBASE-21161
> URL: https://issues.apache.org/jira/browse/HBASE-21161
> Project: HBase
> Issue Type: Test
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21161.patch
>
>
> While giving an addendum to remove the timout in the test I mistakenly
> removed the @Test tag and so the test was not running after the first initial
> runs. During the first commit the @Test tag was there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20741) Split of a region with replicas creates all daughter regions and its replica in same server
[
https://issues.apache.org/jira/browse/HBASE-20741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606049#comment-16606049
]
Hudson commented on HBASE-20741:
Results for branch branch-2
[build #1210 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1210/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1210//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1210//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1210//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Split of a region with replicas creates all daughter regions and its replica
> in same server
> ---
>
> Key: HBASE-20741
> URL: https://issues.apache.org/jira/browse/HBASE-20741
> Project: HBase
> Issue Type: Bug
> Components: read replicas
>Affects Versions: 3.0.0, 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-20741.patch, HBASE-20741_1.patch,
> HBASE-20741_2.patch, HBASE-20741_2.patch, HBASE-20741_3.patch,
> HBASE-20741_4.patch, HBASE-20741_5.patch, HBASE-20741_5.patch,
> HBASE-20741_addendum.patch, HBASE-20741_branch-2.1.patch,
> HBASE-20741_branch-2.1.patch, HBASE-20741_branch-2.patch
>
>
> Generally it is better that the parent region when split creates the daughter
> region in the same target server.
> But for replicas also we do the same and all the replica regions are created
> in the same target server. We should ideally be doing a round robin and only
> the primary daughter region should be opened in the intended target server
> (where the parent was previously opened).
> [~huaxiang] FYI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606047#comment-16606047
]
Ted Yu commented on HBASE-21001:
+1
Thanks for working late.
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch,
> HBASE-21001.master.003.patch, HBASE-21001.master.004.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21162) Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default
[ https://issues.apache.org/jira/browse/HBASE-21162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21162: --- Status: Patch Available (was: Open) > Revert suspicious change to BoundedByteBufferPool and disable use of direct > buffers for IPC reservoir by default > > > Key: HBASE-21162 > URL: https://issues.apache.org/jira/browse/HBASE-21162 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.7 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Critical > Fix For: 1.5.0, 1.4.8 > > Attachments: HBASE-21162-branch-1.patch > > > We had a production incident where we traced the issue to a direct buffer > leak. On a hunch we tried setting hbase.ipc.server.reservoir.enabled = false > and after that no native memory leak could be observed in any regionserver > process under the triggering load. > On HBASE-19239 (Fix findbugs and error-prone issues) I made a change to > BoundedByteBufferPool that is suspicious given this finding. It was committed > to branch-1.4 and branch-1. I'm going to revert this change. > In addition the allocation of direct memory for the server RPC reservoir is a > bit problematic in that tracing native memory or direct buffer leaks to a > particular class or compilation unit is difficult, so I also propose > allocating the reservoir on the heap by default instead. Should there be a > leak it is much easier to do an analysis of a heap dump with familiar tools > to find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605319#comment-16605319
]
Wei-Chiu Chuang commented on HBASE-21001:
-
[~andrewcheng] thanks for the patch. Looks good to me.
Any chance to add a test? I mean, this regression went unnoticed because there
is no existing test that covers it.
Not sure if it is feasible given that ReplicationObserver and HRegion are in
different modules. A test can be added separately in a new jira or in this
patch. Either way works for me.
+1 (non-binding)
Thanks
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
[
https://issues.apache.org/jira/browse/HBASE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605368#comment-16605368
]
Hadoop QA commented on HBASE-21155:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
55s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
42s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 42s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}124m 10s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:42ca976 |
| JIRA Issue | HBASE-21155 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938576/HBASE-21155.branch-2.1.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c99c9a958402 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.1 / 4d7221a68f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/14325/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results
[jira] [Reopened] (HBASE-15728) Add remaining per-table region / store / flush / compaction related metrics
[
https://issues.apache.org/jira/browse/HBASE-15728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reopened HBASE-15728:
-
Reopening because this caused a bunch of findbugs errors on master and branch-2.
precommit failed to run findbugs originally (noted in the "no findbugs
executables" comment from qabot), and then the master branch nightly failed due
to fall out from the ASF jenkins outage last week (noted in the "something went
wrong, please look at this" comment), but branch-2 nightly did give results
([this is the report it nightly attempted to link
to|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1179/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/])
{code}
Tests
FindBugsmodule:hbase-hadoop2-compat
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.compactedInputBytes;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 406]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.compactedOutputBytes;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 424]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.compactionInputFileCountHisto;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 397]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.compactionInputSizeHisto;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 405]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.compactionOutputFileCountHisto;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 415]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.compactionOutputSizeHisto;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 423]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.compactionTimeHisto;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 389]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.flushMemstoreSizeHisto;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 377]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.flushOutputSizeHisto;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 383]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.flushedMemstoreBytes;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 378]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.flushedOutputBytes;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 384]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.majorCompactedInputBytes;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 409]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.majorCompactedOutputBytes;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 427]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.majorCompactionInputFileCountHisto;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 399]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.majorCompactionInputSizeHisto;
locked 50% of time Unsynchronized access at MetricsTableSourceImpl.java:50% of
time Unsynchronized access at MetricsTableSourceImpl.java:[line 408]
Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.majorCompactionOutputFileCountHisto;
locked
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605310#comment-16605310
]
stack commented on HBASE-21154:
---
Folding in acl too sounds good.
bq. thought I had heard chatter of moving that stuff into an optional system
table.
Yeah. Needs to come down out of zk. Table state has made the transition.
bq. The goal here is to avoid having to handle startup ordering, correct?
Yes.
bq. If we split meta or introduce a new system table, we will still have to
handle the startup ordering eventually.
Idea is one system table to worry about only. If split, then only one split
system table to worry about (smile) rather than a split meta AND ns but ns has
to come after meta AND acl AND (I think we're also trying to get away w/ no
system tables but meta). It'll add some data to the meta but we need to split
meta for other reasons anyway (fsredo).
Let me try and write something up after hacking some. Probably best to do this
AFTER meta is made dynamic (needs to be dynamic rather than hardcoded so we can
split meta) so we can add a CF per concern. We might also want to prefix all
rows with something so all rows are clumped rather than spread across
regions Will be back.
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21140) Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to branch-1 . (and backport HBASE-15728 for branch-1)
[ https://issues.apache.org/jira/browse/HBASE-21140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605312#comment-16605312 ] Sean Busbey commented on HBASE-21140: - please make sure any patch here runs successfully through findbugs before being pushed. I just reopened HBASE-15728 because it introduced failures on master and branch-2. > Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to > branch-1 . (and backport HBASE-15728 for branch-1) > --- > > Key: HBASE-21140 > URL: https://issues.apache.org/jira/browse/HBASE-21140 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Duo Zhang >Assignee: Xu Cang >Priority: Major > Fix For: 1.5.0 > > Attachments: > HBASE-21140.diff_against_cf198a65e8d704d28538c4c165a941b9e5bac678.branch-1.001.patch > > > There is no computeIfAbsent method on branch-1 as we still need to support > JDK7, so the fix will be different with branch-2+. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21061:
Resolution: Fixed
Fix Version/s: 1.2.7
1.4.8
1.3.3
1.5.0
Status: Resolved (was: Patch Available)
pushed to relevant branches, with the fix Mike requested.
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.3.3, 1.2.7, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 1.5.0, 1.3.3, 1.4.8, 1.2.7
>
> Attachments: HBASE-21061-branch-1.v0.patch
>
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605388#comment-16605388
]
Hudson commented on HBASE-21061:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #469 (See
[https://builds.apache.org/job/HBase-1.3-IT/469/])
HBASE-21061 Fix inconsistent synchronization in RpcServer (busbey: rev
db45f2f4710eb53f979ab59a4939e423ac57ea4b)
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.3.3, 1.2.7, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 1.5.0, 1.3.3, 1.4.8, 1.2.7
>
> Attachments: HBASE-21061-branch-1.v0.patch
>
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21162) Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default
[ https://issues.apache.org/jira/browse/HBASE-21162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606582#comment-16606582 ] Andrew Purtell edited comment on HBASE-21162 at 9/7/18 12:49 AM: - I think the change is suspicious and should be reverted. Even if we are not allocating from direct buffers we could still leak heap was (Author: apurtell): We want the reservoir for its original purpose of avoiding allocations per RPC, so it should be enabled; but as I suggest not allocated via direct buffers by default. > Revert suspicious change to BoundedByteBufferPool and disable use of direct > buffers for IPC reservoir by default > > > Key: HBASE-21162 > URL: https://issues.apache.org/jira/browse/HBASE-21162 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.7 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Critical > Fix For: 1.5.0, 1.4.8 > > Attachments: HBASE-21162-branch-1.patch > > > We had a production incident where we traced the issue to a direct buffer > leak. On a hunch we tried setting hbase.ipc.server.reservoir.enabled = false > and after that no native memory leak could be observed in any regionserver > process under the triggering load. > On HBASE-19239 (Fix findbugs and error-prone issues) I made a change to > BoundedByteBufferPool that is suspicious given this finding. It was committed > to branch-1.4 and branch-1. I'm going to revert this change. > In addition the allocation of direct memory for the server RPC reservoir is a > bit problematic in that tracing native memory or direct buffer leaks to a > particular class or compilation unit is difficult, so I also propose > allocating the reservoir on the heap by default instead. Should there be a > leak it is much easier to do an analysis of a heap dump with familiar tools > to find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21162) Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default
[ https://issues.apache.org/jira/browse/HBASE-21162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606582#comment-16606582 ] Andrew Purtell commented on HBASE-21162: We want the reservoir for its original purpose of avoiding allocations per RPC, so it should be enabled; but as I suggest not allocated via direct buffers by default. > Revert suspicious change to BoundedByteBufferPool and disable use of direct > buffers for IPC reservoir by default > > > Key: HBASE-21162 > URL: https://issues.apache.org/jira/browse/HBASE-21162 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.7 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Critical > Fix For: 1.5.0, 1.4.8 > > Attachments: HBASE-21162-branch-1.patch > > > We had a production incident where we traced the issue to a direct buffer > leak. On a hunch we tried setting hbase.ipc.server.reservoir.enabled = false > and after that no native memory leak could be observed in any regionserver > process under the triggering load. > On HBASE-19239 (Fix findbugs and error-prone issues) I made a change to > BoundedByteBufferPool that is suspicious given this finding. It was committed > to branch-1.4 and branch-1. I'm going to revert this change. > In addition the allocation of direct memory for the server RPC reservoir is a > bit problematic in that tracing native memory or direct buffer leaks to a > particular class or compilation unit is difficult, so I also propose > allocating the reservoir on the heap by default instead. Should there be a > leak it is much easier to do an analysis of a heap dump with familiar tools > to find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21166) Creating a CoprocessorHConnection re-retrieves the cluster id from ZK
[ https://issues.apache.org/jira/browse/HBASE-21166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606591#comment-16606591 ] Andrew Purtell commented on HBASE-21166: Ugh, +1 > Creating a CoprocessorHConnection re-retrieves the cluster id from ZK > - > > Key: HBASE-21166 > URL: https://issues.apache.org/jira/browse/HBASE-21166 > Project: HBase > Issue Type: Bug >Affects Versions: 1.5.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Major > Attachments: 21166-1.5.txt > > > CoprocessorHConnections are created for example during a call of > CoprocessorHost$Environent.getTable(...). The region server already know the > cluster id, yet, we're resolving it over and over again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606610#comment-16606610
]
Mingliang Liu commented on HBASE-21138:
---
Thanks for helpful review comments, and commit.
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0
>
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch, HBASE-21138.003.patch, HBASE-21138.004.patch,
> HBASE-21138.branch-1.004.patch, HBASE-21138.branch-1.004.patch,
> HBASE-21138.branch-2.004.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18441) ZookeeperWatcher#interruptedException should throw exception
[
https://issues.apache.org/jira/browse/HBASE-18441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18441:
Component/s: Zookeeper
> ZookeeperWatcher#interruptedException should throw exception
>
>
> Key: HBASE-18441
> URL: https://issues.apache.org/jira/browse/HBASE-18441
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Major
> Fix For: 1.4.0, 2.0.0, 1.2.7
>
> Attachments: HBASE-18441.patch, HBASE-18441.trivial.patch
>
>
> Currently Zookeeper#interruptedException will swallow the
> InterruptedException and only log, which might cause unexpected behavior,
> such as when invoking {{ZKUtil#checkExists}} and the watcher thread somehow
> interrupted, the method will return -1 which means the checked znode doesn't
> exist, while actually the znode exists.
> We could also see a TODO tag in the javadoc, which indicates we need some
> fix/improvement here:
> {code}
> /**
>* Handles InterruptedExceptions in client calls.
>*
>* This may be temporary but for now this gives one place to deal with
> these.
>*
>* TODO: Currently, this method does nothing.
>* Is this ever expected to happen? Do we abort or can we let it run?
>* Maybe this should be logged as WARN? It shouldn't happen?
>*
>* @param ie
>*/
> {code}
> Here we propose to throw a {{KeeperException$SystemErrorException}} in
> {{ZookeeperWatcher#interruptedException}}, and will add a UT case to cover
> the interruption scenario.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18441) ZookeeperWatcher#interruptedException should throw exception
[
https://issues.apache.org/jira/browse/HBASE-18441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18441:
Fix Version/s: 1.2.7
> ZookeeperWatcher#interruptedException should throw exception
>
>
> Key: HBASE-18441
> URL: https://issues.apache.org/jira/browse/HBASE-18441
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Major
> Fix For: 1.4.0, 2.0.0, 1.2.7
>
> Attachments: HBASE-18441.patch, HBASE-18441.trivial.patch
>
>
> Currently Zookeeper#interruptedException will swallow the
> InterruptedException and only log, which might cause unexpected behavior,
> such as when invoking {{ZKUtil#checkExists}} and the watcher thread somehow
> interrupted, the method will return -1 which means the checked znode doesn't
> exist, while actually the znode exists.
> We could also see a TODO tag in the javadoc, which indicates we need some
> fix/improvement here:
> {code}
> /**
>* Handles InterruptedExceptions in client calls.
>*
>* This may be temporary but for now this gives one place to deal with
> these.
>*
>* TODO: Currently, this method does nothing.
>* Is this ever expected to happen? Do we abort or can we let it run?
>* Maybe this should be logged as WARN? It shouldn't happen?
>*
>* @param ie
>*/
> {code}
> Here we propose to throw a {{KeeperException$SystemErrorException}} in
> {{ZookeeperWatcher#interruptedException}}, and will add a UT case to cover
> the interruption scenario.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[ https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20162: Fix Version/s: (was: 1.2.8) 1.2.7 > [nightly] depending on pipeline execution we sometimes refer to the wrong > workspace > --- > > Key: HBASE-20162 > URL: https://issues.apache.org/jira/browse/HBASE-20162 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.4.3, 2.0.0, 1.2.7 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 1.3.2, 1.5.0, 1.4.3, 2.0.0, 1.2.7 > > Attachments: HBASE-20162.0.patch > > > we set BASEDIR at the top of our pipeline to point at the component checkout > within WORKSPACE. > but! > a) at that point WORKSPACE is the workspace for the launching task > b) sometimes our parallel executions get a task with a different local > WORKSPACE to allow for coexisting on the same build host > c) when this happens our parallel stages are referring to some other absolute > path on the host > d) in most cases we're referring to dev-support files like e.g. the nightly > build script or machine info script that are the same across branches so > things are fine if we aren't running at the same time as a job that's > overwritting them > e) we also refer to the Dockerfile this way, so weird bugs I'm sure. > f) we build the source tarball from here, so that's probably broken subtly > g) sometimes that other directory _doesn't exist at all_ and we fail with > confusing messages about stuff not found -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20261) Table page (table.jsp) in Master UI does not show replicaIds for hbase meta table
[ https://issues.apache.org/jira/browse/HBASE-20261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20261: Fix Version/s: 1.2.7 > Table page (table.jsp) in Master UI does not show replicaIds for hbase meta > table > - > > Key: HBASE-20261 > URL: https://issues.apache.org/jira/browse/HBASE-20261 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Minor > Fix For: 1.5.0, 2.0.0, 1.2.7 > > Attachments: HBASE-20261.master.001.patch, Screen Shot 2018-03-23 at > 12.05.27.png > > > When region replication is enabled for hbase meta table, the table page > (table.jsp) in Master UI does not show replicaIds for hbase meta table. It > only shows the header: > !Screen Shot 2018-03-23 at 12.05.27.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20164) failed hadoopcheck should add footer link
[ https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20164: Fix Version/s: (was: 1.2.8) 1.2.7 > failed hadoopcheck should add footer link > - > > Key: HBASE-20164 > URL: https://issues.apache.org/jira/browse/HBASE-20164 > Project: HBase > Issue Type: Bug > Components: community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 1.3.2, 1.5.0, 1.4.3, 2.0.0, 1.2.7 > > Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch, > HBASE-20164.wip.patch > > > thought for sure this already had an issue, [~busbey], but I can't find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21166) Creating a CoprocessorHConnection re-retrieves the cluster id from ZK
[ https://issues.apache.org/jira/browse/HBASE-21166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606578#comment-16606578 ] stack commented on HBASE-21166: --- +1 for everywhere. > Creating a CoprocessorHConnection re-retrieves the cluster id from ZK > - > > Key: HBASE-21166 > URL: https://issues.apache.org/jira/browse/HBASE-21166 > Project: HBase > Issue Type: Bug >Affects Versions: 1.5.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Major > Attachments: 21166-1.5.txt > > > CoprocessorHConnections are created for example during a call of > CoprocessorHost$Environent.getTable(...). The region server already know the > cluster id, yet, we're resolving it over and over again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606579#comment-16606579
]
Hadoop QA commented on HBASE-21138:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
24s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} branch-1 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} branch-1 passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
53s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-1 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} branch-1 passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
21s{color} | {color:red} hbase-server: The patch generated 13 new + 67
unchanged - 14 fixed = 80 total (was 81) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}127m 34s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 18s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.coprocessor.TestMetaTableMetrics |
| | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21138 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938725/HBASE-21138.branch-1.004.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs
[jira] [Updated] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-21138:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.2.0
1.5.0
3.0.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Mingliang.
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0
>
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch, HBASE-21138.003.patch, HBASE-21138.004.patch,
> HBASE-21138.branch-1.004.patch, HBASE-21138.branch-1.004.patch,
> HBASE-21138.branch-2.004.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606603#comment-16606603
]
Hadoop QA commented on HBASE-21098:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
17s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} The patch hbase-common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} hbase-server: The patch generated 0 new + 283
unchanged - 1 fixed = 283 total (was 284) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
53s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
52s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 44s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
39s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}132m 50s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 23s{color}
| {color:red}
[jira] [Commented] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606604#comment-16606604
]
Guangxu Cheng commented on HBASE-21001:
---
QA looks good. I will commit it late this day if no other comments.Thanks
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch,
> HBASE-21001.master.003.patch, HBASE-21001.master.004.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21143) Update findbugs-maven-plugin to 3.0.4
[
https://issues.apache.org/jira/browse/HBASE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21143:
--
Resolution: Fixed
Fix Version/s: 2.2.0
3.0.0
Status: Resolved (was: Patch Available)
> Update findbugs-maven-plugin to 3.0.4
> -
>
> Key: HBASE-21143
> URL: https://issues.apache.org/jira/browse/HBASE-21143
> Project: HBase
> Issue Type: Bug
> Components: pom
>Affects Versions: 3.0.0, 2.1.0, 2.2.0, 2.0.2
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21143.master.001.patch
>
>
> {code}
> Failed to execute goal org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs
> (default) on project hbase: Execution default of goal
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs failed: Plugin
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0 or one of its dependencies
> could not be resolved: Failed to collect dependencies at
> org.codehaus.mojo:findbugs-maven-plugin:jar:3.0.0 ->
> org.codehaus.groovy:groovy-all:jar:1.7.4: Failed to read artifact descriptor
> for org.codehaus.groovy:groovy-all:jar:1.7.4: Could not transfer artifact
> org.codehaus.groovy:groovy-all:pom:1.7.4 from/to mirror
> (http://xxx..xxx/nexus/content/groups/public): Failed to transfer file:
> http://xxx..xxx/nexus/content/groups/public/org/codehaus/groovy/groovy-all/1.7.4/groovy-all-1.7.4.pom.
> Return code is: 418 , ReasonPhrase:Artifact is in Tencent Blacklist! Please
> update to the safe version, more information:
> http://xxx..xxx/?tab=blackList.
> {code}
> Recently, when I compile HBase with a new machine, I got the above error.
> Since the machine could not connect to the external network, we visited our
> internal Maven repository, but org.codehaus.groovy:groovy-all:jar:1.7.4 was
> added to the blacklist and could not be downloaded. See details,
> org.codehaus.groovy:groovy-all:jar:1.7.4 is marked as vulnerable by
> [CVE-2015-3253|https://www.cvedetails.com/cve/CVE-2015-3253], so we should
> upgrade the version.
> {code:xml}
> org.codehaus.mojo
> findbugs-maven-plugin
> 3.0.0
>
>
>
> ${project.basedir}/../dev-support/findbugs-exclude.xml
> {code}
> Look at the history commit record, findbugs-maven-plugin has been upgraded to
> 3.0.4 in HBASE-18264, but one place is missing which still using the version
> of 3.0.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21143) Update findbugs-maven-plugin to 3.0.4
[
https://issues.apache.org/jira/browse/HBASE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606616#comment-16606616
]
Guangxu Cheng commented on HBASE-21143:
---
pushed to master and branch-2.
Thank [~mdrob] for review.
> Update findbugs-maven-plugin to 3.0.4
> -
>
> Key: HBASE-21143
> URL: https://issues.apache.org/jira/browse/HBASE-21143
> Project: HBase
> Issue Type: Bug
> Components: pom
>Affects Versions: 3.0.0, 2.1.0, 2.2.0, 2.0.2
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21143.master.001.patch
>
>
> {code}
> Failed to execute goal org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs
> (default) on project hbase: Execution default of goal
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs failed: Plugin
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0 or one of its dependencies
> could not be resolved: Failed to collect dependencies at
> org.codehaus.mojo:findbugs-maven-plugin:jar:3.0.0 ->
> org.codehaus.groovy:groovy-all:jar:1.7.4: Failed to read artifact descriptor
> for org.codehaus.groovy:groovy-all:jar:1.7.4: Could not transfer artifact
> org.codehaus.groovy:groovy-all:pom:1.7.4 from/to mirror
> (http://xxx..xxx/nexus/content/groups/public): Failed to transfer file:
> http://xxx..xxx/nexus/content/groups/public/org/codehaus/groovy/groovy-all/1.7.4/groovy-all-1.7.4.pom.
> Return code is: 418 , ReasonPhrase:Artifact is in Tencent Blacklist! Please
> update to the safe version, more information:
> http://xxx..xxx/?tab=blackList.
> {code}
> Recently, when I compile HBase with a new machine, I got the above error.
> Since the machine could not connect to the external network, we visited our
> internal Maven repository, but org.codehaus.groovy:groovy-all:jar:1.7.4 was
> added to the blacklist and could not be downloaded. See details,
> org.codehaus.groovy:groovy-all:jar:1.7.4 is marked as vulnerable by
> [CVE-2015-3253|https://www.cvedetails.com/cve/CVE-2015-3253], so we should
> upgrade the version.
> {code:xml}
> org.codehaus.mojo
> findbugs-maven-plugin
> 3.0.0
>
>
>
> ${project.basedir}/../dev-support/findbugs-exclude.xml
> {code}
> Look at the history commit record, findbugs-maven-plugin has been upgraded to
> 3.0.4 in HBASE-18264, but one place is missing which still using the version
> of 3.0.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21158) Empty qualifier cell is always returned when using QualifierFilter
[
https://issues.apache.org/jira/browse/HBASE-21158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21158:
--
Attachment: HBASE-21158.master.002.patch
> Empty qualifier cell is always returned when using QualifierFilter
> --
>
> Key: HBASE-21158
> URL: https://issues.apache.org/jira/browse/HBASE-21158
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21158.master.001.patch,
> HBASE-21158.master.002.patch
>
>
> {code:xml}
> hbase(main):002:0> put 'testTable','testrow','f:testcol1','testvalue1'
> 0 row(s) in 0.0040 seconds
> hbase(main):003:0> put 'testTable','testrow','f:','testvalue2'
> 0 row(s) in 0.0070 seconds
> # get row with empty column f:, result is correct.
> hbase(main):004:0> scan 'testTable',{FILTER => "QualifierFilter (=,
> 'binary:')"}
> ROW COLUMN+CELL
>
>
> testrowcolumn=f:,
> timestamp=1536218563581, value=testvalue2
>
> 1 row(s) in 0.0460 seconds
> # get row with column f:testcol1, result is incorrect.
> hbase(main):005:0> scan 'testTable',{FILTER => "QualifierFilter (=,
> 'binary:testcol1')"}
> ROW COLUMN+CELL
>
>
> testrowcolumn=f:,
> timestamp=1536218563581, value=testvalue2
>
> testrowcolumn=f:testcol1,
> timestamp=1536218550827, value=testvalue1
>
> 1 row(s) in 0.0070 seconds
> {code}
> As the above operation, when the row contains empty qualifier column, empty
> qualifier cell is always returned when using QualifierFilter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18099) FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish
[
https://issues.apache.org/jira/browse/HBASE-18099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18099:
Fix Version/s: 1.2.7
> FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish
> -
>
> Key: HBASE-18099
> URL: https://issues.apache.org/jira/browse/HBASE-18099
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 1.4.0, 2.0.0, 1.2.7
>
> Attachments: 18099.v1.txt, 18099.v2.txt, 18099.v3.txt, 18099.v4.txt
>
>
> In the following thread:
> http://search-hadoop.com/m/HBase/YGbbMXkeHlI9zo
> Jacob described the scenario where data from certain region were missing in
> the snapshot.
> Here was related region server log:
> https://pastebin.com/1ECXjhRp
> He pointed out that concurrent flush from MemStoreFlusher.1 thread was not
> initiated from the thread pool for snapshot.
> In RegionSnapshotTask#call() method there is this:
> {code}
> region.flush(true);
> {code}
> The return value is not checked.
> In HRegion#flushcache(), Result.CANNOT_FLUSH may be returned due to:
> {code}
> String msg = "Not flushing since "
> + (writestate.flushing ? "already flushing"
> : "writes not enabled");
> {code}
> This implies that FlushSnapshotSubprocedure may incorrectly skip waiting for
> the concurrent flush to complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21158) Empty qualifier cell is always returned when using QualifierFilter
[
https://issues.apache.org/jira/browse/HBASE-21158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606626#comment-16606626
]
Guangxu Cheng commented on HBASE-21158:
---
Attach 002 patch to fix checkstyle warnings.Thanks
> Empty qualifier cell is always returned when using QualifierFilter
> --
>
> Key: HBASE-21158
> URL: https://issues.apache.org/jira/browse/HBASE-21158
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21158.master.001.patch,
> HBASE-21158.master.002.patch
>
>
> {code:xml}
> hbase(main):002:0> put 'testTable','testrow','f:testcol1','testvalue1'
> 0 row(s) in 0.0040 seconds
> hbase(main):003:0> put 'testTable','testrow','f:','testvalue2'
> 0 row(s) in 0.0070 seconds
> # get row with empty column f:, result is correct.
> hbase(main):004:0> scan 'testTable',{FILTER => "QualifierFilter (=,
> 'binary:')"}
> ROW COLUMN+CELL
>
>
> testrowcolumn=f:,
> timestamp=1536218563581, value=testvalue2
>
> 1 row(s) in 0.0460 seconds
> # get row with column f:testcol1, result is incorrect.
> hbase(main):005:0> scan 'testTable',{FILTER => "QualifierFilter (=,
> 'binary:testcol1')"}
> ROW COLUMN+CELL
>
>
> testrowcolumn=f:,
> timestamp=1536218563581, value=testvalue2
>
> testrowcolumn=f:testcol1,
> timestamp=1536218550827, value=testvalue1
>
> 1 row(s) in 0.0070 seconds
> {code}
> As the above operation, when the row contains empty qualifier column, empty
> qualifier cell is always returned when using QualifierFilter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18467) nightly job needs to run all stages and then comment on jira
[ https://issues.apache.org/jira/browse/HBASE-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18467: Fix Version/s: (was: 1.2.8) 1.2.7 > nightly job needs to run all stages and then comment on jira > > > Key: HBASE-18467 > URL: https://issues.apache.org/jira/browse/HBASE-18467 > Project: HBase > Issue Type: Improvement > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 1.3.2, 1.5.0, 1.4.3, 2.0.0, 1.2.7 > > Attachments: HBASE-18467.0.WIP.patch, HBASE-18467.0.patch, > HBASE-18467.1.patch, HBASE-18467.1.patch, HBASE-18467.2.patch, > HBASE-18467.3.patch, HBASE-18467.4.patch, HBASE-18467.5.patch, > HBASE-18467.6.patch, HBASE-18467.7.addendum.patch > > > follow on from HBASE-18147, need a post action that pings all newly-committed > jiras with result of the branch build -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18568) Correct metric of numRegions
[
https://issues.apache.org/jira/browse/HBASE-18568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18568:
Fix Version/s: 1.2.7
> Correct metric of numRegions
> --
>
> Key: HBASE-18568
> URL: https://issues.apache.org/jira/browse/HBASE-18568
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 3.0.0
>Reporter: Shibin Zhang
>Assignee: Shibin Zhang
>Priority: Critical
> Fix For: 1.4.0, 2.0.0-alpha-3, 2.0.0, 1.2.7
>
> Attachments: HBASE-18568-V1.patch
>
>
> i found the value of metric numReigons in Regions is not correct.
> the metric can not add or remove region correctly as region close or open.
> the metric as follow:
> "name" : "Hadoop:service=HBase,name=RegionServer,sub=Regions",
> "numRegions" : 2,
> after trouble shooting ,i found the reason is in
> MetricsRegionSourceImpl#MetricsRegionSourceImpl
> {code:java}
> agg.register(this);
> ...
> hashCode = regionWrapper.getRegionHashCode();
> {code}
> when add the MetricsRegionSource to set ,but the hashCode has not yet
> initialized.
> So, the setFromMap can not put or remove the object correctly.
> it will be better like this :
> {code:java}
> hashCode = regionWrapper.getRegionHashCode();
> agg.register(this);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20075) remove logic for branch-1.1 nightly testing
[ https://issues.apache.org/jira/browse/HBASE-20075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20075: Fix Version/s: (was: 1.2.8) 1.2.7 > remove logic for branch-1.1 nightly testing > --- > > Key: HBASE-20075 > URL: https://issues.apache.org/jira/browse/HBASE-20075 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 1.1.13 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 1.3.2, 1.5.0, 1.4.3, 2.0.0, 1.2.7 > > Attachments: HBASE-20075.0.patch > > > since branch-1.1 is EOM, remove the branch-1.1 specific logic from our > Jenkinsfile and delete the Jenkinsfile in branch-1.1 so we'll stop running > nightlies for it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20583) SplitLogWorker should handle FileNotFoundException when split a wal
[
https://issues.apache.org/jira/browse/HBASE-20583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20583:
Component/s: wal
> SplitLogWorker should handle FileNotFoundException when split a wal
> ---
>
> Key: HBASE-20583
> URL: https://issues.apache.org/jira/browse/HBASE-20583
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.0.1, 1.2.7
>
> Attachments: HBASE-20583.master.001.patch,
> HBASE-20583.master.001.patch
>
>
> When a split task is finished, master will delete the wal first, then remove
> the task's zk node. So if master crashed after delelte the wal, the zk task
> node may be leaved on zk. When master resubmit this task, the task will
> failed by FileNotFoundException.
> We also handle FileNotFoundException in WALSplitter. But not handle this in
> SplitLogWorker.
>
> {code:java}
> try {
> in = getReader(path, reporter);
> } catch (EOFException e) {
> if (length <= 0) {
> // TODO should we ignore an empty, not-last log file if skip.errors
> // is false? Either way, the caller should decide what to do. E.g.
> // ignore if this is the last log in sequence.
> // TODO is this scenario still possible if the log has been
> // recovered (i.e. closed)
> LOG.warn("Could not open {} for reading. File is empty", path, e);
> }
> // EOFException being ignored
> return null;
> }
> } catch (IOException e) {
> if (e instanceof FileNotFoundException) {
> // A wal file may not exist anymore. Nothing can be recovered so move on
> LOG.warn("File {} does not exist anymore", path, e);
> return null;
> }
> }{code}
> {code:java}
> // Here fs.getFileStatus may throw FileNotFoundException, too. We should
> handle this exception as the WALSplitter.getReader.
> try {
> if (!WALSplitter.splitLogFile(walDir, fs.getFileStatus(new Path(walDir,
> filename)),
> fs, conf, p, sequenceIdChecker,
> server.getCoordinatedStateManager().getSplitLogWorkerCoordination(),
> factory)) {
> return Status.PREEMPTED;
> }
> }
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20505: Fix Version/s: 1.2.7 > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test > Components: Performance >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5, 1.2.7 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20505: Component/s: Performance > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test > Components: Performance >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5, 1.2.7 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20583) SplitLogWorker should handle FileNotFoundException when split a wal
[
https://issues.apache.org/jira/browse/HBASE-20583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20583:
Fix Version/s: 1.2.7
> SplitLogWorker should handle FileNotFoundException when split a wal
> ---
>
> Key: HBASE-20583
> URL: https://issues.apache.org/jira/browse/HBASE-20583
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.0.1, 1.2.7
>
> Attachments: HBASE-20583.master.001.patch,
> HBASE-20583.master.001.patch
>
>
> When a split task is finished, master will delete the wal first, then remove
> the task's zk node. So if master crashed after delelte the wal, the zk task
> node may be leaved on zk. When master resubmit this task, the task will
> failed by FileNotFoundException.
> We also handle FileNotFoundException in WALSplitter. But not handle this in
> SplitLogWorker.
>
> {code:java}
> try {
> in = getReader(path, reporter);
> } catch (EOFException e) {
> if (length <= 0) {
> // TODO should we ignore an empty, not-last log file if skip.errors
> // is false? Either way, the caller should decide what to do. E.g.
> // ignore if this is the last log in sequence.
> // TODO is this scenario still possible if the log has been
> // recovered (i.e. closed)
> LOG.warn("Could not open {} for reading. File is empty", path, e);
> }
> // EOFException being ignored
> return null;
> }
> } catch (IOException e) {
> if (e instanceof FileNotFoundException) {
> // A wal file may not exist anymore. Nothing can be recovered so move on
> LOG.warn("File {} does not exist anymore", path, e);
> return null;
> }
> }{code}
> {code:java}
> // Here fs.getFileStatus may throw FileNotFoundException, too. We should
> handle this exception as the WALSplitter.getReader.
> try {
> if (!WALSplitter.splitLogFile(walDir, fs.getFileStatus(new Path(walDir,
> filename)),
> fs, conf, p, sequenceIdChecker,
> server.getCoordinatedStateManager().getSplitLogWorkerCoordination(),
> factory)) {
> return Status.PREEMPTED;
> }
> }
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20597) Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
[ https://issues.apache.org/jira/browse/HBASE-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20597: Fix Version/s: 1.2.7 > Use a lock to serialize access to a shared reference to ZooKeeperWatcher in > HBaseReplicationEndpoint > > > Key: HBASE-20597 > URL: https://issues.apache.org/jira/browse/HBASE-20597 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.2, 1.4.4 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5, 1.2.7 > > Attachments: HBASE-20597-branch-1.addendum-v2.0.patch, > HBASE-20597-branch-1.patch, HBASE-20597.addendum.0.patch, HBASE-20597.patch > > > The code that closes down a ZKW that fails to initialize when attempting to > connect to the remote cluster is not MT safe and can in theory leak > ZooKeeperWatcher instances. The allocation of a new ZKW and store to the > reference is not atomic. Might have concurrent allocations with only one > winning store, leading to leaked ZKW instances. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21160) Assertion in TestVisibilityLabelsWithDeletes#testDeleteColumnsWithoutAndWithVisibilityLabels is ignored
[
https://issues.apache.org/jira/browse/HBASE-21160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liubangchen reassigned HBASE-21160:
---
Assignee: liubangchen
> Assertion in
> TestVisibilityLabelsWithDeletes#testDeleteColumnsWithoutAndWithVisibilityLabels
> is ignored
> ---
>
> Key: HBASE-21160
> URL: https://issues.apache.org/jira/browse/HBASE-21160
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Trivial
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/14327/artifact/patchprocess/diff-compile-javac-hbase-server.txt
> (HBASE-21138 QA run):
> {code}
> [WARNING]
> /testptch/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDeletes.java:[315,25]
> [AssertionFailureIgnored] This assertion throws an AssertionError if it
> fails, which will be caught by an enclosing try block.
> {code}
> Here is related code:
> {code}
> PrivilegedExceptionAction scanAction = new
> PrivilegedExceptionAction() {
> @Override
> public Void run() throws Exception {
> try (Connection connection =
> ConnectionFactory.createConnection(conf);
> ...
> assertEquals(1, next.length);
> } catch (Throwable t) {
> throw new IOException(t);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-21059) Findbugs failures on branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-21059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved HBASE-21059.
-
Resolution: Fixed
as of build #460 for branch-1.2 nightly we're back to no findbugs failures. yay!
> Findbugs failures on branch-1.2
> ---
>
> Key: HBASE-21059
> URL: https://issues.apache.org/jira/browse/HBASE-21059
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, rpc
>Affects Versions: 1.2.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 1.2.7
>
>
> nightly shows two failures for branch-1.2 findbugs check:
> {code}
> FindBugs module:hbase-server
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java:[line 458]
> Dead store to fsSet in
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
> At
> SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
> At SecureBulkLoadEndpoint.java:[line 145]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21157) Split TableInputFormatScan to individual tests
[
https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605709#comment-16605709
]
Hudson commented on HBASE-21157:
Results for branch branch-2.0
[build #776 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/776/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/776//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/776//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/776//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Split TableInputFormatScan to individual tests
> --
>
> Key: HBASE-21157
> URL: https://issues.apache.org/jira/browse/HBASE-21157
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21157.patch
>
>
> We have done a split in HBASE-8326, which split the test to two parts. But it
> is still a bit slow, split it into several tests can increase the parallelism
> and make the 'mvn test' run faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21140) Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to branch-1 . (and backport HBASE-15728 for branch-1)
[ https://issues.apache.org/jira/browse/HBASE-21140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605556#comment-16605556 ] Duo Zhang commented on HBASE-21140: --- Maybe just use IdReadWriteLock.writeLock? > Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to > branch-1 . (and backport HBASE-15728 for branch-1) > --- > > Key: HBASE-21140 > URL: https://issues.apache.org/jira/browse/HBASE-21140 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Duo Zhang >Assignee: Xu Cang >Priority: Major > Fix For: 1.5.0 > > Attachments: > HBASE-21140.diff_against_cf198a65e8d704d28538c4c165a941b9e5bac678.branch-1.001.patch > > > There is no computeIfAbsent method on branch-1 as we still need to support > JDK7, so the fix will be different with branch-2+. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21161) Enable the test added in HBASE-20741 that was removed accidentally
[ https://issues.apache.org/jira/browse/HBASE-21161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-21161: --- Attachment: HBASE-21161.patch > Enable the test added in HBASE-20741 that was removed accidentally > -- > > Key: HBASE-21161 > URL: https://issues.apache.org/jira/browse/HBASE-21161 > Project: HBase > Issue Type: Test >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Attachments: HBASE-21161.patch > > > While giving an addendum to remove the timout in the test I mistakenly > removed the @Test tag and so the test was not running after the first initial > runs. During the first commit the @Test tag was there. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20741) Split of a region with replicas creates all daughter regions and its replica in same server
[ https://issues.apache.org/jira/browse/HBASE-20741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20741: --- Attachment: HBASE-20741_branch-2.1.patch > Split of a region with replicas creates all daughter regions and its replica > in same server > --- > > Key: HBASE-20741 > URL: https://issues.apache.org/jira/browse/HBASE-20741 > Project: HBase > Issue Type: Bug > Components: read replicas >Affects Versions: 3.0.0, 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20741.patch, HBASE-20741_1.patch, > HBASE-20741_2.patch, HBASE-20741_2.patch, HBASE-20741_3.patch, > HBASE-20741_4.patch, HBASE-20741_5.patch, HBASE-20741_5.patch, > HBASE-20741_addendum.patch, HBASE-20741_branch-2.1.patch, > HBASE-20741_branch-2.1.patch, HBASE-20741_branch-2.patch > > > Generally it is better that the parent region when split creates the daughter > region in the same target server. > But for replicas also we do the same and all the replica regions are created > in the same target server. We should ideally be doing a round robin and only > the primary daughter region should be opened in the intended target server > (where the parent was previously opened). > [~huaxiang] FYI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
[
https://issues.apache.org/jira/browse/HBASE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605434#comment-16605434
]
Hadoop QA commented on HBASE-21155:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
26s{color} | {color:red} hbase-server: The patch generated 2 new + 193
unchanged - 0 fixed = 195 total (was 193) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
4s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
30s{color} | {color:red} hbase-server generated 2 new + 0 unchanged - 0 fixed =
2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}126m 40s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}168m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Null passed for non-null parameter of
ServerManager.findDeadServersAndProcess(Set, Set) in
org.apache.hadoop.hbase.master.RegionServerTracker.start(Set, Set) Method
invoked at RegionServerTracker.java:of
ServerManager.findDeadServersAndProcess(Set, Set) in
org.apache.hadoop.hbase.master.RegionServerTracker.start(Set, Set) Method
invoked at RegionServerTracker.java:[line 138] |
| | Null passed for non-null parameter of
ServerManager.findDeadServersAndProcess(Set, Set) in
org.apache.hadoop.hbase.master.RegionServerTracker.start(Set, Set) Method
invoked at RegionServerTracker.java:of
ServerManager.findDeadServersAndProcess(Set, Set) in
org.apache.hadoop.hbase.master.RegionServerTracker.start(Set, Set) Method
invoked at RegionServerTracker.java:[line 138] |
| Failed junit tests |
hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Updated] (HBASE-20741) Split of a region with replicas creates all daughter regions and its replica in same server
[ https://issues.apache.org/jira/browse/HBASE-20741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20741: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.1.1 Status: Resolved (was: Patch Available) Pushed to branch-2.1 also. Hence resolving. > Split of a region with replicas creates all daughter regions and its replica > in same server > --- > > Key: HBASE-20741 > URL: https://issues.apache.org/jira/browse/HBASE-20741 > Project: HBase > Issue Type: Bug > Components: read replicas >Affects Versions: 3.0.0, 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.1 > > Attachments: HBASE-20741.patch, HBASE-20741_1.patch, > HBASE-20741_2.patch, HBASE-20741_2.patch, HBASE-20741_3.patch, > HBASE-20741_4.patch, HBASE-20741_5.patch, HBASE-20741_5.patch, > HBASE-20741_addendum.patch, HBASE-20741_branch-2.1.patch, > HBASE-20741_branch-2.1.patch, HBASE-20741_branch-2.patch > > > Generally it is better that the parent region when split creates the daughter > region in the same target server. > But for replicas also we do the same and all the replica regions are created > in the same target server. We should ideally be doing a round robin and only > the primary daughter region should be opened in the intended target server > (where the parent was previously opened). > [~huaxiang] FYI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21161) Enable the test added in HBASE-20741 that was removed accidentally
[ https://issues.apache.org/jira/browse/HBASE-21161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-21161. Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 3.0.0 Pushed it so that the subsequent builds can take the new test into account. > Enable the test added in HBASE-20741 that was removed accidentally > -- > > Key: HBASE-21161 > URL: https://issues.apache.org/jira/browse/HBASE-21161 > Project: HBase > Issue Type: Test >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 3.0.0, 2.2.0, 2.1.1 > > Attachments: HBASE-21161.patch > > > While giving an addendum to remove the timout in the test I mistakenly > removed the @Test tag and so the test was not running after the first initial > runs. During the first commit the @Test tag was there. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605647#comment-16605647
]
Hudson commented on HBASE-21088:
Results for branch branch-1.4
[build #450 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/450/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/450//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/450//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/450//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.8, 2.1.1, 2.0.2
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21157) Split TableInputFormatScan to individual tests
[
https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605671#comment-16605671
]
Hudson commented on HBASE-21157:
Results for branch branch-2
[build #1208 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1208/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1208//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1208//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1208//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Split TableInputFormatScan to individual tests
> --
>
> Key: HBASE-21157
> URL: https://issues.apache.org/jira/browse/HBASE-21157
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21157.patch
>
>
> We have done a split in HBASE-8326, which split the test to two parts. But it
> is still a bit slow, split it into several tests can increase the parallelism
> and make the 'mvn test' run faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[ https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605666#comment-16605666 ] ramkrishna.s.vasudevan commented on HBASE-21102: Ping [~huaxiang]. > ServerCrashProcedure should select target server where no other replicas > exist for the current region > - > > Key: HBASE-21102 > URL: https://issues.apache.org/jira/browse/HBASE-21102 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Attachments: HBASE-21102_1.patch, HBASE-21102_2.patch, > HBASE-21102_3.patch, HBASE-21102_initial.patch > > > Currently when a server with region replica crashes, when the target server > is created for the replica region assignment there is no guarentee that a > server is selected where there is no other replica for the current region > getting assigned. It so happens that currently we do an assignment randomly > and later the LB comes and identifies these cases and again does MOVE for > such regions. It will be better if we can identify target servers at least > minimally ensuring that replicas are not colocated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21157) Split TableInputFormatScan to individual tests
[
https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605665#comment-16605665
]
Hudson commented on HBASE-21157:
Results for branch branch-2.1
[build #285 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/285/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/285//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/285//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/285//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Split TableInputFormatScan to individual tests
> --
>
> Key: HBASE-21157
> URL: https://issues.apache.org/jira/browse/HBASE-21157
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21157.patch
>
>
> We have done a split in HBASE-8326, which split the test to two parts. But it
> is still a bit slow, split it into several tests can increase the parallelism
> and make the 'mvn test' run faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605482#comment-16605482
]
Hadoop QA commented on HBASE-21138:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
26s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 51s{color}
| {color:red} hbase-server generated 1 new + 187 unchanged - 1 fixed = 188
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
13s{color} | {color:red} hbase-server: The patch generated 1 new + 28 unchanged
- 23 fixed = 29 total (was 51) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
26s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 51s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}204m
12s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}251m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21138 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938581/HBASE-21138.004.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c07c2b37277a 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5a672b9da9 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/14327/artifact/patchprocess/diff-compile-javac-hbase-server.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14327/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
[jira] [Commented] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605612#comment-16605612
]
Hudson commented on HBASE-21088:
Results for branch branch-1.3
[build #455 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/455/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/455//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/455//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/455//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.8, 2.1.1, 2.0.2
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605613#comment-16605613
]
Hudson commented on HBASE-21061:
Results for branch branch-1.3
[build #455 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/455/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/455//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/455//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/455//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.3.3, 1.2.7, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 1.5.0, 1.3.3, 1.4.8, 1.2.7
>
> Attachments: HBASE-21061-branch-1.v0.patch
>
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605627#comment-16605627
]
Ted Yu commented on HBASE-21098:
Please fix test failure.
{code}
Caused by: java.lang.IllegalArgumentException: Wrong FS:
file:/testptch/hbase/hbase-mapreduce/faefb7c3-18e2-4af7-a177-6741759d73fd/.tmpdir/emptySnaptb0-testExportWithTargetName,
expected: hdfs://localhost:55982
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:647)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:194)
at
org.apache.hadoop.hdfs.DistributedFileSystem.rename(DistributedFileSystem.java:626)
at
org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.completeSnapshot(TakeSnapshotHandler.java:269)
at
org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.process(TakeSnapshotHandler.java:218)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
{code}
> Improve Snapshot Performance with Temporary Snapshot Directory when rootDir
> on S3
> -
>
> Key: HBASE-21098
> URL: https://issues.apache.org/jira/browse/HBASE-21098
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.4.8, 2.1.1
>Reporter: Tyler Mi
>Priority: Major
> Attachments: HBASE-21098.master.001.patch,
> HBASE-21098.master.002.patch, HBASE-21098.master.003.patch,
> HBASE-21098.master.004.patch, HBASE-21098.master.005.patch,
> HBASE-21098.master.006.patch, HBASE-21098.master.007.patch,
> HBASE-21098.master.008.patch, HBASE-21098.master.009.patch,
> HBASE-21098.master.010.patch, HBASE-21098.master.011.patch
>
>
> When using Apache HBase, the snapshot feature can be used to make a point in
> time recovery. To do this, HBase creates a manifest of all the files in all
> of the Regions so that those files can be referenced again when a user
> restores a snapshot. With HBase's S3 storage mode, developers can store their
> data off-cluster on Amazon S3. However, utilizing S3 as a file system is
> inefficient in some operations, namely renames. Most Hadoop ecosystem
> applications use an atomic rename as a method of committing data. However,
> with S3, a rename is a separate copy and then a delete of every file which is
> no longer atomic and, in fact, quite costly. In addition, puts and deletes on
> S3 have latency issues that traditional filesystems do not encounter when
> manipulating the region snapshots to consolidate into a single manifest. When
> HBase on S3 users have a significant amount of regions, puts, deletes, and
> renames (the final commit stage of the snapshot) become the bottleneck
> causing snapshots to take many minutes or even hours to complete.
> The purpose of this patch is to increase the overall performance of snapshots
> while utilizing HBase on S3 through the use of a temporary directory for the
> snapshots that exists on a traditional filesystem like HDFS to circumvent the
> bottlenecks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21159) Add shell command to switch throttle on or off
Yi Mei created HBASE-21159: -- Summary: Add shell command to switch throttle on or off Key: HBASE-21159 URL: https://issues.apache.org/jira/browse/HBASE-21159 Project: HBase Issue Type: Sub-task Affects Versions: 3.0.0, 2.2.0 Reporter: Yi Mei Add shell command to switch throttle on or off. When throttle is off, HBase will not throttle any request. This feature may be useful in production environment. We can use the following commands to switch throttle: throttle_switch true / throttle_switch false -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605533#comment-16605533
]
Hudson commented on HBASE-21088:
Results for branch branch-1
[build #447 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/447/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/447//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/447//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/447//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.8, 2.1.1, 2.0.2
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605534#comment-16605534
]
Hudson commented on HBASE-21061:
Results for branch branch-1
[build #447 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/447/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/447//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/447//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/447//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.3.3, 1.2.7, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 1.5.0, 1.3.3, 1.4.8, 1.2.7
>
> Attachments: HBASE-21061-branch-1.v0.patch
>
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21159) Add shell command to switch throttle on or off
[ https://issues.apache.org/jira/browse/HBASE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605560#comment-16605560 ] Xu Cang commented on HBASE-21159: - (non-binding) nitpick comment: most HBase shell commands are "*verb*_*noun*" So, how about changing command to "enable_throttle" / "disable_throttle"? > Add shell command to switch throttle on or off > -- > > Key: HBASE-21159 > URL: https://issues.apache.org/jira/browse/HBASE-21159 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0 >Reporter: Yi Mei >Priority: Major > > Add shell command to switch throttle on or off. When throttle is off, HBase > will not throttle any request. This feature may be useful in production > environment. > We can use the following commands to switch throttle: > throttle_switch true / throttle_switch false -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20917) MetaTableMetrics#stop references uninitialized requestsMap for non-meta region
[
https://issues.apache.org/jira/browse/HBASE-20917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20917:
---
Fix Version/s: (was: 2.0.2)
> MetaTableMetrics#stop references uninitialized requestsMap for non-meta region
> --
>
> Key: HBASE-20917
> URL: https://issues.apache.org/jira/browse/HBASE-20917
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.6, 2.2.0
>
> Attachments: 20917.addendum, 20917.v1.txt, 20917.v2.txt
>
>
> I noticed the following in test output:
> {code}
> 2018-07-21 15:54:43,181 ERROR [RS_CLOSE_REGION-regionserver/172.17.5.4:0-1]
> executor.EventHandler(186): Caught throwable while processing event
> M_RS_CLOSE_REGION
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics.stop(MetaTableMetrics.java:329)
> at
> org.apache.hadoop.hbase.coprocessor.BaseEnvironment.shutdown(BaseEnvironment.java:91)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:165)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:290)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$4.postEnvCall(RegionCoprocessorHost.java:559)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:622)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:551)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1678)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1484)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:104)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> {code}
> {{requestsMap}} is only initialized for the meta region.
> However, check for meta region is absent in the stop method:
> {code}
> public void stop(CoprocessorEnvironment e) throws IOException {
> // since meta region can move around, clear stale metrics when stop.
> for (String meterName : requestsMap.keySet()) {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605648#comment-16605648
]
Hudson commented on HBASE-21061:
Results for branch branch-1.4
[build #450 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/450/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/450//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/450//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/450//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.3.3, 1.2.7, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 1.5.0, 1.3.3, 1.4.8, 1.2.7
>
> Attachments: HBASE-21061-branch-1.v0.patch
>
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21140) Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to branch-1 . (and backport HBASE-15728 for branch-1)
[ https://issues.apache.org/jira/browse/HBASE-21140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605549#comment-16605549 ] Xu Cang commented on HBASE-21140: - "Striped" is available from guava since 13.0, but HBase branch-1 is using guava 12.0 now. (https://github.com/apache/hbase/blob/master/pom.xml#L1466) What's the guideline for upgrading library such as guava? Thanks! [~Apache9] If upgrading guava is not advised, I will extend IdLock to support String as key type. > Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to > branch-1 . (and backport HBASE-15728 for branch-1) > --- > > Key: HBASE-21140 > URL: https://issues.apache.org/jira/browse/HBASE-21140 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Duo Zhang >Assignee: Xu Cang >Priority: Major > Fix For: 1.5.0 > > Attachments: > HBASE-21140.diff_against_cf198a65e8d704d28538c4c165a941b9e5bac678.branch-1.001.patch > > > There is no computeIfAbsent method on branch-1 as we still need to support > JDK7, so the fix will be different with branch-2+. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21160) Assertion in TestVisibilityLabelsWithDeletes#testDeleteColumnsWithoutAndWithVisibilityLabels is ignored
Ted Yu created HBASE-21160:
--
Summary: Assertion in
TestVisibilityLabelsWithDeletes#testDeleteColumnsWithoutAndWithVisibilityLabels
is ignored
Key: HBASE-21160
URL: https://issues.apache.org/jira/browse/HBASE-21160
Project: HBase
Issue Type: Test
Reporter: Ted Yu
>From
>https://builds.apache.org/job/PreCommit-HBASE-Build/14327/artifact/patchprocess/diff-compile-javac-hbase-server.txt
> (HBASE-21138 QA run):
{code}
[WARNING]
/testptch/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDeletes.java:[315,25]
[AssertionFailureIgnored] This assertion throws an AssertionError if it fails,
which will be caught by an enclosing try block.
{code}
Here is related code:
{code}
PrivilegedExceptionAction scanAction = new
PrivilegedExceptionAction() {
@Override
public Void run() throws Exception {
try (Connection connection = ConnectionFactory.createConnection(conf);
...
assertEquals(1, next.length);
} catch (Throwable t) {
throw new IOException(t);
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21161) Enable the test added in HBASE-20741 that was removed accidentally
ramkrishna.s.vasudevan created HBASE-21161: -- Summary: Enable the test added in HBASE-20741 that was removed accidentally Key: HBASE-21161 URL: https://issues.apache.org/jira/browse/HBASE-21161 Project: HBase Issue Type: Test Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan While giving an addendum to remove the timout in the test I mistakenly removed the @Test tag and so the test was not running after the first initial runs. During the first commit the @Test tag was there. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605767#comment-16605767
]
Guangxu Cheng commented on HBASE-21001:
---
OK, let's add a unit test in {{TestHRegion}} to check wether the region has
successfully loaded ReplicationObserver when bulk loading replication feature
is enable.
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21107) add a metrics for netty direct memory
[
https://issues.apache.org/jira/browse/HBASE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605786#comment-16605786
]
Hudson commented on HBASE-21107:
Results for branch master
[build #476 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/476/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> add a metrics for netty direct memory
> -
>
> Key: HBASE-21107
> URL: https://issues.apache.org/jira/browse/HBASE-21107
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
>Affects Versions: 2.0.1
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21107-master-v1.patch,
> HBASE-21107-master-v1.patch, HBASE-21107-master-v2.patch,
> HBASE-21107-master-v2.patch, HBASE-21107-master-v2.patch,
> HBASE-21107-master-v3.patch, Screen Shot 2018-08-23 at 4.55.01 PM.png
>
>
> This is separated from HBASE-20913 as I realized that netty direct memory
> usage needs to show up in metrics instead of the web page.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605785#comment-16605785
]
Hudson commented on HBASE-21153:
Results for branch master
[build #476 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/476/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21153.0.patch, ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-21129. Resolution: Fixed > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: 21129.addendum, HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch, HBASE-21129.master.003.patch, > HBASE-21129.master.004.patch, HBASE-21129.master.005.patch, > HBASE-21129.master.006.patch, HBASE-21129.master.007.patch, > HBASE-21129.master.008.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21148) [Docs] Some errors in section#Security Configuration Example of hbase book
[
https://issues.apache.org/jira/browse/HBASE-21148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605787#comment-16605787
]
Hudson commented on HBASE-21148:
Results for branch master
[build #476 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/476/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Docs] Some errors in section#Security Configuration Example of hbase book
> --
>
> Key: HBASE-21148
> URL: https://issues.apache.org/jira/browse/HBASE-21148
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Labels: beginner, document
> Fix For: 3.0.0
>
> Attachments: HBASE-21148.master.001.patch
>
>
> * There is a wrong package name about VisibilityController in mentioned
> section: it should be
> org.apache.hadoop.hbase.security.*visibility*.VisibilityController instead of
> org.apache.hadoop.hbase.security.*access*.VisibilityController.
> * A wrong slash: org.apache.hadoop/hbase.security.access.AccessController
> * VisibilityController should not be in
> *hbase.coprocessor.regionserver.classes* which will fail rs starting up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21157) Split TableInputFormatScan to individual tests
[
https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605788#comment-16605788
]
Hudson commented on HBASE-21157:
Results for branch master
[build #476 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/476/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Split TableInputFormatScan to individual tests
> --
>
> Key: HBASE-21157
> URL: https://issues.apache.org/jira/browse/HBASE-21157
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21157.patch
>
>
> We have done a split in HBASE-8326, which split the test to two parts. But it
> is still a bit slow, split it into several tests can increase the parallelism
> and make the 'mvn test' run faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18974) Document "Becoming a Committer"
[
https://issues.apache.org/jira/browse/HBASE-18974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605784#comment-16605784
]
Hudson commented on HBASE-18974:
Results for branch master
[build #476 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/476/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/476//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Document "Becoming a Committer"
> ---
>
> Key: HBASE-18974
> URL: https://issues.apache.org/jira/browse/HBASE-18974
> Project: HBase
> Issue Type: Improvement
> Components: community, documentation
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-18974-copyedit-addendum.patch, HBASE-18974.patch,
> HBASE-18974.v2.patch, HBASE-18974.v3.patch, HBASE-18974.v4.patch
>
>
> Based on the mailing list discussion at
> https://lists.apache.org/thread.html/81c633cbe1f6f78421cbdad5b9549643c67803a723a9d86a513264c0@%3Cdev.hbase.apache.org%3E
> it sounds like we should record some of the thoughts for future contributors
> to refer to.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-21129: - reopening because this patch caused a failure in nightly's scaladoc check. Here's the report for master branch that nightly tried to link to: https://builds.apache.org/job/HBase%20Nightly/job/master/472/General_20Nightly_20Build_20Report/ and here's the specific error log for scaladoc: https://builds.apache.org/job/HBase%20Nightly/job/master/476/artifact/output-general/patch-scaladoc-root.txt please push an addendum to relevant branches or revert. > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch, HBASE-21129.master.003.patch, > HBASE-21129.master.004.patch, HBASE-21129.master.005.patch, > HBASE-21129.master.006.patch, HBASE-21129.master.007.patch, > HBASE-21129.master.008.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21001:
--
Attachment: HBASE-21001.master.002.patch
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21129: --- Attachment: 21129.addendum > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: 21129.addendum, HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch, HBASE-21129.master.003.patch, > HBASE-21129.master.004.patch, HBASE-21129.master.005.patch, > HBASE-21129.master.006.patch, HBASE-21129.master.007.patch, > HBASE-21129.master.008.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605907#comment-16605907
]
Ted Yu commented on HBASE-21001:
Looks good overall.
{code}
+ if (!plugins.contains(replicationCoprocessorClass)) {
+conf.set(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY,
+plugins + "," + replicationCoprocessorClass);
{code}
Since plugins may be empty, please check whether comma is needed.
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21013) Backport "read part" of HBASE-18754 to all active 1.x branches
[ https://issues.apache.org/jira/browse/HBASE-21013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21013: Fix Version/s: (was: 1.2.7) 1.2.8 > Backport "read part" of HBASE-18754 to all active 1.x branches > -- > > Key: HBASE-21013 > URL: https://issues.apache.org/jira/browse/HBASE-21013 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Mingdao Yang >Priority: Critical > Fix For: 1.5.0, 1.3.3, 1.2.8, 1.4.8 > > > The hfiles impacted by HBASE-18754 will have bytes of proto.TimeRangeTracker. > It makes all 1.x branches failed to read the hfile since all 1.x branches > can't deserialize the proto.TimeRangeTracker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20694) Consolidate warning on SecureBulkLoad directory permissions
[ https://issues.apache.org/jira/browse/HBASE-20694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20694: Fix Version/s: (was: 1.2.7) 1.2.8 > Consolidate warning on SecureBulkLoad directory permissions > --- > > Key: HBASE-20694 > URL: https://issues.apache.org/jira/browse/HBASE-20694 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Minor > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8 > > > Follow-on from HBASE-20605: > HBase 1.x has a check which ignores a directory permission check if you're > using a specific filesystem which we think doesnt' do security properly. > HBase 2.x dropped this check. > Since the security of bulk-loaded data is dependent upon this directory > permission (and thus the capabilities of the FileSystem), it would be better > to have a consistent warning across branches. > [~busbey] suggested that we make a WARN message which points admins to our > Book (and write such a section if we don't have something sufficient already) > and our supported filesystems, coupled with an option to disable that warning > message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605890#comment-16605890 ] Reid Chan commented on HBASE-21129: --- Thanks for sean's report and ted's addendum. > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: 21129.addendum, HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch, HBASE-21129.master.003.patch, > HBASE-21129.master.004.patch, HBASE-21129.master.005.patch, > HBASE-21129.master.006.patch, HBASE-21129.master.007.patch, > HBASE-21129.master.008.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21162) Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default
Andrew Purtell created HBASE-21162: -- Summary: Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default Key: HBASE-21162 URL: https://issues.apache.org/jira/browse/HBASE-21162 Project: HBase Issue Type: Bug Affects Versions: 1.4.7 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.5.0, 1.4.8 We had a production incident where we traced the issue to a direct buffer leak. On a hunch we tried setting hbase.ipc.server.reservoir.enabled = false and after that no native memory leak could be observed in any regionserver process under the triggering load. On HBASE-19239 (Fix findbugs and error-prone issues) I made a change to BoundedByteBufferPool that is suspicious given this finding. It was committed to branch-1.4 and branch-1. I'm going to revert this change. In addition the allocation of direct memory for the server RPC reservoir is a bit problematic in that tracing native memory or direct buffer leaks to a particular class or compilation unit is difficult, so I also propose allocating the reservoir on the heap by default instead. Should there be a leak it is much easier to do an analysis of a heap dump with familiar tools to find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605984#comment-16605984
]
Guangxu Cheng commented on HBASE-21001:
---
Attach 003 patch as [~yuzhih...@gmail.com] suggestion.Thanks
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch,
> HBASE-21001.master.003.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21001:
--
Attachment: HBASE-21001.master.003.patch
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch,
> HBASE-21001.master.003.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[
https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606132#comment-16606132
]
Reid Chan commented on HBASE-20734:
---
* {CommonFSUtils}#constructWALRegionDirFromRegionInfo
This name should get aligned with others, like getWALRegionDir for your
reference.
Please add java doc on new methods.
* {AbstractStateMachineTableProcedure.java}#getRegionDir is no longer used.
I think it is ok to remove it, since AbstractStateMachineTableProcedure is
IA.Private.
* {HRegion}
Please add java doc on new methods, and avoid unnecessary style changes like
below:
{code}
-+ " recovered edits file(s) under " + regiondir);
+ + " recovered edits file(s) under " + regionDir);
{code}
Please update the description, since may not be HDFS.
{code}
// Store SeqId in HDFS when a region closes
// checking region folder exists is due to many tests which delete the
table folder while a
// table is still online
-if (this.fs.getFileSystem().exists(this.fs.getRegionDir())) {
- WALSplitter.writeRegionSequenceIdFile(this.fs.getFileSystem(),
this.fs.getRegionDir(),
+if (getWalFileSystem().exists(getWALRegionDir())) {
{code}
Don't we add two new reference? i'm not sure about the ref count, what if
{regionDir} and {walFS} never get initialized which may not happen in real
world, but it is a logical problem.
{code}
+ 52 * ClassSize.REFERENCE
{code}
BTW, due to HBASE-21138, may need some efforts to fix conflicts.
> Colocate recovered edits directory with hbase.wal.dir
> -
>
> Key: HBASE-20734
> URL: https://issues.apache.org/jira/browse/HBASE-20734
> Project: HBase
> Issue Type: Improvement
> Components: MTTR, Recovery, wal
>Reporter: Ted Yu
>Assignee: Zach York
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20734.branch-1.001.patch,
> HBASE-20734.branch-1.002.patch, HBASE-20734.branch-1.003.patch,
> HBASE-20734.master.001.patch, HBASE-20734.master.002.patch,
> HBASE-20734.master.003.patch, HBASE-20734.master.004.patch,
> HBASE-20734.master.005.patch, HBASE-20734.master.006.patch,
> HBASE-20734.master.007.patch, HBASE-20734.master.008.patch,
> HBASE-20734.master.009.patch, HBASE-20734.master.010.patch
>
>
> During investigation of HBASE-20723, I realized that we wouldn't get the best
> performance when hbase.wal.dir is configured to be on different (fast) media
> than hbase rootdir w.r.t. recovered edits since recovered edits directory is
> currently under rootdir.
> Such setup may not result in fast recovery when there is region server
> failover.
> This issue is to find proper (hopefully backward compatible) way in
> colocating recovered edits directory with hbase.wal.dir .
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[ https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605972#comment-16605972 ] huaxiang sun commented on HBASE-21102: -- Sorry, [~ram_krish]. Somehow, I thought this review as the other read replica review(which was committed) and totally ignored this one. I am reviewing now, sorry for the delay! > ServerCrashProcedure should select target server where no other replicas > exist for the current region > - > > Key: HBASE-21102 > URL: https://issues.apache.org/jira/browse/HBASE-21102 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Attachments: HBASE-21102_1.patch, HBASE-21102_2.patch, > HBASE-21102_3.patch, HBASE-21102_initial.patch > > > Currently when a server with region replica crashes, when the target server > is created for the replica region assignment there is no guarentee that a > server is selected where there is no other replica for the current region > getting assigned. It so happens that currently we do an assignment randomly > and later the LB comes and identifies these cases and again does MOVE for > such regions. It will be better if we can identify target servers at least > minimally ensuring that replicas are not colocated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606030#comment-16606030
]
Ted Yu commented on HBASE-21001:
{code}
8686(plugins == "" ? "" : (plugins + ",")) +
replicationCoprocessorClass);
{code}
Please use {{equals}} for the check.
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch,
> HBASE-21001.master.003.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20741) Split of a region with replicas creates all daughter regions and its replica in same server
[
https://issues.apache.org/jira/browse/HBASE-20741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606037#comment-16606037
]
Hudson commented on HBASE-20741:
Results for branch branch-2.1
[build #287 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/287/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/287//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/287//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/287//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Split of a region with replicas creates all daughter regions and its replica
> in same server
> ---
>
> Key: HBASE-20741
> URL: https://issues.apache.org/jira/browse/HBASE-20741
> Project: HBase
> Issue Type: Bug
> Components: read replicas
>Affects Versions: 3.0.0, 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-20741.patch, HBASE-20741_1.patch,
> HBASE-20741_2.patch, HBASE-20741_2.patch, HBASE-20741_3.patch,
> HBASE-20741_4.patch, HBASE-20741_5.patch, HBASE-20741_5.patch,
> HBASE-20741_addendum.patch, HBASE-20741_branch-2.1.patch,
> HBASE-20741_branch-2.1.patch, HBASE-20741_branch-2.patch
>
>
> Generally it is better that the parent region when split creates the daughter
> region in the same target server.
> But for replicas also we do the same and all the replica regions are created
> in the same target server. We should ideally be doing a round robin and only
> the primary daughter region should be opened in the intended target server
> (where the parent was previously opened).
> [~huaxiang] FYI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21001:
--
Attachment: HBASE-21001.master.004.patch
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch,
> HBASE-21001.master.003.patch, HBASE-21001.master.004.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21162) Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default
[ https://issues.apache.org/jira/browse/HBASE-21162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21162: --- Attachment: HBASE-21162-branch-1.patch > Revert suspicious change to BoundedByteBufferPool and disable use of direct > buffers for IPC reservoir by default > > > Key: HBASE-21162 > URL: https://issues.apache.org/jira/browse/HBASE-21162 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.7 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Critical > Fix For: 1.5.0, 1.4.8 > > Attachments: HBASE-21162-branch-1.patch > > > We had a production incident where we traced the issue to a direct buffer > leak. On a hunch we tried setting hbase.ipc.server.reservoir.enabled = false > and after that no native memory leak could be observed in any regionserver > process under the triggering load. > On HBASE-19239 (Fix findbugs and error-prone issues) I made a change to > BoundedByteBufferPool that is suspicious given this finding. It was committed > to branch-1.4 and branch-1. I'm going to revert this change. > In addition the allocation of direct memory for the server RPC reservoir is a > bit problematic in that tracing native memory or direct buffer leaks to a > particular class or compilation unit is difficult, so I also propose > allocating the reservoir on the heap by default instead. Should there be a > leak it is much easier to do an analysis of a heap dump with familiar tools > to find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21162) Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default
[ https://issues.apache.org/jira/browse/HBASE-21162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606069#comment-16606069 ] Andrew Purtell commented on HBASE-21162: BoundedByteBufferPool was removed in branch-2 and up so this is only a branch-1 issue. It was replaced with ByteBufferPool, a reimplementation. > Revert suspicious change to BoundedByteBufferPool and disable use of direct > buffers for IPC reservoir by default > > > Key: HBASE-21162 > URL: https://issues.apache.org/jira/browse/HBASE-21162 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.7 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Critical > Fix For: 1.5.0, 1.4.8 > > Attachments: HBASE-21162-branch-1.patch > > > We had a production incident where we traced the issue to a direct buffer > leak. On a hunch we tried setting hbase.ipc.server.reservoir.enabled = false > and after that no native memory leak could be observed in any regionserver > process under the triggering load. > On HBASE-19239 (Fix findbugs and error-prone issues) I made a change to > BoundedByteBufferPool that is suspicious given this finding. It was committed > to branch-1.4 and branch-1. I'm going to revert this change. > In addition the allocation of direct memory for the server RPC reservoir is a > bit problematic in that tracing native memory or direct buffer leaks to a > particular class or compilation unit is difficult, so I also propose > allocating the reservoir on the heap by default instead. Should there be a > leak it is much easier to do an analysis of a heap dump with familiar tools > to find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21158) Empty qualifier cell is always returned when using QualifierFilter
[
https://issues.apache.org/jira/browse/HBASE-21158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21158:
--
Attachment: HBASE-21158.master.001.patch
> Empty qualifier cell is always returned when using QualifierFilter
> --
>
> Key: HBASE-21158
> URL: https://issues.apache.org/jira/browse/HBASE-21158
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21158.master.001.patch
>
>
> {code:xml}
> hbase(main):002:0> put 'testTable','testrow','f:testcol1','testvalue1'
> 0 row(s) in 0.0040 seconds
> hbase(main):003:0> put 'testTable','testrow','f:','testvalue2'
> 0 row(s) in 0.0070 seconds
> # get row with empty column f:, result is correct.
> hbase(main):004:0> scan 'testTable',{FILTER => "QualifierFilter (=,
> 'binary:')"}
> ROW COLUMN+CELL
>
>
> testrowcolumn=f:,
> timestamp=1536218563581, value=testvalue2
>
> 1 row(s) in 0.0460 seconds
> # get row with column f:testcol1, result is incorrect.
> hbase(main):005:0> scan 'testTable',{FILTER => "QualifierFilter (=,
> 'binary:testcol1')"}
> ROW COLUMN+CELL
>
>
> testrowcolumn=f:,
> timestamp=1536218563581, value=testvalue2
>
> testrowcolumn=f:testcol1,
> timestamp=1536218550827, value=testvalue1
>
> 1 row(s) in 0.0070 seconds
> {code}
> As the above operation, when the row contains empty qualifier column, empty
> qualifier cell is always returned when using QualifierFilter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[ https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606098#comment-16606098 ] ramkrishna.s.vasudevan commented on HBASE-21102: No problem. Appreciate your time. Here is the RB link - https://reviews.apache.org/r/68605/. > ServerCrashProcedure should select target server where no other replicas > exist for the current region > - > > Key: HBASE-21102 > URL: https://issues.apache.org/jira/browse/HBASE-21102 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Attachments: HBASE-21102_1.patch, HBASE-21102_2.patch, > HBASE-21102_3.patch, HBASE-21102_initial.patch > > > Currently when a server with region replica crashes, when the target server > is created for the replica region assignment there is no guarentee that a > server is selected where there is no other replica for the current region > getting assigned. It so happens that currently we do an assignment randomly > and later the LB comes and identifies these cases and again does MOVE for > such regions. It will be better if we can identify target servers at least > minimally ensuring that replicas are not colocated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606097#comment-16606097
]
Ted Yu commented on HBASE-21138:
Patch doesn't apply to branch-2:
{code}
error: patch failed:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java:341
error:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java:
patch does not apply
{code}
FYI
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch, HBASE-21138.003.patch, HBASE-21138.004.patch,
> HBASE-21138.branch-1.004.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21001:
--
Attachment: (was: HBASE-21001.master.003.patch)
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch, HBASE-21001.master.002.patch,
> HBASE-21001.master.003.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21143) Update findbugs-maven-plugin to 3.0.4
[
https://issues.apache.org/jira/browse/HBASE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606006#comment-16606006
]
Guangxu Cheng commented on HBASE-21143:
---
So if no objections I will commit the patch to branch-2.0+ later.Thanks
> Update findbugs-maven-plugin to 3.0.4
> -
>
> Key: HBASE-21143
> URL: https://issues.apache.org/jira/browse/HBASE-21143
> Project: HBase
> Issue Type: Bug
> Components: pom
>Affects Versions: 3.0.0, 2.1.0, 2.2.0, 2.0.2
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21143.master.001.patch
>
>
> {code}
> Failed to execute goal org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs
> (default) on project hbase: Execution default of goal
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs failed: Plugin
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0 or one of its dependencies
> could not be resolved: Failed to collect dependencies at
> org.codehaus.mojo:findbugs-maven-plugin:jar:3.0.0 ->
> org.codehaus.groovy:groovy-all:jar:1.7.4: Failed to read artifact descriptor
> for org.codehaus.groovy:groovy-all:jar:1.7.4: Could not transfer artifact
> org.codehaus.groovy:groovy-all:pom:1.7.4 from/to mirror
> (http://xxx..xxx/nexus/content/groups/public): Failed to transfer file:
> http://xxx..xxx/nexus/content/groups/public/org/codehaus/groovy/groovy-all/1.7.4/groovy-all-1.7.4.pom.
> Return code is: 418 , ReasonPhrase:Artifact is in Tencent Blacklist! Please
> update to the safe version, more information:
> http://xxx..xxx/?tab=blackList.
> {code}
> Recently, when I compile HBase with a new machine, I got the above error.
> Since the machine could not connect to the external network, we visited our
> internal Maven repository, but org.codehaus.groovy:groovy-all:jar:1.7.4 was
> added to the blacklist and could not be downloaded. See details,
> org.codehaus.groovy:groovy-all:jar:1.7.4 is marked as vulnerable by
> [CVE-2015-3253|https://www.cvedetails.com/cve/CVE-2015-3253], so we should
> upgrade the version.
> {code:xml}
> org.codehaus.mojo
> findbugs-maven-plugin
> 3.0.0
>
>
>
> ${project.basedir}/../dev-support/findbugs-exclude.xml
> {code}
> Look at the history commit record, findbugs-maven-plugin has been upgraded to
> 3.0.4 in HBASE-18264, but one place is missing which still using the version
> of 3.0.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
