date:20171128

[jira] [Updated] (HBASE-19340) SPLIT_POLICY and FLUSH_POLICY cann't be set directly by hbase shell

2017-11-28 Thread zhaoyuan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhaoyuan updated HBASE-19340:
-
Attachment: HBASE-19340.branch-1.v0.batch

> SPLIT_POLICY and FLUSH_POLICY cann't be set directly by hbase shell
> ---
>
> Key: HBASE-19340
> URL: https://issues.apache.org/jira/browse/HBASE-19340
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.6
>Reporter: zhaoyuan
>Assignee: zhaoyuan
> Fix For: 1.2.8
>
> Attachments: HBASE-19340.branch-1.v0.batch
>
>
> Recently I wanna try to alter the split policy for a table on my cluster 
> which version is 1.2.6 and as far as I know The SPLIT_POLICY is an attribute 
> of the HTable so I run the command in hbase shell console below. 
> alter 'tablex',SPLIT_POLICY => 
> 'org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy'
> However, It gave the information like this and I confused 
> Unknown argument ignored: SPLIT_POLICY
> Updating all regions with the new schema...
> So I check the source code That admin.rb might miss the setting for this 
> argument .
> htd.setMaxFileSize(JLong.valueOf(arg.delete(MAX_FILESIZE))) if 
> arg[MAX_FILESIZE]
> htd.setReadOnly(JBoolean.valueOf(arg.delete(READONLY))) if arg[READONLY]
> ...
> So I think it may be a bug  ,is it?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19371) Running WALPerformanceEvaluation against asyncfswal throws exceptions

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270328#comment-16270328
 ] 

Duo Zhang commented on HBASE-19371:
---

Tried reverting HBASE-17049 but did not work. Let me dig more.

> Running WALPerformanceEvaluation against asyncfswal throws exceptions
> -
>
> Key: HBASE-19371
> URL: https://issues.apache.org/jira/browse/HBASE-19371
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Fix For: 2.0.0-beta-1
>
>
> Was trying to do a perf eval on asyncfswal. I ran w/ these args:
>  Performance counter stats for '/home/stack/hbase/bin/hbase --config 
> /home/stack/conf_hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation 
> -path /user/stack/logs/ -verify -threads 25 -iterations 100 -keySize 50 
> -valueSize 100 -syncInterval 10':
> The verify fails on all runs:
> Exception in thread "main" java.lang.IllegalStateException: Counted=12390228, 
> expected=2500
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.run(WALPerformanceEvaluation.java:368)
>   
>   at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.innerMain(WALPerformanceEvaluation.java:597)
>   
> at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.main(WALPerformanceEvaluation.java:601)
> I need to fix test or figure what is wrong in asyncfswal.
> Also seeing these when I run w/ one thread only:
> 2017-11-28 21:25:49,952 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> Log has a spew of them.
> Has stuff like this too:
> 2017-11-28 21:25:40,065 WARN  [Close-WAL-Writer-3] 
> wal.AsyncProtobufLogWriter: normal close failed, try recover  
>  
> java.io.IOException: stream already broken
>   
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:510)
>   
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.lambda$close$12(FanOutOneBlockAsyncDFSOutput.java:550)
>   
>at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> 
> at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
>   at java.lang.Thread.run(Thread.java:748)
> Starts out spewing EMPTY here:
> 2017-11-28 21:16:52,051 INFO  [main] regionserver.HRegion: Setting 
> FlushNonSloppyStoresFirstPolicy for the 
> region=WALPerformanceEvaluation:0,,1511932610787.deca03e0ca447fa25d02fe9cd6e31aa4.
> 2017-11-28 21:16:52,058 DEBUG [main] regionserver.HRegion: Found 0 recovered 
> edits file(s) under 
> hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4
> 2017-11-28 21:16:52,068 DEBUG [main] regionserver.FlushLargeStoresPolicy: No 
> hbase.hregion.percolumnfamilyflush.size.lower.bound set in description of 
> table WALPerformanceEvaluation:0, use config (134217728) instead
> 2017-11-28 21:16:52,084 DEBUG [main] wal.WALSplitter: Wrote 
> file=hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4/recovered.edits/2.seqid,
>  newSeqId=2, maxSeqId=0
> 2017-11-28 21:16:52,084 INFO  [main] regionserver.HRegion: Onlined 
> deca03e0ca447fa25d02fe9cd6e31aa4; next sequenceid=2
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185 WARN

[jira] [Commented] (HBASE-19371) Running WALPerformanceEvaluation against asyncfswal throws exceptions

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270321#comment-16270321
 ] 

Duo Zhang commented on HBASE-19371:
---

OK, I got the same problem

{quote}
2017-11-29 15:29:40,910 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: RingBufferTruck 
with unexpected type: EMPTY
{quote}

Let me check.

> Running WALPerformanceEvaluation against asyncfswal throws exceptions
> -
>
> Key: HBASE-19371
> URL: https://issues.apache.org/jira/browse/HBASE-19371
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Fix For: 2.0.0-beta-1
>
>
> Was trying to do a perf eval on asyncfswal. I ran w/ these args:
>  Performance counter stats for '/home/stack/hbase/bin/hbase --config 
> /home/stack/conf_hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation 
> -path /user/stack/logs/ -verify -threads 25 -iterations 100 -keySize 50 
> -valueSize 100 -syncInterval 10':
> The verify fails on all runs:
> Exception in thread "main" java.lang.IllegalStateException: Counted=12390228, 
> expected=2500
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.run(WALPerformanceEvaluation.java:368)
>   
>   at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.innerMain(WALPerformanceEvaluation.java:597)
>   
> at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.main(WALPerformanceEvaluation.java:601)
> I need to fix test or figure what is wrong in asyncfswal.
> Also seeing these when I run w/ one thread only:
> 2017-11-28 21:25:49,952 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> Log has a spew of them.
> Has stuff like this too:
> 2017-11-28 21:25:40,065 WARN  [Close-WAL-Writer-3] 
> wal.AsyncProtobufLogWriter: normal close failed, try recover  
>  
> java.io.IOException: stream already broken
>   
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:510)
>   
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.lambda$close$12(FanOutOneBlockAsyncDFSOutput.java:550)
>   
>at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> 
> at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
>   at java.lang.Thread.run(Thread.java:748)
> Starts out spewing EMPTY here:
> 2017-11-28 21:16:52,051 INFO  [main] regionserver.HRegion: Setting 
> FlushNonSloppyStoresFirstPolicy for the 
> region=WALPerformanceEvaluation:0,,1511932610787.deca03e0ca447fa25d02fe9cd6e31aa4.
> 2017-11-28 21:16:52,058 DEBUG [main] regionserver.HRegion: Found 0 recovered 
> edits file(s) under 
> hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4
> 2017-11-28 21:16:52,068 DEBUG [main] regionserver.FlushLargeStoresPolicy: No 
> hbase.hregion.percolumnfamilyflush.size.lower.bound set in description of 
> table WALPerformanceEvaluation:0, use config (134217728) instead
> 2017-11-28 21:16:52,084 DEBUG [main] wal.WALSplitter: Wrote 
> file=hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4/recovered.edits/2.seqid,
>  newSeqId=2, maxSeqId=0
> 2017-11-28 21:16:52,084 INFO  [main] regionserver.HRegion: Onlined 
> deca03e0ca447fa25d02fe9cd6e31aa4; next sequenceid=2
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185 WARN

[jira] [Commented] (HBASE-19371) Running WALPerformanceEvaluation against asyncfswal throws exceptions

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270319#comment-16270319
 ] 

Duo Zhang commented on HBASE-19371:
---

Oh shit I forget to change the config to AsyncFSWAL... Let me try again.

> Running WALPerformanceEvaluation against asyncfswal throws exceptions
> -
>
> Key: HBASE-19371
> URL: https://issues.apache.org/jira/browse/HBASE-19371
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Fix For: 2.0.0-beta-1
>
>
> Was trying to do a perf eval on asyncfswal. I ran w/ these args:
>  Performance counter stats for '/home/stack/hbase/bin/hbase --config 
> /home/stack/conf_hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation 
> -path /user/stack/logs/ -verify -threads 25 -iterations 100 -keySize 50 
> -valueSize 100 -syncInterval 10':
> The verify fails on all runs:
> Exception in thread "main" java.lang.IllegalStateException: Counted=12390228, 
> expected=2500
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.run(WALPerformanceEvaluation.java:368)
>   
>   at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.innerMain(WALPerformanceEvaluation.java:597)
>   
> at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.main(WALPerformanceEvaluation.java:601)
> I need to fix test or figure what is wrong in asyncfswal.
> Also seeing these when I run w/ one thread only:
> 2017-11-28 21:25:49,952 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> Log has a spew of them.
> Has stuff like this too:
> 2017-11-28 21:25:40,065 WARN  [Close-WAL-Writer-3] 
> wal.AsyncProtobufLogWriter: normal close failed, try recover  
>  
> java.io.IOException: stream already broken
>   
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:510)
>   
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.lambda$close$12(FanOutOneBlockAsyncDFSOutput.java:550)
>   
>at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> 
> at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
>   at java.lang.Thread.run(Thread.java:748)
> Starts out spewing EMPTY here:
> 2017-11-28 21:16:52,051 INFO  [main] regionserver.HRegion: Setting 
> FlushNonSloppyStoresFirstPolicy for the 
> region=WALPerformanceEvaluation:0,,1511932610787.deca03e0ca447fa25d02fe9cd6e31aa4.
> 2017-11-28 21:16:52,058 DEBUG [main] regionserver.HRegion: Found 0 recovered 
> edits file(s) under 
> hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4
> 2017-11-28 21:16:52,068 DEBUG [main] regionserver.FlushLargeStoresPolicy: No 
> hbase.hregion.percolumnfamilyflush.size.lower.bound set in description of 
> table WALPerformanceEvaluation:0, use config (134217728) instead
> 2017-11-28 21:16:52,084 DEBUG [main] wal.WALSplitter: Wrote 
> file=hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4/recovered.edits/2.seqid,
>  newSeqId=2, maxSeqId=0
> 2017-11-28 21:16:52,084 INFO  [main] regionserver.HRegion: Onlined 
> deca03e0ca447fa25d02fe9cd6e31aa4; next sequenceid=2
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185

[jira] [Commented] (HBASE-19371) Running WALPerformanceEvaluation against asyncfswal throws exceptions

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270315#comment-16270315
 ] 

Duo Zhang commented on HBASE-19371:
---

[~stack] I tried the same command with you, it worked. Which HDFS version do 
you use? Seems you use 2.8.0? That maybe a problem as we only run UTs with 
2.7.x, so maybe AsyncFSWAL is not compatible with 2.8.x...

Thanks.

> Running WALPerformanceEvaluation against asyncfswal throws exceptions
> -
>
> Key: HBASE-19371
> URL: https://issues.apache.org/jira/browse/HBASE-19371
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Fix For: 2.0.0-beta-1
>
>
> Was trying to do a perf eval on asyncfswal. I ran w/ these args:
>  Performance counter stats for '/home/stack/hbase/bin/hbase --config 
> /home/stack/conf_hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation 
> -path /user/stack/logs/ -verify -threads 25 -iterations 100 -keySize 50 
> -valueSize 100 -syncInterval 10':
> The verify fails on all runs:
> Exception in thread "main" java.lang.IllegalStateException: Counted=12390228, 
> expected=2500
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.run(WALPerformanceEvaluation.java:368)
>   
>   at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.innerMain(WALPerformanceEvaluation.java:597)
>   
> at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.main(WALPerformanceEvaluation.java:601)
> I need to fix test or figure what is wrong in asyncfswal.
> Also seeing these when I run w/ one thread only:
> 2017-11-28 21:25:49,952 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> Log has a spew of them.
> Has stuff like this too:
> 2017-11-28 21:25:40,065 WARN  [Close-WAL-Writer-3] 
> wal.AsyncProtobufLogWriter: normal close failed, try recover  
>  
> java.io.IOException: stream already broken
>   
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:510)
>   
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.lambda$close$12(FanOutOneBlockAsyncDFSOutput.java:550)
>   
>at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> 
> at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
>   at java.lang.Thread.run(Thread.java:748)
> Starts out spewing EMPTY here:
> 2017-11-28 21:16:52,051 INFO  [main] regionserver.HRegion: Setting 
> FlushNonSloppyStoresFirstPolicy for the 
> region=WALPerformanceEvaluation:0,,1511932610787.deca03e0ca447fa25d02fe9cd6e31aa4.
> 2017-11-28 21:16:52,058 DEBUG [main] regionserver.HRegion: Found 0 recovered 
> edits file(s) under 
> hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4
> 2017-11-28 21:16:52,068 DEBUG [main] regionserver.FlushLargeStoresPolicy: No 
> hbase.hregion.percolumnfamilyflush.size.lower.bound set in description of 
> table WALPerformanceEvaluation:0, use config (134217728) instead
> 2017-11-28 21:16:52,084 DEBUG [main] wal.WALSplitter: Wrote 
> file=hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4/recovered.edits/2.seqid,
>  newSeqId=2, maxSeqId=0
> 2017-11-28 21:16:52,084 INFO  [main] regionserver.HRegion: Onlined 
> deca03e0ca447fa25d02fe9cd6e31aa4; next sequenceid=2
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected

[jira] [Commented] (HBASE-17049) Do not issue sync request when there are still entries in ringbuffer

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270311#comment-16270311
 ] 

Hudson commented on HBASE-17049:


FAILURE: Integrated in Jenkins build HBase-2.0 #934 (See 
[https://builds.apache.org/job/HBase-2.0/934/])
HBASE-17049 Do not issue sync request when there are still entries in 
(zhangduo: rev 8688da9e9c4b54d5ccd22bc10eab5ac873325522)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java


> Do not issue sync request when there are still entries in ringbuffer
> 
>
> Key: HBASE-17049
> URL: https://issues.apache.org/jira/browse/HBASE-17049
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17049.patch, delay-sync.patch
>
>
> https://issues.apache.org/jira/browse/HBASE-16890?focusedCommentId=15647590=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15647590



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19096) Add RowMutions batch support in AsyncTable

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270312#comment-16270312
 ] 

Hudson commented on HBASE-19096:


FAILURE: Integrated in Jenkins build HBase-2.0 #934 (See 
[https://builds.apache.org/job/HBase-2.0/934/])
HBASE-19096 Add RowMutions batch support in AsyncTable (jerryjch: rev 
0c4c3955380e1927311a8f4b092e23532d2e795f)
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTable.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncBatchRpcRetryingCaller.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableBatch.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java


> Add RowMutions batch support in AsyncTable
> --
>
> Key: HBASE-19096
> URL: https://issues.apache.org/jira/browse/HBASE-19096
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19096-master-v2.patch, 
> HBASE-19096-master-v3.patch, HBASE-19096-master-v4.patch, 
> HBASE-19096-master.patch
>
>
> Batch support for RowMutations has been added in the Table interface, but is 
> not in AsyncTable. This JIRA will add it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19290) Reduce zk request when doing split log

2017-11-28 Thread binlijin (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

binlijin updated HBASE-19290:
-
Attachment: HBASE-19290.master.006.patch

> Reduce zk request when doing split log
> --
>
> Key: HBASE-19290
> URL: https://issues.apache.org/jira/browse/HBASE-19290
> Project: HBase
>  Issue Type: Improvement
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-19290.master.001.patch, 
> HBASE-19290.master.002.patch, HBASE-19290.master.003.patch, 
> HBASE-19290.master.004.patch, HBASE-19290.master.005.patch, 
> HBASE-19290.master.006.patch, HBASE-19290.master.006.patch
>
>
> We observe once the cluster has 1000+ nodes and when hundreds of nodes abort 
> and doing split log, the split is very very slow, and we find the 
> regionserver and master wait on the zookeeper response, so we need to reduce 
> zookeeper request and pressure for big cluster.
> (1) Reduce request to rsZNode, every time calculateAvailableSplitters will 
> get rsZNode's children from zookeeper, when cluster is huge, this is heavy. 
> This patch reduce the request. 
> (2) When the regionserver has max split tasks running, it may still trying to 
> grab task and issue zookeeper request, we should sleep and wait until we can 
> grab tasks again.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19362) Remove unused imports from hbase-thrift module

2017-11-28 Thread Guangxu Cheng (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng updated HBASE-19362:
--
Assignee: Guangxu Cheng
  Status: Patch Available  (was: Open)

> Remove unused imports from hbase-thrift module
> --
>
> Key: HBASE-19362
> URL: https://issues.apache.org/jira/browse/HBASE-19362
> Project: HBase
>  Issue Type: Sub-task
>  Components: Thrift
>Reporter: Jan Hentschel
>Assignee: Guangxu Cheng
>Priority: Minor
> Attachments: HBASE-19362.master.001.patch, UnusedImport.xml
>
>
> Currently there are three warnings regarding unused imports in the 
> *hbase-thrift* module (see the attached XML). The unused imports should be 
> removed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19362) Remove unused imports from hbase-thrift module

2017-11-28 Thread Guangxu Cheng (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng updated HBASE-19362:
--
Attachment: HBASE-19362.master.001.patch

Trivial patch for master. Thanks

> Remove unused imports from hbase-thrift module
> --
>
> Key: HBASE-19362
> URL: https://issues.apache.org/jira/browse/HBASE-19362
> Project: HBase
>  Issue Type: Sub-task
>  Components: Thrift
>Reporter: Jan Hentschel
>Priority: Minor
> Attachments: HBASE-19362.master.001.patch, UnusedImport.xml
>
>
> Currently there are three warnings regarding unused imports in the 
> *hbase-thrift* module (see the attached XML). The unused imports should be 
> removed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19357) Bucket cache no longer L2 for LRU cache

2017-11-28 Thread Anoop Sam John (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-19357:
---
Attachment: HBASE-19357_V2.patch

Kept the name L2 as is (This can be Bucket cache as well as External cache)..  
For L1/LRU, this is an on heap cache we have.  So named its variables/methods 
with OnHeapCache way.  How is that? But the other is again L2. So still 
confusion?  Should just keep L1 only?

> Bucket cache no longer L2 for LRU cache
> ---
>
> Key: HBASE-19357
> URL: https://issues.apache.org/jira/browse/HBASE-19357
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19357.patch, HBASE-19357.patch, 
> HBASE-19357_V2.patch
>
>
> When Bucket cache is used, by default we dont configure it as an L2 cache 
> alone. The default setting is combined mode ON where the data blocks to 
> Bucket cache and index/bloom blocks go to LRU cache. But there is a way to 
> turn this off and make LRU as L1 and Bucket cache as a victim handler for L1. 
> It will be just L2.   
> After the off heap read path optimization Bucket cache is no longer slower 
> compared to L1. We have test results on data sizes from 12 GB.  The Alibaba 
> use case was also with 12 GB and they have observed a ~30% QPS improve over 
> the LRU cache.
> This issue is to remove the option for combined mode = false. So when Bucket 
> cache is in use, data blocks will go to it only and LRU will get only index 
> /meta/bloom blocks.   Bucket cache will no longer be configured as a victim 
> handler for LRU.
> Note : WHen external cache is in use, there only the L1 L2 thing comes. LRU 
> will be L1 and external cache act as its L2. That make full sense.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19290) Reduce zk request when doing split log

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270247#comment-16270247
 ] 

Hadoop QA commented on HBASE-19290:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
42s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
36s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
63m 33s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}137m 50s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
19s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}224m 49s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19290 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12899741/HBASE-19290.master.006.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 96502406ebab 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 
14:13:22 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / e67a3699c4 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10094/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10094/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10094/console |
| Powered by | Apache Yetus 0.6.0

[jira] [Updated] (HBASE-19359) Revisit the default config of hbase client retries number

2017-11-28 Thread Guanghao Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-19359:
---
Fix Version/s: 2.0.0-beta-1

> Revisit the default config of hbase client retries number
> -
>
> Key: HBASE-19359
> URL: https://issues.apache.org/jira/browse/HBASE-19359
> Project: HBase
>  Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19359.master.001.patch, 
> HBASE-19359.master.001.patch, HBASE-19359.master.001.patch
>
>
> This should be sub-task of HBASE-19148. As the retries number effect too many 
> unit tests. So I open this issue to see the Hadoop QA result.
> The default value of hbase.client.retries.number is 35. Plan to reduce this 
> to 10.
> And for server side, the default hbase.client.serverside.retries.multiplier 
> is 10. So the server side retries number is 35 * 10 = 350. It is too big! 
> Plan to reduce hbase.client.serverside.retries.multiplier to 3.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19359) Revisit the default config of hbase client retries number

2017-11-28 Thread Guanghao Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-19359:
---
Attachment: HBASE-19359.master.001.patch

> Revisit the default config of hbase client retries number
> -
>
> Key: HBASE-19359
> URL: https://issues.apache.org/jira/browse/HBASE-19359
> Project: HBase
>  Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-19359.master.001.patch, 
> HBASE-19359.master.001.patch, HBASE-19359.master.001.patch
>
>
> This should be sub-task of HBASE-19148. As the retries number effect too many 
> unit tests. So I open this issue to see the Hadoop QA result.
> The default value of hbase.client.retries.number is 35. Plan to reduce this 
> to 10.
> And for server side, the default hbase.client.serverside.retries.multiplier 
> is 10. So the server side retries number is 35 * 10 = 350. It is too big! 
> Plan to reduce hbase.client.serverside.retries.multiplier to 3.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19359) Revisit the default config of hbase client retries number

2017-11-28 Thread Guanghao Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270229#comment-16270229
 ] 

Guanghao Zhang commented on HBASE-19359:


All ut passed! Ping [~stack] for reviewing.
Let me retry for Hadoop QA.

> Revisit the default config of hbase client retries number
> -
>
> Key: HBASE-19359
> URL: https://issues.apache.org/jira/browse/HBASE-19359
> Project: HBase
>  Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-19359.master.001.patch, 
> HBASE-19359.master.001.patch, HBASE-19359.master.001.patch
>
>
> This should be sub-task of HBASE-19148. As the retries number effect too many 
> unit tests. So I open this issue to see the Hadoop QA result.
> The default value of hbase.client.retries.number is 35. Plan to reduce this 
> to 10.
> And for server side, the default hbase.client.serverside.retries.multiplier 
> is 10. So the server side retries number is 35 * 10 = 350. It is too big! 
> Plan to reduce hbase.client.serverside.retries.multiplier to 3.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19354) [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270223#comment-16270223
 ] 

Hadoop QA commented on HBASE-19354:
---

(!) A patch to the testing environment has been detected. 
Re-executing against the patched versions to perform further tests. 
The console is at 
https://builds.apache.org/job/PreCommit-HBASE-Build/10102/console in case of 
problems.


> [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151
> ---
>
> Key: HBASE-19354
> URL: https://issues.apache.org/jira/browse/HBASE-19354
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.3.2, 1.2.7
>
> Attachments: HBASE-19354.branch-1.2.001.patch, 
> HBASE-19354.branch-1.4.001.patch, HBASE-19354.branch-1.4.002.patch
>
>
> HDFS mini cluster hangs when it runs on openjdk 151. See parent issue where 
> [~chia7712] turns up the hang and then Xiao Chen confirms up in the parent 
> issue in a comment. Lets do what Xiao Chen suggests which comes from a Todd 
> approach over in kudu where we install newer versions of the jdk so we 
> by-pass the hdfs hangs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18233) We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270215#comment-16270215
 ] 

Hadoop QA commented on HBASE-18233:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m  
1s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1.3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 1s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
33s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
34s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
23s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
47s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
52s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
34s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
34s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
17s{color} | {color:green} hbase-server: The patch generated 0 new + 397 
unchanged - 1 fixed = 397 total (was 398) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  2m 
19s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
25m 24s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3. 
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 73m 13s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}134m 27s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.regionserver.TestReplicator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:dca6535 |
| JIRA Issue | HBASE-18233 |
| JIRA Patch URL |

[jira] [Updated] (HBASE-19350) TestMetaWithReplicas is flaky in branch-1

2017-11-28 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19350:
--
Attachment: HBASE-19350.branch-1.v0.test (1).patch

> TestMetaWithReplicas is flaky in branch-1
> -
>
> Key: HBASE-19350
> URL: https://issues.apache.org/jira/browse/HBASE-19350
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 1.4.0, 2.0.0-beta-1
>
> Attachments: HBASE-19350.branch-1.v0.test (1).patch, 
> HBASE-19350.branch-1.v0.test.patch, HBASE-19350.branch-1.v0.test.patch, 
> HBASE-19350.branch-1.v0.test.patch
>
>
> If the size of RegionsInTransition is zero, the list passed to 
> {{ClusterStatus}} will be null.
> {code:title=ClusterStatus.java}
> Set rit = null;
> if (!proto.getRegionsInTransitionList().isEmpty()) {
>   rit = new 
> HashSet(proto.getRegionsInTransitionList().size());
>   for (RegionInTransition region : proto.getRegionsInTransitionList()) {
> RegionState value = RegionState.convert(region.getRegionState());
> rit.add(value);
>   }
> }
> {code}
> It causes NPE if someone try to do the for-each work. The HBaseFsckRepair is 
> a real-life example.
> {code:title=HBaseFsckRepair.java}
> for (RegionState rs: 
> admin.getClusterStatus().getRegionsInTransition()) {
>   if (rs.getRegion().equals(region)) {
> inTransition = true;
> break;
>   }
> }
> {code}
> branch-2/master don't have this issue as the list of RegionsInTransition 
> passed to {{ClusterStatus}} never be null.
> {code:title=ProtobufUtil.java}
> List rit =
>   new ArrayList<>(proto.getRegionsInTransitionList().size());
> for (RegionInTransition region : proto.getRegionsInTransitionList()) {
>   RegionState value = RegionState.convert(region.getRegionState());
>   rit.add(value);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19350) TestMetaWithReplicas is flaky in branch-1

2017-11-28 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270214#comment-16270214
 ] 

stack commented on HBASE-19350:
---

https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/10100/

> TestMetaWithReplicas is flaky in branch-1
> -
>
> Key: HBASE-19350
> URL: https://issues.apache.org/jira/browse/HBASE-19350
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 1.4.0, 2.0.0-beta-1
>
> Attachments: HBASE-19350.branch-1.v0.test (1).patch, 
> HBASE-19350.branch-1.v0.test.patch, HBASE-19350.branch-1.v0.test.patch, 
> HBASE-19350.branch-1.v0.test.patch
>
>
> If the size of RegionsInTransition is zero, the list passed to 
> {{ClusterStatus}} will be null.
> {code:title=ClusterStatus.java}
> Set rit = null;
> if (!proto.getRegionsInTransitionList().isEmpty()) {
>   rit = new 
> HashSet(proto.getRegionsInTransitionList().size());
>   for (RegionInTransition region : proto.getRegionsInTransitionList()) {
> RegionState value = RegionState.convert(region.getRegionState());
> rit.add(value);
>   }
> }
> {code}
> It causes NPE if someone try to do the for-each work. The HBaseFsckRepair is 
> a real-life example.
> {code:title=HBaseFsckRepair.java}
> for (RegionState rs: 
> admin.getClusterStatus().getRegionsInTransition()) {
>   if (rs.getRegion().equals(region)) {
> inTransition = true;
> break;
>   }
> }
> {code}
> branch-2/master don't have this issue as the list of RegionsInTransition 
> passed to {{ClusterStatus}} never be null.
> {code:title=ProtobufUtil.java}
> List rit =
>   new ArrayList<>(proto.getRegionsInTransitionList().size());
> for (RegionInTransition region : proto.getRegionsInTransitionList()) {
>   RegionState value = RegionState.convert(region.getRegionState());
>   rit.add(value);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19350) TestMetaWithReplicas is flaky in branch-1

2017-11-28 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270213#comment-16270213
 ] 

stack commented on HBASE-19350:
---

Failed here

java.lang.AssertionError: Did not find a new RegionServer to use
at 
org.apache.hadoop.hbase.client.TestHTableMultiplexerFlushCache.testOnRegionMove(TestHTableMultiplexerFlushCache.java:160)

Let me retry then if it is basically working.

> TestMetaWithReplicas is flaky in branch-1
> -
>
> Key: HBASE-19350
> URL: https://issues.apache.org/jira/browse/HBASE-19350
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 1.4.0, 2.0.0-beta-1
>
> Attachments: HBASE-19350.branch-1.v0.test.patch, 
> HBASE-19350.branch-1.v0.test.patch, HBASE-19350.branch-1.v0.test.patch
>
>
> If the size of RegionsInTransition is zero, the list passed to 
> {{ClusterStatus}} will be null.
> {code:title=ClusterStatus.java}
> Set rit = null;
> if (!proto.getRegionsInTransitionList().isEmpty()) {
>   rit = new 
> HashSet(proto.getRegionsInTransitionList().size());
>   for (RegionInTransition region : proto.getRegionsInTransitionList()) {
> RegionState value = RegionState.convert(region.getRegionState());
> rit.add(value);
>   }
> }
> {code}
> It causes NPE if someone try to do the for-each work. The HBaseFsckRepair is 
> a real-life example.
> {code:title=HBaseFsckRepair.java}
> for (RegionState rs: 
> admin.getClusterStatus().getRegionsInTransition()) {
>   if (rs.getRegion().equals(region)) {
> inTransition = true;
> break;
>   }
> }
> {code}
> branch-2/master don't have this issue as the list of RegionsInTransition 
> passed to {{ClusterStatus}} never be null.
> {code:title=ProtobufUtil.java}
> List rit =
>   new ArrayList<>(proto.getRegionsInTransitionList().size());
> for (RegionInTransition region : proto.getRegionsInTransitionList()) {
>   RegionState value = RegionState.convert(region.getRegionState());
>   rit.add(value);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19354) [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270210#comment-16270210
 ] 

Hadoop QA commented on HBASE-19354:
---

(!) A patch to the testing environment has been detected. 
Re-executing against the patched versions to perform further tests. 
The console is at 
https://builds.apache.org/job/PreCommit-HBASE-Build/10099/console in case of 
problems.


> [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151
> ---
>
> Key: HBASE-19354
> URL: https://issues.apache.org/jira/browse/HBASE-19354
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.3.2, 1.2.7
>
> Attachments: HBASE-19354.branch-1.2.001.patch, 
> HBASE-19354.branch-1.4.001.patch, HBASE-19354.branch-1.4.002.patch
>
>
> HDFS mini cluster hangs when it runs on openjdk 151. See parent issue where 
> [~chia7712] turns up the hang and then Xiao Chen confirms up in the parent 
> issue in a comment. Lets do what Xiao Chen suggests which comes from a Todd 
> approach over in kudu where we install newer versions of the jdk so we 
> by-pass the hdfs hangs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19354) [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151

2017-11-28 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19354:
--
Attachment: HBASE-19354.branch-1.4.002.patch

> [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151
> ---
>
> Key: HBASE-19354
> URL: https://issues.apache.org/jira/browse/HBASE-19354
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.3.2, 1.2.7
>
> Attachments: HBASE-19354.branch-1.2.001.patch, 
> HBASE-19354.branch-1.4.001.patch, HBASE-19354.branch-1.4.002.patch
>
>
> HDFS mini cluster hangs when it runs on openjdk 151. See parent issue where 
> [~chia7712] turns up the hang and then Xiao Chen confirms up in the parent 
> issue in a comment. Lets do what Xiao Chen suggests which comes from a Todd 
> approach over in kudu where we install newer versions of the jdk so we 
> by-pass the hdfs hangs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19371) Running WALPerformanceEvaluation against asyncfswal throws exceptions

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270207#comment-16270207
 ] 

Duo Zhang commented on HBASE-19371:
---

{quote}
This is tip of branch-2 minus the commit of the change to the ringbuffer (I'd 
not pulled in that change).
{quote}

Which commit?

> Running WALPerformanceEvaluation against asyncfswal throws exceptions
> -
>
> Key: HBASE-19371
> URL: https://issues.apache.org/jira/browse/HBASE-19371
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Fix For: 2.0.0-beta-1
>
>
> Was trying to do a perf eval on asyncfswal. I ran w/ these args:
>  Performance counter stats for '/home/stack/hbase/bin/hbase --config 
> /home/stack/conf_hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation 
> -path /user/stack/logs/ -verify -threads 25 -iterations 100 -keySize 50 
> -valueSize 100 -syncInterval 10':
> The verify fails on all runs:
> Exception in thread "main" java.lang.IllegalStateException: Counted=12390228, 
> expected=2500
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.run(WALPerformanceEvaluation.java:368)
>   
>   at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.innerMain(WALPerformanceEvaluation.java:597)
>   
> at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.main(WALPerformanceEvaluation.java:601)
> I need to fix test or figure what is wrong in asyncfswal.
> Also seeing these when I run w/ one thread only:
> 2017-11-28 21:25:49,952 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> Log has a spew of them.
> Has stuff like this too:
> 2017-11-28 21:25:40,065 WARN  [Close-WAL-Writer-3] 
> wal.AsyncProtobufLogWriter: normal close failed, try recover  
>  
> java.io.IOException: stream already broken
>   
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:510)
>   
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.lambda$close$12(FanOutOneBlockAsyncDFSOutput.java:550)
>   
>at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> 
> at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
>   at java.lang.Thread.run(Thread.java:748)
> Starts out spewing EMPTY here:
> 2017-11-28 21:16:52,051 INFO  [main] regionserver.HRegion: Setting 
> FlushNonSloppyStoresFirstPolicy for the 
> region=WALPerformanceEvaluation:0,,1511932610787.deca03e0ca447fa25d02fe9cd6e31aa4.
> 2017-11-28 21:16:52,058 DEBUG [main] regionserver.HRegion: Found 0 recovered 
> edits file(s) under 
> hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4
> 2017-11-28 21:16:52,068 DEBUG [main] regionserver.FlushLargeStoresPolicy: No 
> hbase.hregion.percolumnfamilyflush.size.lower.bound set in description of 
> table WALPerformanceEvaluation:0, use config (134217728) instead
> 2017-11-28 21:16:52,084 DEBUG [main] wal.WALSplitter: Wrote 
> file=hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4/recovered.edits/2.seqid,
>  newSeqId=2, maxSeqId=0
> 2017-11-28 21:16:52,084 INFO  [main] regionserver.HRegion: Onlined 
> deca03e0ca447fa25d02fe9cd6e31aa4; next sequenceid=2
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
>

[jira] [Commented] (HBASE-15536) Make AsyncFSWAL as our default WAL

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270204#comment-16270204
 ] 

Duo Zhang commented on HBASE-15536:
---

2.7.4 is OK sir. In FanOutOneBlockAsyncDFSOutputHelper we will use reflection 
to decide to use PBHelper or PBHelperClient. Maybe the error message is a bit 
confusing and makes you think we can not work together with 2.7.x.

> Make AsyncFSWAL as our default WAL
> --
>
> Key: HBASE-15536
> URL: https://issues.apache.org/jira/browse/HBASE-15536
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch, 
> HBASE-15536-v3.patch, HBASE-15536-v4.patch, HBASE-15536-v5.patch, 
> HBASE-15536.patch, latesttrunk_asyncWAL_50threads_10cols.jfr, 
> latesttrunk_defaultWAL_50threads_10cols.jfr
>
>
> As it should be predicated on passing basic cluster ITBLL



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19371) Running WALPerformanceEvaluation against asyncfswal throws exceptions

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270206#comment-16270206
 ] 

Duo Zhang commented on HBASE-19371:
---

I haven't seen this before. Let me try.

> Running WALPerformanceEvaluation against asyncfswal throws exceptions
> -
>
> Key: HBASE-19371
> URL: https://issues.apache.org/jira/browse/HBASE-19371
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Fix For: 2.0.0-beta-1
>
>
> Was trying to do a perf eval on asyncfswal. I ran w/ these args:
>  Performance counter stats for '/home/stack/hbase/bin/hbase --config 
> /home/stack/conf_hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation 
> -path /user/stack/logs/ -verify -threads 25 -iterations 100 -keySize 50 
> -valueSize 100 -syncInterval 10':
> The verify fails on all runs:
> Exception in thread "main" java.lang.IllegalStateException: Counted=12390228, 
> expected=2500
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.run(WALPerformanceEvaluation.java:368)
>   
>   at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.innerMain(WALPerformanceEvaluation.java:597)
>   
> at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.main(WALPerformanceEvaluation.java:601)
> I need to fix test or figure what is wrong in asyncfswal.
> Also seeing these when I run w/ one thread only:
> 2017-11-28 21:25:49,952 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> Log has a spew of them.
> Has stuff like this too:
> 2017-11-28 21:25:40,065 WARN  [Close-WAL-Writer-3] 
> wal.AsyncProtobufLogWriter: normal close failed, try recover  
>  
> java.io.IOException: stream already broken
>   
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:510)
>   
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.lambda$close$12(FanOutOneBlockAsyncDFSOutput.java:550)
>   
>at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> 
> at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
>   at java.lang.Thread.run(Thread.java:748)
> Starts out spewing EMPTY here:
> 2017-11-28 21:16:52,051 INFO  [main] regionserver.HRegion: Setting 
> FlushNonSloppyStoresFirstPolicy for the 
> region=WALPerformanceEvaluation:0,,1511932610787.deca03e0ca447fa25d02fe9cd6e31aa4.
> 2017-11-28 21:16:52,058 DEBUG [main] regionserver.HRegion: Found 0 recovered 
> edits file(s) under 
> hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4
> 2017-11-28 21:16:52,068 DEBUG [main] regionserver.FlushLargeStoresPolicy: No 
> hbase.hregion.percolumnfamilyflush.size.lower.bound set in description of 
> table WALPerformanceEvaluation:0, use config (134217728) instead
> 2017-11-28 21:16:52,084 DEBUG [main] wal.WALSplitter: Wrote 
> file=hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4/recovered.edits/2.seqid,
>  newSeqId=2, maxSeqId=0
> 2017-11-28 21:16:52,084 INFO  [main] regionserver.HRegion: Onlined 
> deca03e0ca447fa25d02fe9cd6e31aa4; next sequenceid=2
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1]

[jira] [Commented] (HBASE-19357) Bucket cache no longer L2 for LRU cache

2017-11-28 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270192#comment-16270192
 ] 

Anoop Sam John commented on HBASE-19357:


{quote}
In CacheConfig.java :
-  public static final String BUCKET_CACHE_COMBINED_KEY =
-  "hbase.bucketcache.combinedcache.enabled";
I assume deprecation should be added for the above.
{quote}
No need I believe as CacheConfig is marked Private.  The config as such is 
deprecated. We add log when user explicitly set it with false.  On that log, we 
normally log warn for these kind of situations. It is not an error occurred in 
the server run time right?  I think warn is enough. WDYT?

> Bucket cache no longer L2 for LRU cache
> ---
>
> Key: HBASE-19357
> URL: https://issues.apache.org/jira/browse/HBASE-19357
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19357.patch, HBASE-19357.patch
>
>
> When Bucket cache is used, by default we dont configure it as an L2 cache 
> alone. The default setting is combined mode ON where the data blocks to 
> Bucket cache and index/bloom blocks go to LRU cache. But there is a way to 
> turn this off and make LRU as L1 and Bucket cache as a victim handler for L1. 
> It will be just L2.   
> After the off heap read path optimization Bucket cache is no longer slower 
> compared to L1. We have test results on data sizes from 12 GB.  The Alibaba 
> use case was also with 12 GB and they have observed a ~30% QPS improve over 
> the LRU cache.
> This issue is to remove the option for combined mode = false. So when Bucket 
> cache is in use, data blocks will go to it only and LRU will get only index 
> /meta/bloom blocks.   Bucket cache will no longer be configured as a victim 
> handler for LRU.
> Note : WHen external cache is in use, there only the L1 L2 thing comes. LRU 
> will be L1 and external cache act as its L2. That make full sense.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19354) [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270185#comment-16270185
 ] 

Hadoop QA commented on HBASE-19354:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue}  0m  
7s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} branch-1.4 Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green}  0m 
 0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}  0m 39s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:89abc29 |
| JIRA Issue | HBASE-19354 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12899661/HBASE-19354.branch-1.4.001.patch
 |
| Optional Tests |  asflicense  shellcheck  shelldocs  |
| uname | Linux 62f3da256df6 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | branch-1.4 / 5f0219b |
| maven | version: Apache Maven 3.0.5 |
| shellcheck | v0.4.6 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10098/artifact/patchprocess/whitespace-eol.txt
 |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10098/console |
| Powered by | Apache Yetus 0.6.0   http://yetus.apache.org |


This message was automatically generated.



> [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151
> ---
>
> Key: HBASE-19354
> URL: https://issues.apache.org/jira/browse/HBASE-19354
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.3.2, 1.2.7
>
> Attachments: HBASE-19354.branch-1.2.001.patch, 
> HBASE-19354.branch-1.4.001.patch
>
>
> HDFS mini cluster hangs when it runs on openjdk 151. See parent issue where 
> [~chia7712] turns up the hang and then Xiao Chen confirms up in the parent 
> issue in a comment. Lets do what Xiao Chen suggests which comes from a Todd 
> approach over in kudu where we install newer versions of the jdk so we 
> by-pass the hdfs hangs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19354) [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270184#comment-16270184
 ] 

Hadoop QA commented on HBASE-19354:
---

(!) A patch to the testing environment has been detected. 
Re-executing against the patched versions to perform further tests. 
The console is at 
https://builds.apache.org/job/PreCommit-HBASE-Build/10098/console in case of 
problems.


> [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151
> ---
>
> Key: HBASE-19354
> URL: https://issues.apache.org/jira/browse/HBASE-19354
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.3.2, 1.2.7
>
> Attachments: HBASE-19354.branch-1.2.001.patch, 
> HBASE-19354.branch-1.4.001.patch
>
>
> HDFS mini cluster hangs when it runs on openjdk 151. See parent issue where 
> [~chia7712] turns up the hang and then Xiao Chen confirms up in the parent 
> issue in a comment. Lets do what Xiao Chen suggests which comes from a Todd 
> approach over in kudu where we install newer versions of the jdk so we 
> by-pass the hdfs hangs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19354) [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151

2017-11-28 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19354:
--
Status: Patch Available  (was: Open)

> [branch-1] Build using a jdk that is beyond ubuntu trusty's openjdk-151
> ---
>
> Key: HBASE-19354
> URL: https://issues.apache.org/jira/browse/HBASE-19354
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.3.2, 1.2.7
>
> Attachments: HBASE-19354.branch-1.2.001.patch, 
> HBASE-19354.branch-1.4.001.patch
>
>
> HDFS mini cluster hangs when it runs on openjdk 151. See parent issue where 
> [~chia7712] turns up the hang and then Xiao Chen confirms up in the parent 
> issue in a comment. Lets do what Xiao Chen suggests which comes from a Todd 
> approach over in kudu where we install newer versions of the jdk so we 
> by-pass the hdfs hangs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19350) TestMetaWithReplicas is flaky in branch-1

2017-11-28 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270180#comment-16270180
 ] 

stack commented on HBASE-19350:
---

Its been running 7 hours.  Let me move up to branch-1 this change:

commit e77c5787491045ab45884d4ccceb6c66f51efe7d
Author: Michael Stack 
Date:   Mon Nov 27 15:27:32 2017 -0800

HBASE-19354 [branch-1] Build using a jdk that is beyond ubuntu trusty's
openjdk-151

Amend our DockerFile so it gets jdks from azul repo.

> TestMetaWithReplicas is flaky in branch-1
> -
>
> Key: HBASE-19350
> URL: https://issues.apache.org/jira/browse/HBASE-19350
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 1.4.0, 2.0.0-beta-1
>
> Attachments: HBASE-19350.branch-1.v0.test.patch, 
> HBASE-19350.branch-1.v0.test.patch, HBASE-19350.branch-1.v0.test.patch
>
>
> If the size of RegionsInTransition is zero, the list passed to 
> {{ClusterStatus}} will be null.
> {code:title=ClusterStatus.java}
> Set rit = null;
> if (!proto.getRegionsInTransitionList().isEmpty()) {
>   rit = new 
> HashSet(proto.getRegionsInTransitionList().size());
>   for (RegionInTransition region : proto.getRegionsInTransitionList()) {
> RegionState value = RegionState.convert(region.getRegionState());
> rit.add(value);
>   }
> }
> {code}
> It causes NPE if someone try to do the for-each work. The HBaseFsckRepair is 
> a real-life example.
> {code:title=HBaseFsckRepair.java}
> for (RegionState rs: 
> admin.getClusterStatus().getRegionsInTransition()) {
>   if (rs.getRegion().equals(region)) {
> inTransition = true;
> break;
>   }
> }
> {code}
> branch-2/master don't have this issue as the list of RegionsInTransition 
> passed to {{ClusterStatus}} never be null.
> {code:title=ProtobufUtil.java}
> List rit =
>   new ArrayList<>(proto.getRegionsInTransitionList().size());
> for (RegionInTransition region : proto.getRegionsInTransitionList()) {
>   RegionState value = RegionState.convert(region.getRegionState());
>   rit.add(value);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19371) Running WALPerformanceEvaluation against asyncfswal throws exceptions

2017-11-28 Thread stack (JIRA)

stack created HBASE-19371:
-

 Summary: Running WALPerformanceEvaluation against asyncfswal 
throws exceptions
 Key: HBASE-19371
 URL: https://issues.apache.org/jira/browse/HBASE-19371
 Project: HBase
  Issue Type: Bug
Reporter: stack


Was trying to do a perf eval on asyncfswal. I ran w/ these args:

 Performance counter stats for '/home/stack/hbase/bin/hbase --config 
/home/stack/conf_hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation 
-path /user/stack/logs/ -verify -threads 25 -iterations 100 -keySize 50 
-valueSize 100 -syncInterval 10':

The verify fails on all runs:

Exception in thread "main" java.lang.IllegalStateException: Counted=12390228, 
expected=2500
  at 
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.run(WALPerformanceEvaluation.java:368)

at 
org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
  at 
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.innerMain(WALPerformanceEvaluation.java:597)

  at 
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.main(WALPerformanceEvaluation.java:601)

I need to fix test or figure what is wrong in asyncfswal.

Also seeing these when I run w/ one thread only:

2017-11-28 21:25:49,952 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: RingBufferTruck 
with unexpected type: EMPTY

Log has a spew of them.

Has stuff like this too:

2017-11-28 21:25:40,065 WARN  [Close-WAL-Writer-3] wal.AsyncProtobufLogWriter: 
normal close failed, try recover
   java.io.IOException: stream already 
broken  

at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:510)

at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.lambda$close$12(FanOutOneBlockAsyncDFSOutput.java:550)

 at 
org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
  at 
org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
  at 
org.apache.hadoop.hbase.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
  at 
org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at 
org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
  at java.lang.Thread.run(Thread.java:748)

Starts out spewing EMPTY here:

2017-11-28 21:16:52,051 INFO  [main] regionserver.HRegion: Setting 
FlushNonSloppyStoresFirstPolicy for the 
region=WALPerformanceEvaluation:0,,1511932610787.deca03e0ca447fa25d02fe9cd6e31aa4.
2017-11-28 21:16:52,058 DEBUG [main] regionserver.HRegion: Found 0 recovered 
edits file(s) under 
hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4
2017-11-28 21:16:52,068 DEBUG [main] regionserver.FlushLargeStoresPolicy: No 
hbase.hregion.percolumnfamilyflush.size.lower.bound set in description of table 
WALPerformanceEvaluation:0, use config (134217728) instead
2017-11-28 21:16:52,084 DEBUG [main] wal.WALSplitter: Wrote 
file=hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4/recovered.edits/2.seqid,
 newSeqId=2, maxSeqId=0
2017-11-28 21:16:52,084 INFO  [main] regionserver.HRegion: Onlined 
deca03e0ca447fa25d02fe9cd6e31aa4; next sequenceid=2
2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: RingBufferTruck 
with unexpected type: EMPTY
2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: RingBufferTruck 
with unexpected type: EMPTY
2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: RingBufferTruck 
with unexpected type: EMPTY
2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: RingBufferTruck 
with unexpected type: EMPTY
2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: RingBufferTruck 
with unexpected type: EMPTY
2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: RingBufferTruck 
with unexpected type: EMPTY
2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1]

[jira] [Updated] (HBASE-19371) Running WALPerformanceEvaluation against asyncfswal throws exceptions

2017-11-28 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19371:
--
Fix Version/s: 2.0.0-beta-1

> Running WALPerformanceEvaluation against asyncfswal throws exceptions
> -
>
> Key: HBASE-19371
> URL: https://issues.apache.org/jira/browse/HBASE-19371
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Fix For: 2.0.0-beta-1
>
>
> Was trying to do a perf eval on asyncfswal. I ran w/ these args:
>  Performance counter stats for '/home/stack/hbase/bin/hbase --config 
> /home/stack/conf_hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation 
> -path /user/stack/logs/ -verify -threads 25 -iterations 100 -keySize 50 
> -valueSize 100 -syncInterval 10':
> The verify fails on all runs:
> Exception in thread "main" java.lang.IllegalStateException: Counted=12390228, 
> expected=2500
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.run(WALPerformanceEvaluation.java:368)
>   
>   at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>   at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.innerMain(WALPerformanceEvaluation.java:597)
>   
> at 
> org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.main(WALPerformanceEvaluation.java:601)
> I need to fix test or figure what is wrong in asyncfswal.
> Also seeing these when I run w/ one thread only:
> 2017-11-28 21:25:49,952 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> Log has a spew of them.
> Has stuff like this too:
> 2017-11-28 21:25:40,065 WARN  [Close-WAL-Writer-3] 
> wal.AsyncProtobufLogWriter: normal close failed, try recover  
>  
> java.io.IOException: stream already broken
>   
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:510)
>   
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.lambda$close$12(FanOutOneBlockAsyncDFSOutput.java:550)
>   
>at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
>   at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> 
> at 
> org.apache.hadoop.hbase.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
>   at java.lang.Thread.run(Thread.java:748)
> Starts out spewing EMPTY here:
> 2017-11-28 21:16:52,051 INFO  [main] regionserver.HRegion: Setting 
> FlushNonSloppyStoresFirstPolicy for the 
> region=WALPerformanceEvaluation:0,,1511932610787.deca03e0ca447fa25d02fe9cd6e31aa4.
> 2017-11-28 21:16:52,058 DEBUG [main] regionserver.HRegion: Found 0 recovered 
> edits file(s) under 
> hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4
> 2017-11-28 21:16:52,068 DEBUG [main] regionserver.FlushLargeStoresPolicy: No 
> hbase.hregion.percolumnfamilyflush.size.lower.bound set in description of 
> table WALPerformanceEvaluation:0, use config (134217728) instead
> 2017-11-28 21:16:52,084 DEBUG [main] wal.WALSplitter: Wrote 
> file=hdfs://ve0524.halxg.cloudera.com:8020/user/stack/logs/data/WALPerformanceEvaluation/0/deca03e0ca447fa25d02fe9cd6e31aa4/recovered.edits/2.seqid,
>  newSeqId=2, maxSeqId=0
> 2017-11-28 21:16:52,084 INFO  [main] regionserver.HRegion: Onlined 
> deca03e0ca447fa25d02fe9cd6e31aa4; next sequenceid=2
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
> 2017-11-28 21:16:52,185 WARN  [AsyncFSWAL-1-1] wal.AsyncFSWAL: 
> RingBufferTruck with unexpected type: EMPTY
>

[jira] [Commented] (HBASE-15536) Make AsyncFSWAL as our default WAL

2017-11-28 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270157#comment-16270157
 ] 

stack commented on HBASE-15536:
---

I think its ok. We can update our hadoop np.

> Make AsyncFSWAL as our default WAL
> --
>
> Key: HBASE-15536
> URL: https://issues.apache.org/jira/browse/HBASE-15536
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch, 
> HBASE-15536-v3.patch, HBASE-15536-v4.patch, HBASE-15536-v5.patch, 
> HBASE-15536.patch, latesttrunk_asyncWAL_50threads_10cols.jfr, 
> latesttrunk_defaultWAL_50threads_10cols.jfr
>
>
> As it should be predicated on passing basic cluster ITBLL



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19163) "Maximum lock count exceeded" from region server's batch processing

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270156#comment-16270156
 ] 

Hadoop QA commented on HBASE-19163:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 7 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
 7s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
 3s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
54m  7s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}113m 26s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
19s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}187m 15s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19163 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12899731/HBASE-19163.master.009.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux ad04935feeda 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / f6582400be |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10093/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10093/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10093/console |
| Powered by | Apache Yetus 0.6.0   http://yetus.apache.org |


This message was automatically generated.



> "Maximum lock count exceeded" from region server's batch

[jira] [Commented] (HBASE-19370) Replace all uses of RowTooBigException from o.a.h.server with class of same name from o.a.h.client

2017-11-28 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270152#comment-16270152
 ] 

Anoop Sam John commented on HBASE-19370:


Was it for BC, the throwing Exception type was not changed for branch-1?   In 
Master we refer the o.a.h.client.RowTooBigException only.   

> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> --
>
> Key: HBASE-19370
> URL: https://issues.apache.org/jira/browse/HBASE-19370
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 1.4.0, 2.0.0-beta-1
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Attachments: HBASE-19370-branch-1.patch, HBASE-19370.patch
>
>
> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> Make sure o.a.h.server.RowTooBigException is tagged as deprecated.
> RowTooBigException should derive from DoNotRetryIOException



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15536) Make AsyncFSWAL as our default WAL

2017-11-28 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270150#comment-16270150
 ] 

Anoop Sam John commented on HBASE-15536:


bq.Requires Hadoop-2.8.0 at least. Depends on PBHelperClient added here:
Oh!  But we are still at 2.7.4 
officially.

> Make AsyncFSWAL as our default WAL
> --
>
> Key: HBASE-15536
> URL: https://issues.apache.org/jira/browse/HBASE-15536
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch, 
> HBASE-15536-v3.patch, HBASE-15536-v4.patch, HBASE-15536-v5.patch, 
> HBASE-15536.patch, latesttrunk_asyncWAL_50threads_10cols.jfr, 
> latesttrunk_defaultWAL_50threads_10cols.jfr
>
>
> As it should be predicated on passing basic cluster ITBLL



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19357) Bucket cache no longer L2 for LRU cache

2017-11-28 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270148#comment-16270148
 ] 

stack commented on HBASE-19357:
---

Sounds good.

> Bucket cache no longer L2 for LRU cache
> ---
>
> Key: HBASE-19357
> URL: https://issues.apache.org/jira/browse/HBASE-19357
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19357.patch, HBASE-19357.patch
>
>
> When Bucket cache is used, by default we dont configure it as an L2 cache 
> alone. The default setting is combined mode ON where the data blocks to 
> Bucket cache and index/bloom blocks go to LRU cache. But there is a way to 
> turn this off and make LRU as L1 and Bucket cache as a victim handler for L1. 
> It will be just L2.   
> After the off heap read path optimization Bucket cache is no longer slower 
> compared to L1. We have test results on data sizes from 12 GB.  The Alibaba 
> use case was also with 12 GB and they have observed a ~30% QPS improve over 
> the LRU cache.
> This issue is to remove the option for combined mode = false. So when Bucket 
> cache is in use, data blocks will go to it only and LRU will get only index 
> /meta/bloom blocks.   Bucket cache will no longer be configured as a victim 
> handler for LRU.
> Note : WHen external cache is in use, there only the L1 L2 thing comes. LRU 
> will be L1 and external cache act as its L2. That make full sense.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19357) Bucket cache no longer L2 for LRU cache

2017-11-28 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270143#comment-16270143
 ] 

Anoop Sam John commented on HBASE-19357:


Why the second cache name came is - It might not be BucketCache always. We have 
the external cache also.  Ya that external cache is L2 for LRU cache any way.  
If it is not that much confusing ya for name we can keep it as L1 and L2 only 
as of now?
bq.I can configure the seond cache to be on heap too, IIRC? 
No. Already removed that support
bq.You want to deprecate HCD# setCacheDataInL1 ? Or you figure no need because 
whole class is deprecated?
Ya as class is fully deprecated I did not add. I can add deprecation there too 
in next patch
bq.We still need a victim handler?
I feel not needed. The index blocks only go to L1 now. If there is no space let 
eviction happen. Any way eviction is needed there as we will have compaction 
and some old reads which are no longer active now.. Let them be evicted out 
rather than to L2 disturbing the data cache part.  Any way that will give a 
better idea for admin from the UI whether the evictions are more in L1 or not. 
If more, may be they should increase the L1 size.  We are moving the max of the 
heap size requirements out of heap now.  

> Bucket cache no longer L2 for LRU cache
> ---
>
> Key: HBASE-19357
> URL: https://issues.apache.org/jira/browse/HBASE-19357
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19357.patch, HBASE-19357.patch
>
>
> When Bucket cache is used, by default we dont configure it as an L2 cache 
> alone. The default setting is combined mode ON where the data blocks to 
> Bucket cache and index/bloom blocks go to LRU cache. But there is a way to 
> turn this off and make LRU as L1 and Bucket cache as a victim handler for L1. 
> It will be just L2.   
> After the off heap read path optimization Bucket cache is no longer slower 
> compared to L1. We have test results on data sizes from 12 GB.  The Alibaba 
> use case was also with 12 GB and they have observed a ~30% QPS improve over 
> the LRU cache.
> This issue is to remove the option for combined mode = false. So when Bucket 
> cache is in use, data blocks will go to it only and LRU will get only index 
> /meta/bloom blocks.   Bucket cache will no longer be configured as a victim 
> handler for LRU.
> Note : WHen external cache is in use, there only the L1 L2 thing comes. LRU 
> will be L1 and external cache act as its L2. That make full sense.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18233) We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270142#comment-16270142
 ] 

Hadoop QA commented on HBASE-18233:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1.3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
40s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
28s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
30s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 6s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
 6s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
40s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
22s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
30s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
29s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
4s{color} | {color:red} hbase-server: The patch generated 3 new + 394 unchanged 
- 4 fixed = 397 total (was 398) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  2m 
 0s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
23m 11s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3. 
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
21s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
28s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 74m 13s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}112m 52s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestAtomicOperation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:dca6535 |
| JIRA Issue | HBASE-18233 |
| JIRA Patch URL |

[jira] [Updated] (HBASE-19096) Add RowMutions batch support in AsyncTable

2017-11-28 Thread Jerry He (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry He updated HBASE-19096:
-
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: (was: 2.0.0)
   2.0.0-beta-1
   Status: Resolved  (was: Patch Available)

Pushed to branch-2 and master. Thanks for the review!

> Add RowMutions batch support in AsyncTable
> --
>
> Key: HBASE-19096
> URL: https://issues.apache.org/jira/browse/HBASE-19096
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19096-master-v2.patch, 
> HBASE-19096-master-v3.patch, HBASE-19096-master-v4.patch, 
> HBASE-19096-master.patch
>
>
> Batch support for RowMutations has been added in the Table interface, but is 
> not in AsyncTable. This JIRA will add it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19242) Add MOB compact support for AsyncAdmin

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270102#comment-16270102
 ] 

Hudson commented on HBASE-19242:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4135 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4135/])
HBASE-19242 Add MOB compact support for AsyncAdmin (stack: rev 
f6582400bec7a9b2450437efbfb7139f212e5631)
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java


> Add MOB compact support for AsyncAdmin
> --
>
> Key: HBASE-19242
> URL: https://issues.apache.org/jira/browse/HBASE-19242
> Project: HBase
>  Issue Type: Sub-task
>  Components: Admin, mob
>Reporter: Duo Zhang
>Assignee: Balazs Meszaros
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19242.master.001.patch, 
> HBASE-19242.master.002.patch, HBASE-19242.master.002.patch, 
> HBASE-19242.master.003.patch
>
>
> {code}
>   private CompletableFuture compact(TableName tableName, byte[] 
> columnFamily, boolean major,
>   CompactType compactType) {
> if (CompactType.MOB.equals(compactType)) {
>   // TODO support MOB compact.
>   return failedFuture(new UnsupportedOperationException("MOB compact does 
> not support"));
> }
> {code}
> We need to support it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-15536) Make AsyncFSWAL as our default WAL

2017-11-28 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15536:
--
Release Note: 
Now the default WALProvider is AsyncFSWALProvider, i.e. 'asyncfs'.
If you want to change back to use FSHLog, please add this in hbase-site.xml
{code}

hbase.wal.provider
filesystem

{code}
If you want to use FSHLog with multiwal, please add this in hbase-site.xml
{code}

hbase.wal.regiongrouping.delegate.provider
filesystem

{code}

Requires Hadoop-2.8.0 at least. Depends on PBHelperClient added here:

commit 95f8e93691ad79b77ae9a4a13208c0d7ca405c96
Author: Haohui Mai 
Date:   Sat Aug 22 13:30:19 2015 -0700

HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang 
Liu.

  was:
Now the default WALProvider is AsyncFSWALProvider, i.e. 'asyncfs'.
If you want to change back to use FSHLog, please add this in hbase-site.xml
{code}

hbase.wal.provider
filesystem

{code}
If you want to use FSHLog with multiwal, please add this in hbase-site.xml
{code}

hbase.wal.regiongrouping.delegate.provider
filesystem

{code}


> Make AsyncFSWAL as our default WAL
> --
>
> Key: HBASE-15536
> URL: https://issues.apache.org/jira/browse/HBASE-15536
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch, 
> HBASE-15536-v3.patch, HBASE-15536-v4.patch, HBASE-15536-v5.patch, 
> HBASE-15536.patch, latesttrunk_asyncWAL_50threads_10cols.jfr, 
> latesttrunk_defaultWAL_50threads_10cols.jfr
>
>
> As it should be predicated on passing basic cluster ITBLL



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19000) Group multiple block cache clear requests per server

2017-11-28 Thread Guangxu Cheng (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270093#comment-16270093
 ] 

Guangxu Cheng commented on HBASE-19000:
---

hi, [~zyork]
are you working on this jira? if not, may i?

> Group multiple block cache clear requests per server
> 
>
> Key: HBASE-19000
> URL: https://issues.apache.org/jira/browse/HBASE-19000
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Zach York
>  Labels: rpc
>
> During the review of HBASE-18624, I brought up the following:
> There would be multiple regions on the same server whose block cache is to be 
> cleared.
> Multiple block cache clear requests should be grouped per server to reduce 
> the number of RPC calls.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18233) We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock

2017-11-28 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18233:
--
Attachment: HBASE-18233.branch-1.3.003.patch

> We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock
> -
>
> Key: HBASE-18233
> URL: https://issues.apache.org/jira/browse/HBASE-18233
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.7
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Blocker
> Fix For: 1.3.2, 1.4.1, 1.2.7
>
> Attachments: HBASE-18233-branch-1.2.patch, 
> HBASE-18233-branch-1.2.trivial.patch, HBASE-18233-branch-1.2.v2.patch, 
> HBASE-18233-branch-1.2.v3.patch, HBASE-18233-branch-1.2.v4 (1).patch, 
> HBASE-18233-branch-1.2.v4 (1).patch, HBASE-18233-branch-1.2.v4 (2).patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v5 (1).patch, HBASE-18233-branch-1.2.v5.patch, 
> HBASE-18233-branch-1.2.v5.patch, HBASE-18233-branch-1.2.v6.patch, 
> HBASE-18233.branch-1.2.UT.log, HBASE-18233.branch-1.3.001.patch, 
> HBASE-18233.branch-1.3.001.patch, HBASE-18233.branch-1.3.002.patch, 
> HBASE-18233.branch-1.3.002.patch, HBASE-18233.branch-1.3.003.patch
>
>
> Please refer to the discuss in HBASE-18144
> https://issues.apache.org/jira/browse/HBASE-18144?focusedCommentId=16051701=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16051701



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18233) We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock

2017-11-28 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270094#comment-16270094
 ] 

stack commented on HBASE-18233:
---

1.3.003 Fixes the mock in the test. Was stack overflowing.

> We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock
> -
>
> Key: HBASE-18233
> URL: https://issues.apache.org/jira/browse/HBASE-18233
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.7
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Blocker
> Fix For: 1.3.2, 1.4.1, 1.2.7
>
> Attachments: HBASE-18233-branch-1.2.patch, 
> HBASE-18233-branch-1.2.trivial.patch, HBASE-18233-branch-1.2.v2.patch, 
> HBASE-18233-branch-1.2.v3.patch, HBASE-18233-branch-1.2.v4 (1).patch, 
> HBASE-18233-branch-1.2.v4 (1).patch, HBASE-18233-branch-1.2.v4 (2).patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v5 (1).patch, HBASE-18233-branch-1.2.v5.patch, 
> HBASE-18233-branch-1.2.v5.patch, HBASE-18233-branch-1.2.v6.patch, 
> HBASE-18233.branch-1.2.UT.log, HBASE-18233.branch-1.3.001.patch, 
> HBASE-18233.branch-1.3.001.patch, HBASE-18233.branch-1.3.002.patch, 
> HBASE-18233.branch-1.3.002.patch, HBASE-18233.branch-1.3.003.patch
>
>
> Please refer to the discuss in HBASE-18144
> https://issues.apache.org/jira/browse/HBASE-18144?focusedCommentId=16051701=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16051701



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HBASE-18440) ITs and Actions modify immutable TableDescriptors

2017-11-28 Thread Mike Drob (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob reassigned HBASE-18440:
-

Assignee: (was: Mike Drob)

unassigning this since i don't have time in the short term to handle it. if 
somebody else is available to pick it up in the meanwhile, go for it

> ITs and Actions modify immutable TableDescriptors
> -
>
> Key: HBASE-18440
> URL: https://issues.apache.org/jira/browse/HBASE-18440
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Mike Drob
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18440.patch, HBASE-18440.v2.patch, 
> HBASE-18440.v3.patch, HBASE-18440.v3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18440) ITs and Actions modify immutable TableDescriptors

2017-11-28 Thread Mike Drob (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-18440:
--
Status: Open  (was: Patch Available)

> ITs and Actions modify immutable TableDescriptors
> -
>
> Key: HBASE-18440
> URL: https://issues.apache.org/jira/browse/HBASE-18440
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Mike Drob
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18440.patch, HBASE-18440.v2.patch, 
> HBASE-18440.v3.patch, HBASE-18440.v3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19368) [nightly] Make xml test non-voting in branch-1.2

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270065#comment-16270065
 ] 

Hudson commented on HBASE-19368:


FAILURE: Integrated in Jenkins build HBase-1.5 #171 (See 
[https://builds.apache.org/job/HBase-1.5/171/])
HBASE-19368 [nightly] Make xml test non-voting in branch-1.2 (stack: rev 
4fe4d755ce5007ab9160995355a510cb3062dee6)
* (edit) dev-support/Jenkinsfile


> [nightly] Make xml test non-voting in branch-1.2
> 
>
> Key: HBASE-19368
> URL: https://issues.apache.org/jira/browse/HBASE-19368
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.4.0, 1.3.2, 1.2.7
>
> Attachments: HBASE-19368.branch-1.2.001.patch
>
>
>  HBASE-19354 moved branch-1.2 to use azul openjdk. Turns out zulu-7 does does 
> not have a js script engine so jrunscript fails which causes the xml check to 
> fail with
> 19:55:59 |  -1  |xml  |   0m  7s   | The source tree has 60 
> ill-formed XML 
> 19:55:59 |  | || file(s).
> The xml.txt file has this in it:
> script engine for language js can not be found
> If I run jrunscript from zulu-7 on cmd-line, it does this:
> $ $JAVA_HOME/bin/jrunscript
> script engine for language js can not be found
> For now, make the xml check non-voting in branch-1.2 so can see if we can get 
> the nightly running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19242) Add MOB compact support for AsyncAdmin

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270060#comment-16270060
 ] 

Hudson commented on HBASE-19242:


FAILURE: Integrated in Jenkins build HBase-2.0 #933 (See 
[https://builds.apache.org/job/HBase-2.0/933/])
HBASE-19242 Add MOB compact support for AsyncAdmin (stack: rev 
e946d9d841a72d41c0c7458667c7fedc495bd306)
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java


> Add MOB compact support for AsyncAdmin
> --
>
> Key: HBASE-19242
> URL: https://issues.apache.org/jira/browse/HBASE-19242
> Project: HBase
>  Issue Type: Sub-task
>  Components: Admin, mob
>Reporter: Duo Zhang
>Assignee: Balazs Meszaros
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19242.master.001.patch, 
> HBASE-19242.master.002.patch, HBASE-19242.master.002.patch, 
> HBASE-19242.master.003.patch
>
>
> {code}
>   private CompletableFuture compact(TableName tableName, byte[] 
> columnFamily, boolean major,
>   CompactType compactType) {
> if (CompactType.MOB.equals(compactType)) {
>   // TODO support MOB compact.
>   return failedFuture(new UnsupportedOperationException("MOB compact does 
> not support"));
> }
> {code}
> We need to support it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19367) Refactoring in RegionStates, and RSProcedureDispatcher

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270058#comment-16270058
 ] 

Hadoop QA commented on HBASE-19367:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  2m 
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 7 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  7m 
32s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
29s{color} | {color:green} hbase-client: The patch generated 0 new + 99 
unchanged - 3 fixed = 99 total (was 102) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} hbase-zookeeper: The patch generated 0 new + 61 
unchanged - 1 fixed = 61 total (was 62) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} The patch hbase-procedure passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
4s{color} | {color:red} hbase-server: The patch generated 3 new + 412 unchanged 
- 4 fixed = 415 total (was 416) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
16s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} The patch hbase-rsgroup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
46s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
53m 17s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
37s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
21s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green}

[jira] [Commented] (HBASE-19368) [nightly] Make xml test non-voting in branch-1.2

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270057#comment-16270057
 ] 

Hudson commented on HBASE-19368:


FAILURE: Integrated in Jenkins build HBase-1.4 #1029 (See 
[https://builds.apache.org/job/HBase-1.4/1029/])
HBASE-19368 [nightly] Make xml test non-voting in branch-1.2 (stack: rev 
5f0219b86de69cdcf42e0e9f909bc5fd10757635)
* (edit) dev-support/Jenkinsfile


> [nightly] Make xml test non-voting in branch-1.2
> 
>
> Key: HBASE-19368
> URL: https://issues.apache.org/jira/browse/HBASE-19368
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.4.0, 1.3.2, 1.2.7
>
> Attachments: HBASE-19368.branch-1.2.001.patch
>
>
>  HBASE-19354 moved branch-1.2 to use azul openjdk. Turns out zulu-7 does does 
> not have a js script engine so jrunscript fails which causes the xml check to 
> fail with
> 19:55:59 |  -1  |xml  |   0m  7s   | The source tree has 60 
> ill-formed XML 
> 19:55:59 |  | || file(s).
> The xml.txt file has this in it:
> script engine for language js can not be found
> If I run jrunscript from zulu-7 on cmd-line, it does this:
> $ $JAVA_HOME/bin/jrunscript
> script engine for language js can not be found
> For now, make the xml check non-voting in branch-1.2 so can see if we can get 
> the nightly running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17049) Do not issue sync request when there are still entries in ringbuffer

2017-11-28 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270043#comment-16270043
 ] 

stack commented on HBASE-17049:
---

I'm running a little compare now too... will be back.

> Do not issue sync request when there are still entries in ringbuffer
> 
>
> Key: HBASE-17049
> URL: https://issues.apache.org/jira/browse/HBASE-17049
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17049.patch, delay-sync.patch
>
>
> https://issues.apache.org/jira/browse/HBASE-16890?focusedCommentId=15647590=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15647590



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18233) We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock

2017-11-28 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270041#comment-16270041
 ] 

stack commented on HBASE-18233:
---

Failure looks related. Looking.


> We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock
> -
>
> Key: HBASE-18233
> URL: https://issues.apache.org/jira/browse/HBASE-18233
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.7
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Blocker
> Fix For: 1.3.2, 1.4.1, 1.2.7
>
> Attachments: HBASE-18233-branch-1.2.patch, 
> HBASE-18233-branch-1.2.trivial.patch, HBASE-18233-branch-1.2.v2.patch, 
> HBASE-18233-branch-1.2.v3.patch, HBASE-18233-branch-1.2.v4 (1).patch, 
> HBASE-18233-branch-1.2.v4 (1).patch, HBASE-18233-branch-1.2.v4 (2).patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v5 (1).patch, HBASE-18233-branch-1.2.v5.patch, 
> HBASE-18233-branch-1.2.v5.patch, HBASE-18233-branch-1.2.v6.patch, 
> HBASE-18233.branch-1.2.UT.log, HBASE-18233.branch-1.3.001.patch, 
> HBASE-18233.branch-1.3.001.patch, HBASE-18233.branch-1.3.002.patch, 
> HBASE-18233.branch-1.3.002.patch
>
>
> Please refer to the discuss in HBASE-18144
> https://issues.apache.org/jira/browse/HBASE-18144?focusedCommentId=16051701=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16051701



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18233) We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock

2017-11-28 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18233:
--
Attachment: HBASE-18233.branch-1.3.002.patch

> We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock
> -
>
> Key: HBASE-18233
> URL: https://issues.apache.org/jira/browse/HBASE-18233
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.7
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Blocker
> Fix For: 1.3.2, 1.4.1, 1.2.7
>
> Attachments: HBASE-18233-branch-1.2.patch, 
> HBASE-18233-branch-1.2.trivial.patch, HBASE-18233-branch-1.2.v2.patch, 
> HBASE-18233-branch-1.2.v3.patch, HBASE-18233-branch-1.2.v4 (1).patch, 
> HBASE-18233-branch-1.2.v4 (1).patch, HBASE-18233-branch-1.2.v4 (2).patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, 
> HBASE-18233-branch-1.2.v5 (1).patch, HBASE-18233-branch-1.2.v5.patch, 
> HBASE-18233-branch-1.2.v5.patch, HBASE-18233-branch-1.2.v6.patch, 
> HBASE-18233.branch-1.2.UT.log, HBASE-18233.branch-1.3.001.patch, 
> HBASE-18233.branch-1.3.001.patch, HBASE-18233.branch-1.3.002.patch, 
> HBASE-18233.branch-1.3.002.patch
>
>
> Please refer to the discuss in HBASE-18144
> https://issues.apache.org/jira/browse/HBASE-18144?focusedCommentId=16051701=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16051701



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding

2017-11-28 Thread Alex Leblang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270037#comment-16270037
 ] 

Alex Leblang commented on HBASE-19369:
--

Yes, I will try to post a patch tomorrow 

> HBase Should use Builder Pattern to Create Log Files while using WAL on 
> Erasure Coding
> --
>
> Key: HBASE-19369
> URL: https://issues.apache.org/jira/browse/HBASE-19369
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Alex Leblang
>Assignee: Alex Leblang
>
> Right now an HBase instance using the WAL won't function properly in an 
> Erasure Coded environment. We should change the following line to use the 
> hdfs.DistributedFileSystem builder pattern 
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19344) improve asyncWAL by using Independent thread for netty #IO in FanOutOneBlockAsyncDFSOutput

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270002#comment-16270002
 ] 

Duo Zhang commented on HBASE-19344:
---

Fix a bug found by [~chancelq].

https://issues.apache.org/jira/browse/HBASE-19344

Please try this branch also [~ram_krish] to see if there is improvement?

Thanks.

> improve asyncWAL by using Independent thread for netty #IO in 
> FanOutOneBlockAsyncDFSOutput 
> ---
>
> Key: HBASE-19344
> URL: https://issues.apache.org/jira/browse/HBASE-19344
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Affects Versions: 2.0.0-beta-1
>Reporter: Chance Li
>Assignee: Chance Li
> Fix For: 2.0.0
>
> Attachments: HBASE-19344-branch2.patch, 
> HBASE-19344-branch2.patch.2.POC, wal-1-test-result.png, 
> wal-8-test-result.png, ycsb_result_apache20_async_wal.pdf
>
>
> The logic now is that the netty #IO thread and asyncWal's thread are the same 
> one.
> Improvement proposal:
> 1, Split into two.
> 2, All multiWal share the netty #IO thread pool. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18601) Update Htrace to 4.2

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269991#comment-16269991
 ] 

Duo Zhang commented on HBASE-18601:
---

Belated but what does this code mean?

{code}
  @Override
  public void sync(long txid) throws IOException {
if (highestSyncedTxid.get() >= txid) {
  return;
}
try (TraceScope scope = TraceUtil.createTrace("AsyncFSWAL.sync")) {
  // here we do not use ring buffer sequence as txid
  long sequence = waitingConsumePayloads.next();
  SyncFuture future = null;
  try {
if (scope != null) {
  future = getSyncFuture(txid, scope.getSpan());
  RingBufferTruck truck = waitingConsumePayloads.get(sequence);
  truck.load(future);
}
  } finally {
waitingConsumePayloads.publish(sequence);
  }
  if (shouldScheduleConsumer()) {
consumeExecutor.execute(consumer);
  }
  // TODO handle htrace API change, see HBASE-18895
  // scope = Trace.continueSpan(blockOnSync(future));
  if (future != null) {
blockOnSync(future);
  }
}
  }
{code}

We will give up syncing on WAL if there is no TraceScope provided?

> Update Htrace to 4.2
> 
>
> Key: HBASE-18601
> URL: https://issues.apache.org/jira/browse/HBASE-18601
> Project: HBase
>  Issue Type: Improvement
>  Components: dependencies, tracing
>Affects Versions: 2.0.0, 3.0.0
>Reporter: Tamas Penzes
>Assignee: Balazs Meszaros
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18601.master.001.patch, 
> HBASE-18601.master.002.patch, HBASE-18601.master.003 (3).patch, 
> HBASE-18601.master.003.patch, HBASE-18601.master.004.patch, 
> HBASE-18601.master.004.patch, HBASE-18601.master.005.patch, 
> HBASE-18601.master.006.patch, HBASE-18601.master.006.patch, 
> HBASE-18601.master.007.patch, HBASE-18601.master.007.patch, 
> HBASE-18601.master.007.patch, HBASE-18601.master.008.patch, 
> HBASE-18601.master.009.patch, HBASE-18601.master.009.patch, 
> HBASE-18601.master.010.patch, HBASE-18601.master.010.patch, 
> HBASE-18601.master.011.patch, HBASE-18601.master.012.patch, 
> HBASE-18601.master.013.patch, HBASE-18601.master.014.patch, 
> HBASE-18601.master.014.patch, HBASE-18601.master.015.patch, 
> HBASE-18601.master.016.patch
>
>
> HTrace is not perfectly integrated into HBase, the version 3.2.0 is buggy, 
> the upgrade to 4.x is not trivial and would take time. It might not worth to 
> keep it in this state, so would be better to remove it.
> Of course it doesn't mean tracing would be useless, just that in this form 
> the use of HTrace 3.2 might not add any value to the project and fixing it 
> would be far too much effort.
> -
> Based on the decision of the community we keep htrace now and update version



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19290) Reduce zk request when doing split log

2017-11-28 Thread binlijin (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269984#comment-16269984
 ] 

binlijin commented on HBASE-19290:
--

HBASE-19290.master.006.patch fix test failed.

> Reduce zk request when doing split log
> --
>
> Key: HBASE-19290
> URL: https://issues.apache.org/jira/browse/HBASE-19290
> Project: HBase
>  Issue Type: Improvement
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-19290.master.001.patch, 
> HBASE-19290.master.002.patch, HBASE-19290.master.003.patch, 
> HBASE-19290.master.004.patch, HBASE-19290.master.005.patch, 
> HBASE-19290.master.006.patch
>
>
> We observe once the cluster has 1000+ nodes and when hundreds of nodes abort 
> and doing split log, the split is very very slow, and we find the 
> regionserver and master wait on the zookeeper response, so we need to reduce 
> zookeeper request and pressure for big cluster.
> (1) Reduce request to rsZNode, every time calculateAvailableSplitters will 
> get rsZNode's children from zookeeper, when cluster is huge, this is heavy. 
> This patch reduce the request. 
> (2) When the regionserver has max split tasks running, it may still trying to 
> grab task and issue zookeeper request, we should sleep and wait until we can 
> grab tasks again.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19290) Reduce zk request when doing split log

2017-11-28 Thread binlijin (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

binlijin updated HBASE-19290:
-
Attachment: HBASE-19290.master.006.patch

> Reduce zk request when doing split log
> --
>
> Key: HBASE-19290
> URL: https://issues.apache.org/jira/browse/HBASE-19290
> Project: HBase
>  Issue Type: Improvement
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-19290.master.001.patch, 
> HBASE-19290.master.002.patch, HBASE-19290.master.003.patch, 
> HBASE-19290.master.004.patch, HBASE-19290.master.005.patch, 
> HBASE-19290.master.006.patch
>
>
> We observe once the cluster has 1000+ nodes and when hundreds of nodes abort 
> and doing split log, the split is very very slow, and we find the 
> regionserver and master wait on the zookeeper response, so we need to reduce 
> zookeeper request and pressure for big cluster.
> (1) Reduce request to rsZNode, every time calculateAvailableSplitters will 
> get rsZNode's children from zookeeper, when cluster is huge, this is heavy. 
> This patch reduce the request. 
> (2) When the regionserver has max split tasks running, it may still trying to 
> grab task and issue zookeeper request, we should sleep and wait until we can 
> grab tasks again.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17049) Do not issue sync request when there are still entries in ringbuffer

2017-11-28 Thread Duo Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-17049:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to master and branch-2.

Thanks all for reviewing.

> Do not issue sync request when there are still entries in ringbuffer
> 
>
> Key: HBASE-17049
> URL: https://issues.apache.org/jira/browse/HBASE-17049
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17049.patch, delay-sync.patch
>
>
> https://issues.apache.org/jira/browse/HBASE-16890?focusedCommentId=15647590=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15647590



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19367) Refactoring in RegionStates, and RSProcedureDispatcher

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269945#comment-16269945
 ] 

Hadoop QA commented on HBASE-19367:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 7 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  7m 
37s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  2m 
14s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
40s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 40s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
29s{color} | {color:green} hbase-client: The patch generated 0 new + 99 
unchanged - 3 fixed = 99 total (was 102) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} hbase-zookeeper: The patch generated 0 new + 61 
unchanged - 1 fixed = 61 total (was 62) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} The patch hbase-procedure passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m 
13s{color} | {color:red} hbase-server: The patch generated 3 new + 419 
unchanged - 4 fixed = 422 total (was 423) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} The patch hbase-rsgroup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red}  3m 
39s{color} | {color:red} patch has 14 errors when building our shaded 
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  5m 
31s{color} | {color:red} The patch causes 15 errors with Hadoop v2.6.1. {color} 
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  7m 
23s{color} | {color:red} The patch causes 15 errors with Hadoop v2.6.2. {color} 
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  9m 
14s{color} | {color:red} The patch causes 15 errors with Hadoop v2.6.3. {color} 
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 11m  
7s{color} | {color:red} The patch causes 15 errors with Hadoop v2.6.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 13m  
3s{color} | {color:red} The patch causes 15 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 14m 
57s{color} | {color:red} The patch causes 15 errors with Hadoop v2.7.1. {color} 
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 16m 
45s{color} | {color:red}

[jira] [Commented] (HBASE-19163) "Maximum lock count exceeded" from region server's batch processing

2017-11-28 Thread huaxiang sun (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269938#comment-16269938
 ] 

huaxiang sun commented on HBASE-19163:
--

TestFlushSnapshotFromClient failure is similar and I uploaded a new patch. The 
rest two failures passed locally in my laptop.

> "Maximum lock count exceeded" from region server's batch processing
> ---
>
> Key: HBASE-19163
> URL: https://issues.apache.org/jira/browse/HBASE-19163
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 3.0.0, 1.2.7, 2.0.0-alpha-3
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Attachments: HBASE-19163-master-v001.patch, 
> HBASE-19163.master.001.patch, HBASE-19163.master.002.patch, 
> HBASE-19163.master.004.patch, HBASE-19163.master.005.patch, 
> HBASE-19163.master.006.patch, HBASE-19163.master.007.patch, 
> HBASE-19163.master.008.patch, HBASE-19163.master.009.patch, unittest-case.diff
>
>
> In one of use cases, we found the following exception and replication is 
> stuck.
> {code}
> 2017-10-25 19:41:17,199 WARN  [hconnection-0x28db294f-shared--pool4-t936] 
> client.AsyncProcess: #3, table=foo, attempt=5/5 failed=262836ops, last 
> exception: java.io.IOException: java.io.IOException: Maximum lock count 
> exceeded
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2215)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
> Caused by: java.lang.Error: Maximum lock count exceeded
> at 
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:528)
> at 
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:488)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1327)
> at 
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:871)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:5163)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3018)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2877)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2819)
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:753)
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:715)
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2148)
> at 
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
> ... 3 more
> {code}
> While we are still examining the data pattern, it is sure that there are too 
> many mutations in the batch against the same row, this exceeds the maximum 
> 64k shared lock count and it throws an error and failed the whole batch.
> There are two approaches to solve this issue.
> 1). Let's say there are mutations against the same row in the batch, we just 
> need to acquire the lock once for the same row vs to acquire the lock for 
> each mutation.
> 2). We catch the error and start to process whatever it gets and loop back.
> With HBASE-17924, approach 1 seems easy to implement now. 
> Create the jira and will post update/patch when investigation moving forward.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19359) Revisit the default config of hbase client retries number

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269936#comment-16269936
 ] 

Hadoop QA commented on HBASE-19359:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  2m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
57s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
21s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
49m 51s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  1m 
57s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
38s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m 
14s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
44s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}167m 44s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19359 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12899703/HBASE-19359.master.001.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  xml  |
| uname | Linux 84be6e6890a8 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |

[jira] [Updated] (HBASE-19163) "Maximum lock count exceeded" from region server's batch processing

2017-11-28 Thread huaxiang sun (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

huaxiang sun updated HBASE-19163:
-
Attachment: HBASE-19163.master.009.patch

> "Maximum lock count exceeded" from region server's batch processing
> ---
>
> Key: HBASE-19163
> URL: https://issues.apache.org/jira/browse/HBASE-19163
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 3.0.0, 1.2.7, 2.0.0-alpha-3
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Attachments: HBASE-19163-master-v001.patch, 
> HBASE-19163.master.001.patch, HBASE-19163.master.002.patch, 
> HBASE-19163.master.004.patch, HBASE-19163.master.005.patch, 
> HBASE-19163.master.006.patch, HBASE-19163.master.007.patch, 
> HBASE-19163.master.008.patch, HBASE-19163.master.009.patch, unittest-case.diff
>
>
> In one of use cases, we found the following exception and replication is 
> stuck.
> {code}
> 2017-10-25 19:41:17,199 WARN  [hconnection-0x28db294f-shared--pool4-t936] 
> client.AsyncProcess: #3, table=foo, attempt=5/5 failed=262836ops, last 
> exception: java.io.IOException: java.io.IOException: Maximum lock count 
> exceeded
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2215)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
> Caused by: java.lang.Error: Maximum lock count exceeded
> at 
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:528)
> at 
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:488)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1327)
> at 
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:871)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:5163)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3018)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2877)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2819)
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:753)
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:715)
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2148)
> at 
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
> ... 3 more
> {code}
> While we are still examining the data pattern, it is sure that there are too 
> many mutations in the batch against the same row, this exceeds the maximum 
> 64k shared lock count and it throws an error and failed the whole batch.
> There are two approaches to solve this issue.
> 1). Let's say there are mutations against the same row in the batch, we just 
> need to acquire the lock once for the same row vs to acquire the lock for 
> each mutation.
> 2). We catch the error and start to process whatever it gets and loop back.
> With HBASE-17924, approach 1 seems easy to implement now. 
> Create the jira and will post update/patch when investigation moving forward.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19336) Improve rsgroup to allow assign all tables within a specified namespace by only writing namespace

2017-11-28 Thread Guanghao Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269927#comment-16269927
 ] 

Guanghao Zhang commented on HBASE-19336:


Please fix the rubocop result which introduced by this patch. +1 for else.

> Improve rsgroup to allow assign all tables within a specified namespace by 
> only writing namespace
> -
>
> Key: HBASE-19336
> URL: https://issues.apache.org/jira/browse/HBASE-19336
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Affects Versions: 2.0.0-alpha-4
>Reporter: xinxin fan
>Assignee: xinxin fan
> Attachments: HBASE-19336-master-V2.patch, 
> HBASE-19336-master-V3.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master-V4.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master.patch
>
>
> Currently, use can only assign tables within a namespace from one group to 
> another by writing all table names in move_tables_rsgroup command. Allowing 
> to assign all tables within a specifed namespace by only wirting namespace 
> name is useful.
> Usage as follows:
> {code:java}
> hbase(main):055:0> move_namespaces_rsgroup 'dest_rsgroup',['ns1']
> Took 2.2211 seconds
> {code}
> {code:java}
> hbase(main):051:0* move_servers_namespaces_rsgroup 
> 'dest_rsgroup',['hbase39.lt.163.org:60020'],['ns1','ns2']
> Took 15.3710 seconds 
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19336) Improve rsgroup to allow assign all tables within a specified namespace by only writing namespace

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269916#comment-16269916
 ] 

Hadoop QA commented on HBASE-19336:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  9m 
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  9m 
 7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red}  0m 
14s{color} | {color:red} The patch generated 31 new + 44 unchanged - 1 fixed = 
75 total (was 45) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red}  0m  
7s{color} | {color:red} The patch generated 17 new + 43 unchanged - 2 fixed = 
60 total (was 45) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  8m 
13s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
21s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 15s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19336 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12899551/HBASE-19336-master-V4.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  rubocop  ruby_lint  |
| uname | Linux e2734f5b2ae5 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / f6582400be |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| rubocop | v0.51.0 |
| rubocop | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10089/artifact/patchprocess/diff-patch-rubocop.txt
 |
| ruby-lint | v2.3.1 |
| ruby-lint | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10089/artifact/patchprocess/diff-patch-ruby-lint.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10089/testReport/ |
| modules | C: hbase-shell U: hbase-shell |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10089/console |
| Powered by | Apache Yetus 0.6.0   http://yetus.apache.org |


This message was automatically generated.



> Improve rsgroup to allow assign all tables within a specified namespace by 
> only writing namespace
> -
>
> Key: HBASE-19336
> URL: https://issues.apache.org/jira/browse/HBASE-19336
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Affects Versions: 2.0.0-alpha-4
>Reporter: xinxin fan
>Assignee: xinxin fan
> Attachments: HBASE-19336-master-V2.patch, 
> HBASE-19336-master-V3.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master-V4.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master.patch
>
>
> Currently, use can only assign tables within a namespace from one group to 
> another by writing all table names in move_tables_rsgroup command. Allowing 
> to assign all tables within a specifed namespace by only wirting namespace 
> name is useful.
> Usage as follows:
> {code:java}
> hbase(main):055:0> move_namespaces_rsgroup 'dest_rsgroup',['ns1']
> Took 2.2211 seconds
> {code}
> {code:java}
> hbase(main):051:0*

[jira] [Updated] (HBASE-19252) Move the transform logic of FilterList into transformCell() method to avoid extra ref to question cell

2017-11-28 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-19252:
-
Attachment: HBASE-19252-branch-1.4.v1.patch

> Move the transform logic of FilterList into transformCell() method to avoid 
> extra ref to question cell 
> ---
>
> Key: HBASE-19252
> URL: https://issues.apache.org/jira/browse/HBASE-19252
> Project: HBase
>  Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Fix For: 3.0.0, 1.4.1, 2.0.0-beta-1
>
> Attachments: HBASE-19252-branch-1.4.v1.patch, 
> HBASE-19252-branch-1.4.v1.patch, HBASE-19252-branch-1.4.v1.patch, 
> HBASE-19252.v1.patch, HBASE-19252.v2.patch, HBASE-19252.v3.patch, 
> HBASE-19252.v4.patch
>
>
> As [~anoop.hbase] and I discussed,  we can implement the filterKeyValue () 
> and transformCell() methods as following  to avoid saving transformedCell & 
> referenceCell state in FilterList, and we can avoid the costly cell clone. 
> {code}
> ReturnCode filterKeyValue(Cell c){
>   ReturnCode rc = null;
>   for(Filter filter: sub-filters){
>   // ...
>   rc = mergeReturnCode(rc, filter.filterKeyValue(c));
>   // ... 
>   }
>   return rc;
> }
> Cell transformCell(Cell c) throws IOException {
>   Cell transformed = c; 
>   for(Filter filter: sub-filters){
>   if(filter.filterKeyValue(c) is INCLUDE*) { //  > line#1
>   transformed = filter.transformCell(transformed);
> 
>   }
>   }
>   return transformed; 
> }
> {code}
> For line #1,  we need to remember the return code of the sub-filter for its 
> filterKeyValue().  because only INCLUDE*  ReturnCode,   we need to 
> transformCell for sub-filter.  
> A new boolean array will be introduced in FilterList.  and the cost of 
> maintaining  the boolean array will be less than  the cost of maintaining the 
> two ref of question cell. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-19252) Move the transform logic of FilterList into transformCell() method to avoid extra ref to question cell

2017-11-28 Thread Zheng Hu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269912#comment-16269912
 ] 

Zheng Hu edited comment on HBASE-19252 at 11/29/17 1:49 AM:


Previous Hadoop QA is timeout ,   Re-trigger again. 

https://builds.apache.org/job/PreCommit-HBASE-Build/10062/


was (Author: openinx):
Previous Hadoop QA is timeout ,   Re-trigger again. 

> Move the transform logic of FilterList into transformCell() method to avoid 
> extra ref to question cell 
> ---
>
> Key: HBASE-19252
> URL: https://issues.apache.org/jira/browse/HBASE-19252
> Project: HBase
>  Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Fix For: 3.0.0, 1.4.1, 2.0.0-beta-1
>
> Attachments: HBASE-19252-branch-1.4.v1.patch, 
> HBASE-19252-branch-1.4.v1.patch, HBASE-19252-branch-1.4.v1.patch, 
> HBASE-19252.v1.patch, HBASE-19252.v2.patch, HBASE-19252.v3.patch, 
> HBASE-19252.v4.patch
>
>
> As [~anoop.hbase] and I discussed,  we can implement the filterKeyValue () 
> and transformCell() methods as following  to avoid saving transformedCell & 
> referenceCell state in FilterList, and we can avoid the costly cell clone. 
> {code}
> ReturnCode filterKeyValue(Cell c){
>   ReturnCode rc = null;
>   for(Filter filter: sub-filters){
>   // ...
>   rc = mergeReturnCode(rc, filter.filterKeyValue(c));
>   // ... 
>   }
>   return rc;
> }
> Cell transformCell(Cell c) throws IOException {
>   Cell transformed = c; 
>   for(Filter filter: sub-filters){
>   if(filter.filterKeyValue(c) is INCLUDE*) { //  > line#1
>   transformed = filter.transformCell(transformed);
> 
>   }
>   }
>   return transformed; 
> }
> {code}
> For line #1,  we need to remember the return code of the sub-filter for its 
> filterKeyValue().  because only INCLUDE*  ReturnCode,   we need to 
> transformCell for sub-filter.  
> A new boolean array will be introduced in FilterList.  and the cost of 
> maintaining  the boolean array will be less than  the cost of maintaining the 
> two ref of question cell. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17049) Do not issue sync request when there are still entries in ringbuffer

2017-11-28 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269914#comment-16269914
 ] 

Duo Zhang commented on HBASE-17049:
---

OK, let me commit then.

Thanks [~zghaobac] And [~ram_krish] for confirming the numbers.

> Do not issue sync request when there are still entries in ringbuffer
> 
>
> Key: HBASE-17049
> URL: https://issues.apache.org/jira/browse/HBASE-17049
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17049.patch, delay-sync.patch
>
>
> https://issues.apache.org/jira/browse/HBASE-16890?focusedCommentId=15647590=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15647590



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19336) Improve rsgroup to allow assign all tables within a specified namespace by only writing namespace

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269911#comment-16269911
 ] 

Hadoop QA commented on HBASE-19336:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red}  0m 
11s{color} | {color:red} The patch generated 31 new + 44 unchanged - 1 fixed = 
75 total (was 45) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red}  0m  
5s{color} | {color:red} The patch generated 17 new + 43 unchanged - 2 fixed = 
60 total (was 45) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  7m 
12s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 9s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 41s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19336 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12899727/HBASE-19336-master-V4.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  rubocop  ruby_lint  |
| uname | Linux 3db1ce216d5e 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / f6582400be |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| rubocop | v0.51.0 |
| rubocop | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10091/artifact/patchprocess/diff-patch-rubocop.txt
 |
| ruby-lint | v2.3.1 |
| ruby-lint | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10091/artifact/patchprocess/diff-patch-ruby-lint.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10091/testReport/ |
| modules | C: hbase-shell U: hbase-shell |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10091/console |
| Powered by | Apache Yetus 0.6.0   http://yetus.apache.org |


This message was automatically generated.



> Improve rsgroup to allow assign all tables within a specified namespace by 
> only writing namespace
> -
>
> Key: HBASE-19336
> URL: https://issues.apache.org/jira/browse/HBASE-19336
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Affects Versions: 2.0.0-alpha-4
>Reporter: xinxin fan
>Assignee: xinxin fan
> Attachments: HBASE-19336-master-V2.patch, 
> HBASE-19336-master-V3.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master-V4.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master.patch
>
>
> Currently, use can only assign tables within a namespace from one group to 
> another by writing all table names in move_tables_rsgroup command. Allowing 
> to assign all tables within a specifed namespace by only wirting namespace 
> name is useful.
> Usage as follows:
> {code:java}
> hbase(main):055:0> move_namespaces_rsgroup 'dest_rsgroup',['ns1']
> Took 2.2211 seconds
> {code}
> {code:java}
> hbase(main):051:0*

[jira] [Commented] (HBASE-19351) Deprecated is missing in Table implementations

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269909#comment-16269909
 ] 

Hudson commented on HBASE-19351:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4134 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4134/])
HBASE-19351 Deprecated is missing in Table implementations (stack: rev 
b5a01685f4aff0d97ec53a6eca26bd37dd9d9037)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReadRequestMetrics.java
* (edit) 
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/RegionAsTable.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java


> Deprecated is missing in Table implementations
> --
>
> Key: HBASE-19351
> URL: https://issues.apache.org/jira/browse/HBASE-19351
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19351.master.001.patch
>
>
> Table interface has some deprecated methods and the implementations do not 
> have it so those methods are visible as non-deprecated.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19252) Move the transform logic of FilterList into transformCell() method to avoid extra ref to question cell

2017-11-28 Thread Zheng Hu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269912#comment-16269912
 ] 

Zheng Hu commented on HBASE-19252:
--

Previous Hadoop QA is timeout ,   Re-trigger again. 

> Move the transform logic of FilterList into transformCell() method to avoid 
> extra ref to question cell 
> ---
>
> Key: HBASE-19252
> URL: https://issues.apache.org/jira/browse/HBASE-19252
> Project: HBase
>  Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
> Fix For: 3.0.0, 1.4.1, 2.0.0-beta-1
>
> Attachments: HBASE-19252-branch-1.4.v1.patch, 
> HBASE-19252-branch-1.4.v1.patch, HBASE-19252.v1.patch, HBASE-19252.v2.patch, 
> HBASE-19252.v3.patch, HBASE-19252.v4.patch
>
>
> As [~anoop.hbase] and I discussed,  we can implement the filterKeyValue () 
> and transformCell() methods as following  to avoid saving transformedCell & 
> referenceCell state in FilterList, and we can avoid the costly cell clone. 
> {code}
> ReturnCode filterKeyValue(Cell c){
>   ReturnCode rc = null;
>   for(Filter filter: sub-filters){
>   // ...
>   rc = mergeReturnCode(rc, filter.filterKeyValue(c));
>   // ... 
>   }
>   return rc;
> }
> Cell transformCell(Cell c) throws IOException {
>   Cell transformed = c; 
>   for(Filter filter: sub-filters){
>   if(filter.filterKeyValue(c) is INCLUDE*) { //  > line#1
>   transformed = filter.transformCell(transformed);
> 
>   }
>   }
>   return transformed; 
> }
> {code}
> For line #1,  we need to remember the return code of the sub-filter for its 
> filterKeyValue().  because only INCLUDE*  ReturnCode,   we need to 
> transformCell for sub-filter.  
> A new boolean array will be introduced in FilterList.  and the cost of 
> maintaining  the boolean array will be less than  the cost of maintaining the 
> two ref of question cell. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19363) Tests under TestCheckAndMutate are identical

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269910#comment-16269910
 ] 

Hudson commented on HBASE-19363:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4134 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4134/])
HBASE-19363 Tests under TestCheckAndMutate are identical (stack: rev 
520b9efc2de34b3b02ee86bad7ccb6e169788fc1)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCheckAndMutate.java


> Tests under TestCheckAndMutate are identical
> 
>
> Key: HBASE-19363
> URL: https://issues.apache.org/jira/browse/HBASE-19363
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19363.master.001.patch
>
>
> TestCheckAndMutate has 2 tests and the content is the same in both.
> Originally testCheckAndMutate and 
> testCheckAndMutateUsingNewComparisonOperatorInstead was different in calling 
> table.checkAndMutate with CompareFilter.CompareOp.EQUAL vs. 
> CompareOperator.EQUAL.
> Since the implementations in HTable use the string value of the enums these 
> were were identical before.
> Remove one of the tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19096) Add RowMutions batch support in AsyncTable

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269895#comment-16269895
 ] 

Hadoop QA commented on HBASE-19096:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
35s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
24s{color} | {color:green} hbase-client: The patch generated 0 new + 126 
unchanged - 4 fixed = 126 total (was 130) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
54s{color} | {color:green} The patch hbase-server passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
16s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
47m 18s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
46s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 86m 
44s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
34s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}157m  7s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19096 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12899700/HBASE-19096-master-v4.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 32f7ec922762 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision |

[jira] [Commented] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269882#comment-16269882
 ] 

Hudson commented on HBASE-18090:


FAILURE: Integrated in Jenkins build HBase-1.5 #170 (See 
[https://builds.apache.org/job/HBase-1.5/170/])
HBASE-18090 Improve TableSnapshotInputFormat to allow more multiple (stack: rev 
41b4877950fc3ff5cda10b1b25c398d75eab5677)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableSnapshotInputFormat.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormat.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitter.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/client/ClientSideRegionScanner.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapred/TestTableSnapshotInputFormat.java
* (edit) 
hbase-it/src/test/java/org/apache/hadoop/hbase/mapreduce/IntegrationTestTableSnapshotInputFormat.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/RegionSplitter.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatTestBase.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableSnapshotInputFormat.java


> Improve TableSnapshotInputFormat to allow more multiple mappers per region
> --
>
> Key: HBASE-18090
> URL: https://issues.apache.org/jira/browse/HBASE-18090
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: Mikhail Antonov
>Assignee: xinxin fan
> Fix For: 1.4.0, 1.3.2, 2.0.0-beta-1
>
> Attachments: HBASE-18090-V3-master.patch, 
> HBASE-18090-V4-master.patch, HBASE-18090-V5-master.patch, 
> HBASE-18090-branch-1-v2.patch, HBASE-18090-branch-1-v2.patch, 
> HBASE-18090-branch-1.3-v1.patch, HBASE-18090-branch-1.3-v2.patch, 
> HBASE-18090.branch-1.3.001.patch, HBASE-18090.branch-1.3.001.patch, 
> HBASE-18090.branch-1.patch
>
>
> TableSnapshotInputFormat runs one map task per region in the table snapshot. 
> This places unnecessary restriction that the region layout of the original 
> table needs to take the processing resources available to MR job into 
> consideration. Allowing to run multiple mappers per region (assuming 
> reasonably even key distribution) would be useful.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19336) Improve rsgroup to allow assign all tables within a specified namespace by only writing namespace

2017-11-28 Thread xinxin fan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xinxin fan updated HBASE-19336:
---
Attachment: HBASE-19336-master-V4.patch

> Improve rsgroup to allow assign all tables within a specified namespace by 
> only writing namespace
> -
>
> Key: HBASE-19336
> URL: https://issues.apache.org/jira/browse/HBASE-19336
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Affects Versions: 2.0.0-alpha-4
>Reporter: xinxin fan
>Assignee: xinxin fan
> Attachments: HBASE-19336-master-V2.patch, 
> HBASE-19336-master-V3.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master-V4.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master.patch
>
>
> Currently, use can only assign tables within a namespace from one group to 
> another by writing all table names in move_tables_rsgroup command. Allowing 
> to assign all tables within a specifed namespace by only wirting namespace 
> name is useful.
> Usage as follows:
> {code:java}
> hbase(main):055:0> move_namespaces_rsgroup 'dest_rsgroup',['ns1']
> Took 2.2211 seconds
> {code}
> {code:java}
> hbase(main):051:0* move_servers_namespaces_rsgroup 
> 'dest_rsgroup',['hbase39.lt.163.org:60020'],['ns1','ns2']
> Took 15.3710 seconds 
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19336) Improve rsgroup to allow assign all tables within a specified namespace by only writing namespace

2017-11-28 Thread xinxin fan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269871#comment-16269871
 ] 

xinxin fan commented on HBASE-19336:


reattach for Haddop QA

> Improve rsgroup to allow assign all tables within a specified namespace by 
> only writing namespace
> -
>
> Key: HBASE-19336
> URL: https://issues.apache.org/jira/browse/HBASE-19336
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Affects Versions: 2.0.0-alpha-4
>Reporter: xinxin fan
>Assignee: xinxin fan
> Attachments: HBASE-19336-master-V2.patch, 
> HBASE-19336-master-V3.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master-V4.patch, HBASE-19336-master.patch
>
>
> Currently, use can only assign tables within a namespace from one group to 
> another by writing all table names in move_tables_rsgroup command. Allowing 
> to assign all tables within a specifed namespace by only wirting namespace 
> name is useful.
> Usage as follows:
> {code:java}
> hbase(main):055:0> move_namespaces_rsgroup 'dest_rsgroup',['ns1']
> Took 2.2211 seconds
> {code}
> {code:java}
> hbase(main):051:0* move_servers_namespaces_rsgroup 
> 'dest_rsgroup',['hbase39.lt.163.org:60020'],['ns1','ns2']
> Took 15.3710 seconds 
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269870#comment-16269870
 ] 

Hudson commented on HBASE-18090:


FAILURE: Integrated in Jenkins build HBase-1.4 #1028 (See 
[https://builds.apache.org/job/HBase-1.4/1028/])
HBASE-18090 Improve TableSnapshotInputFormat to allow more multiple (stack: rev 
4face2b662c5cee216ce7b710116bad23e8dfccc)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java
* (edit) 
hbase-it/src/test/java/org/apache/hadoop/hbase/mapreduce/IntegrationTestTableSnapshotInputFormat.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableSnapshotInputFormat.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitter.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormat.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatTestBase.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapred/TestTableSnapshotInputFormat.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableSnapshotInputFormat.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/RegionSplitter.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/client/ClientSideRegionScanner.java


> Improve TableSnapshotInputFormat to allow more multiple mappers per region
> --
>
> Key: HBASE-18090
> URL: https://issues.apache.org/jira/browse/HBASE-18090
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: Mikhail Antonov
>Assignee: xinxin fan
> Fix For: 1.4.0, 1.3.2, 2.0.0-beta-1
>
> Attachments: HBASE-18090-V3-master.patch, 
> HBASE-18090-V4-master.patch, HBASE-18090-V5-master.patch, 
> HBASE-18090-branch-1-v2.patch, HBASE-18090-branch-1-v2.patch, 
> HBASE-18090-branch-1.3-v1.patch, HBASE-18090-branch-1.3-v2.patch, 
> HBASE-18090.branch-1.3.001.patch, HBASE-18090.branch-1.3.001.patch, 
> HBASE-18090.branch-1.patch
>
>
> TableSnapshotInputFormat runs one map task per region in the table snapshot. 
> This places unnecessary restriction that the region layout of the original 
> table needs to take the processing resources available to MR job into 
> consideration. Allowing to run multiple mappers per region (assuming 
> reasonably even key distribution) would be useful.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19336) Improve rsgroup to allow assign all tables within a specified namespace by only writing namespace

2017-11-28 Thread xinxin fan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xinxin fan updated HBASE-19336:
---
Status: Patch Available  (was: In Progress)

> Improve rsgroup to allow assign all tables within a specified namespace by 
> only writing namespace
> -
>
> Key: HBASE-19336
> URL: https://issues.apache.org/jira/browse/HBASE-19336
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Affects Versions: 2.0.0-alpha-4
>Reporter: xinxin fan
>Assignee: xinxin fan
> Attachments: HBASE-19336-master-V2.patch, 
> HBASE-19336-master-V3.patch, HBASE-19336-master-V4.patch, 
> HBASE-19336-master-V4.patch, HBASE-19336-master.patch
>
>
> Currently, use can only assign tables within a namespace from one group to 
> another by writing all table names in move_tables_rsgroup command. Allowing 
> to assign all tables within a specifed namespace by only wirting namespace 
> name is useful.
> Usage as follows:
> {code:java}
> hbase(main):055:0> move_namespaces_rsgroup 'dest_rsgroup',['ns1']
> Took 2.2211 seconds
> {code}
> {code:java}
> hbase(main):051:0* move_servers_namespaces_rsgroup 
> 'dest_rsgroup',['hbase39.lt.163.org:60020'],['ns1','ns2']
> Took 15.3710 seconds 
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19367) Refactoring in RegionStates, and RSProcedureDispatcher

2017-11-28 Thread Appy (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-19367:
-
Attachment: HBASE-19367.master.003.patch

> Refactoring in RegionStates, and RSProcedureDispatcher
> --
>
> Key: HBASE-19367
> URL: https://issues.apache.org/jira/browse/HBASE-19367
> Project: HBase
>  Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
> Attachments: HBASE-19367.master.001.patch, 
> HBASE-19367.master.002.patch, HBASE-19367.master.003.patch
>
>
> Working on a bug fix, was in these parts for first time to understand new AM 
> and trying to make sense of things. Did a few improvements on the way.
> - Adding javadoc comments
> - Bug: ServerStateNode#regions is HashSet but there's no synchronization to 
> prevent concurrent addRegion/removeRegion. Let's use concurrent set instead.
> - Use getRegionsInTransitionCount() directly to avoid instead of 
> getRegionsInTransition().size() because the latter copies everything into a 
> new array - what a waste for just the size.
> - There's mixed use of getRegionNode and getRegionStateNode for same return 
> type - RegionStateNode. Changing everything to getRegionStateNode. Similarly 
> rename other *RegionNode() fns to *RegionStateNode().
> - RegionStateNode#transitionState() return value is useless since it always 
> returns it's first param.
> - Other minor improvements



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19370) Replace all uses of RowTooBigException from o.a.h.server with class of same name from o.a.h.client

2017-11-28 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269858#comment-16269858
 ] 

Andrew Purtell commented on HBASE-19370:


This is a trivial change I plan to commit shortly unless objection. Waiting for 
tests to pass first.


> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> --
>
> Key: HBASE-19370
> URL: https://issues.apache.org/jira/browse/HBASE-19370
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 1.4.0, 2.0.0-beta-1
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-19370-branch-1.patch, HBASE-19370.patch
>
>
> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> Make sure o.a.h.server.RowTooBigException is tagged as deprecated.
> RowTooBigException should derive from DoNotRetryIOException



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19370) Replace all uses of RowTooBigException from o.a.h.server with class of same name from o.a.h.client

2017-11-28 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19370:
---
Priority: Trivial  (was: Minor)

> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> --
>
> Key: HBASE-19370
> URL: https://issues.apache.org/jira/browse/HBASE-19370
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 1.4.0, 2.0.0-beta-1
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Attachments: HBASE-19370-branch-1.patch, HBASE-19370.patch
>
>
> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> Make sure o.a.h.server.RowTooBigException is tagged as deprecated.
> RowTooBigException should derive from DoNotRetryIOException



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18233) We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269857#comment-16269857
 ] 

Hadoop QA commented on HBASE-18233:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 20m 
40s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1.3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
11s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
19s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
59s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
 5s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
42s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
20s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
12s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
3s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
21s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
56s{color} | {color:green} hbase-server: The patch generated 0 new + 396 
unchanged - 2 fixed = 396 total (was 398) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
41s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
42m 16s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3. 
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
12s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
5s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}152m 47s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
28s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}246m 56s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.util.TestHBaseFsck |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:dca6535 |
| JIRA Issue | HBASE-18233 |
| JIRA Patch URL |

[jira] [Commented] (HBASE-19368) [nightly] Make xml test non-voting in branch-1.2

2017-11-28 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269838#comment-16269838
 ] 

Hudson commented on HBASE-19368:


SUCCESS: Integrated in Jenkins build HBase-1.3-IT #298 (See 
[https://builds.apache.org/job/HBase-1.3-IT/298/])
HBASE-19368 [nightly] Make xml test non-voting in branch-1.2 (stack: rev 
604266f7b75927704a520b3ac905327312383e9e)
* (edit) dev-support/Jenkinsfile


> [nightly] Make xml test non-voting in branch-1.2
> 
>
> Key: HBASE-19368
> URL: https://issues.apache.org/jira/browse/HBASE-19368
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.4.0, 1.3.2, 1.2.7
>
> Attachments: HBASE-19368.branch-1.2.001.patch
>
>
>  HBASE-19354 moved branch-1.2 to use azul openjdk. Turns out zulu-7 does does 
> not have a js script engine so jrunscript fails which causes the xml check to 
> fail with
> 19:55:59 |  -1  |xml  |   0m  7s   | The source tree has 60 
> ill-formed XML 
> 19:55:59 |  | || file(s).
> The xml.txt file has this in it:
> script engine for language js can not be found
> If I run jrunscript from zulu-7 on cmd-line, it does this:
> $ $JAVA_HOME/bin/jrunscript
> script engine for language js can not be found
> For now, make the xml check non-voting in branch-1.2 so can see if we can get 
> the nightly running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18233) We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock

2017-11-28 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269830#comment-16269830
 ] 

Hadoop QA commented on HBASE-18233:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1.3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
42s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
28s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
32s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
14s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
19s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
44s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
22s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
35s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
34s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
32s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
19s{color} | {color:green} hbase-server: The patch generated 0 new + 397 
unchanged - 1 fixed = 397 total (was 398) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  2m 
 4s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
23m 48s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3. 
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
23s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 73m 31s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}114m 39s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestAtomicOperation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:dca6535 |
| JIRA Issue | HBASE-18233 |
| JIRA Patch URL |

[jira] [Resolved] (HBASE-19368) [nightly] Make xml test non-voting in branch-1.2

2017-11-28 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-19368.
---
  Resolution: Fixed
Assignee: stack
Hadoop Flags: Reviewed

See here how the xml check doesn't fail us anymore and we move on to run unit 
tests:

https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/145/console

Because of the above, I pushed the patch to branch-1.3, 1.4, and branch-1 too.  
On branch-2, we should have a js engine so if xml is failing, it probably not 
for the same reason; i.e. no js engine available .

> [nightly] Make xml test non-voting in branch-1.2
> 
>
> Key: HBASE-19368
> URL: https://issues.apache.org/jira/browse/HBASE-19368
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 1.4.0, 1.3.2, 1.2.7
>
> Attachments: HBASE-19368.branch-1.2.001.patch
>
>
>  HBASE-19354 moved branch-1.2 to use azul openjdk. Turns out zulu-7 does does 
> not have a js script engine so jrunscript fails which causes the xml check to 
> fail with
> 19:55:59 |  -1  |xml  |   0m  7s   | The source tree has 60 
> ill-formed XML 
> 19:55:59 |  | || file(s).
> The xml.txt file has this in it:
> script engine for language js can not be found
> If I run jrunscript from zulu-7 on cmd-line, it does this:
> $ $JAVA_HOME/bin/jrunscript
> script engine for language js can not be found
> For now, make the xml check non-voting in branch-1.2 so can see if we can get 
> the nightly running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19368) [nightly] Make xml test non-voting in branch-1.2

2017-11-28 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19368:
--
Fix Version/s: 1.2.7
   1.3.2
   1.4.0

> [nightly] Make xml test non-voting in branch-1.2
> 
>
> Key: HBASE-19368
> URL: https://issues.apache.org/jira/browse/HBASE-19368
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
> Fix For: 1.4.0, 1.3.2, 1.2.7
>
> Attachments: HBASE-19368.branch-1.2.001.patch
>
>
>  HBASE-19354 moved branch-1.2 to use azul openjdk. Turns out zulu-7 does does 
> not have a js script engine so jrunscript fails which causes the xml check to 
> fail with
> 19:55:59 |  -1  |xml  |   0m  7s   | The source tree has 60 
> ill-formed XML 
> 19:55:59 |  | || file(s).
> The xml.txt file has this in it:
> script engine for language js can not be found
> If I run jrunscript from zulu-7 on cmd-line, it does this:
> $ $JAVA_HOME/bin/jrunscript
> script engine for language js can not be found
> For now, make the xml check non-voting in branch-1.2 so can see if we can get 
> the nightly running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19349) Introduce wrong version depencency of servlet-api jar

2017-11-28 Thread Appy (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269821#comment-16269821
 ] 

Appy commented on HBASE-19349:
--

Yup, not present before the change. So it's coming from one of the modules 
being added in this change.

> Introduce wrong version depencency of servlet-api jar
> -
>
> Key: HBASE-19349
> URL: https://issues.apache.org/jira/browse/HBASE-19349
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Guanghao Zhang
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
>
> Build a tarball.
> {code}
> mvn -DskipTests clean install && mvn -DskipTests package assembly:single
> tar zxvf hbase-2.0.0-beta-1-SNAPSHOT-bin.tar.gz
> {code}
> Then I found there is a servlet-api-2.5.jar in the lib directory. The right 
> depencency should be javax.servlet-api-3.1.0.jar.
> Start a distributed cluster with this tarball. And got exception when access 
> Master/RS info jsp.
> {code}
> 2017-11-27,10:02:05,066 WARN org.eclipse.jetty.server.HttpChannel: /
> java.lang.NoSuchMethodError: 
> javax.servlet.http.HttpServletRequest.isAsyncSupported()Z
> at 
> org.eclipse.jetty.server.ResourceService.sendData(ResourceService.java:689)
> at 
> org.eclipse.jetty.server.ResourceService.doGet(ResourceService.java:294)
> at 
> org.eclipse.jetty.servlet.DefaultServlet.doGet(DefaultServlet.java:458)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at 
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
> at 
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1374)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> {code}
> Try mvn depencency:tree but didn't find why servlet-api-2.5.jar was 
> introduced.
> I download hbase-2.0.0-alpha4-bin.tar.gz and didn't find servlet-api-2.5.jar. 
> And build a tar from hbase-2.0.0-alpha4-src.tar.gz and didn't find 
> servlet-api-2.5.jar, too. So this may be introduced by recently commits. And 
> should fix this when release 2.0.0-beta1.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19349) Introduce wrong version depencency of servlet-api jar

2017-11-28 Thread Appy (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269791#comment-16269791
 ] 

Appy commented on HBASE-19349:
--

Cool, let me revert that change and see if it's not there. If so, that change 
can indeed help us find root cause. 

> Introduce wrong version depencency of servlet-api jar
> -
>
> Key: HBASE-19349
> URL: https://issues.apache.org/jira/browse/HBASE-19349
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Guanghao Zhang
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
>
> Build a tarball.
> {code}
> mvn -DskipTests clean install && mvn -DskipTests package assembly:single
> tar zxvf hbase-2.0.0-beta-1-SNAPSHOT-bin.tar.gz
> {code}
> Then I found there is a servlet-api-2.5.jar in the lib directory. The right 
> depencency should be javax.servlet-api-3.1.0.jar.
> Start a distributed cluster with this tarball. And got exception when access 
> Master/RS info jsp.
> {code}
> 2017-11-27,10:02:05,066 WARN org.eclipse.jetty.server.HttpChannel: /
> java.lang.NoSuchMethodError: 
> javax.servlet.http.HttpServletRequest.isAsyncSupported()Z
> at 
> org.eclipse.jetty.server.ResourceService.sendData(ResourceService.java:689)
> at 
> org.eclipse.jetty.server.ResourceService.doGet(ResourceService.java:294)
> at 
> org.eclipse.jetty.servlet.DefaultServlet.doGet(DefaultServlet.java:458)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at 
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
> at 
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1374)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> {code}
> Try mvn depencency:tree but didn't find why servlet-api-2.5.jar was 
> introduced.
> I download hbase-2.0.0-alpha4-bin.tar.gz and didn't find servlet-api-2.5.jar. 
> And build a tar from hbase-2.0.0-alpha4-src.tar.gz and didn't find 
> servlet-api-2.5.jar, too. So this may be introduced by recently commits. And 
> should fix this when release 2.0.0-beta1.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19366) Backport to branch-1 HBASE-19035 Miss metrics when coprocessor use region scanner to read data

2017-11-28 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19366:
---
Fix Version/s: 1.4.0

> Backport to branch-1 HBASE-19035 Miss metrics when coprocessor use region 
> scanner to read data
> --
>
> Key: HBASE-19366
> URL: https://issues.apache.org/jira/browse/HBASE-19366
> Project: HBase
>  Issue Type: Sub-task
>  Components: metrics
>Reporter: stack
> Fix For: 1.4.0
>
> Attachments: HBASE-19035.branch-1.2.001.patch, 
> HBASE-19366.branch-1.3.001.patch
>
>
> Making subissue to backport the parent issue to branch-1. I'll attach first 
> attempt at a backport. It is failing in TestRegionServerMetrics in an assert.
> Making a new issue because time has elapsed since parent went into master and 
> branch-1 and I want to resolve the parent. Thanks. FYI [~zghaobac] If you've 
> input, just say sir and I can take another look.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19367) Refactoring in RegionStates, and RSProcedureDispatcher

2017-11-28 Thread Appy (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-19367:
-
Attachment: HBASE-19367.master.002.patch

> Refactoring in RegionStates, and RSProcedureDispatcher
> --
>
> Key: HBASE-19367
> URL: https://issues.apache.org/jira/browse/HBASE-19367
> Project: HBase
>  Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
> Attachments: HBASE-19367.master.001.patch, 
> HBASE-19367.master.002.patch
>
>
> Working on a bug fix, was in these parts for first time to understand new AM 
> and trying to make sense of things. Did a few improvements on the way.
> - Adding javadoc comments
> - Bug: ServerStateNode#regions is HashSet but there's no synchronization to 
> prevent concurrent addRegion/removeRegion. Let's use concurrent set instead.
> - Use getRegionsInTransitionCount() directly to avoid instead of 
> getRegionsInTransition().size() because the latter copies everything into a 
> new array - what a waste for just the size.
> - There's mixed use of getRegionNode and getRegionStateNode for same return 
> type - RegionStateNode. Changing everything to getRegionStateNode. Similarly 
> rename other *RegionNode() fns to *RegionStateNode().
> - RegionStateNode#transitionState() return value is useless since it always 
> returns it's first param.
> - Other minor improvements



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19370) Replace all uses of RowTooBigException from o.a.h.server with class of same name from o.a.h.client

2017-11-28 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269780#comment-16269780
 ] 

Andrew Purtell commented on HBASE-19370:


This should do it [~churromorales]

> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> --
>
> Key: HBASE-19370
> URL: https://issues.apache.org/jira/browse/HBASE-19370
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 1.4.0, 2.0.0-beta-1
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-19370-branch-1.patch, HBASE-19370.patch
>
>
> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> Make sure o.a.h.server.RowTooBigException is tagged as deprecated.
> RowTooBigException should derive from DoNotRetryIOException



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19370) Replace all uses of RowTooBigException from o.a.h.server with class of same name from o.a.h.client

2017-11-28 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19370:
---
Status: Patch Available  (was: Open)

> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> --
>
> Key: HBASE-19370
> URL: https://issues.apache.org/jira/browse/HBASE-19370
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 1.4.0, 2.0.0-beta-1
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-19370-branch-1.patch, HBASE-19370.patch
>
>
> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> Make sure o.a.h.server.RowTooBigException is tagged as deprecated.
> RowTooBigException should derive from DoNotRetryIOException



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19370) Replace all uses of RowTooBigException from o.a.h.server with class of same name from o.a.h.client

2017-11-28 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19370:
---
Attachment: HBASE-19370-branch-1.patch

> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> --
>
> Key: HBASE-19370
> URL: https://issues.apache.org/jira/browse/HBASE-19370
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 1.4.0, 2.0.0-beta-1
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-19370-branch-1.patch, HBASE-19370.patch
>
>
> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> Make sure o.a.h.server.RowTooBigException is tagged as deprecated.
> RowTooBigException should derive from DoNotRetryIOException



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19370) Replace all uses of RowTooBigException from o.a.h.server with class of same name from o.a.h.client

2017-11-28 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19370:
---
Attachment: HBASE-19370.patch

> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> --
>
> Key: HBASE-19370
> URL: https://issues.apache.org/jira/browse/HBASE-19370
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 1.4.0, 2.0.0-beta-1
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-19370.patch
>
>
> Replace all uses of RowTooBigException from o.a.h.server with class of same 
> name from o.a.h.client
> Make sure o.a.h.server.RowTooBigException is tagged as deprecated.
> RowTooBigException should derive from DoNotRetryIOException



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19349) Introduce wrong version depencency of servlet-api jar

2017-11-28 Thread Guanghao Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269773#comment-16269773
 ] 

Guanghao Zhang commented on HBASE-19349:


bq. that change itself won't be the cause, it would be just surfacing the 
problem present elsewhere.
Ok. I found server-api-2.5.jar in the comment 
https://issues.apache.org/jira/browse/HBASE-19089?focusedCommentId=16221504=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16221504
 . This can help us find the root cause.

> Introduce wrong version depencency of servlet-api jar
> -
>
> Key: HBASE-19349
> URL: https://issues.apache.org/jira/browse/HBASE-19349
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Guanghao Zhang
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
>
> Build a tarball.
> {code}
> mvn -DskipTests clean install && mvn -DskipTests package assembly:single
> tar zxvf hbase-2.0.0-beta-1-SNAPSHOT-bin.tar.gz
> {code}
> Then I found there is a servlet-api-2.5.jar in the lib directory. The right 
> depencency should be javax.servlet-api-3.1.0.jar.
> Start a distributed cluster with this tarball. And got exception when access 
> Master/RS info jsp.
> {code}
> 2017-11-27,10:02:05,066 WARN org.eclipse.jetty.server.HttpChannel: /
> java.lang.NoSuchMethodError: 
> javax.servlet.http.HttpServletRequest.isAsyncSupported()Z
> at 
> org.eclipse.jetty.server.ResourceService.sendData(ResourceService.java:689)
> at 
> org.eclipse.jetty.server.ResourceService.doGet(ResourceService.java:294)
> at 
> org.eclipse.jetty.servlet.DefaultServlet.doGet(DefaultServlet.java:458)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at 
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
> at 
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1374)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> {code}
> Try mvn depencency:tree but didn't find why servlet-api-2.5.jar was 
> introduced.
> I download hbase-2.0.0-alpha4-bin.tar.gz and didn't find servlet-api-2.5.jar. 
> And build a tar from hbase-2.0.0-alpha4-src.tar.gz and didn't find 
> servlet-api-2.5.jar, too. So this may be introduced by recently commits. And 
> should fix this when release 2.0.0-beta1.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

2017-11-28 Thread Vladimir Rodionov (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269756#comment-16269756
 ] 

Vladimir Rodionov commented on HBASE-17852:
---

{quote}
Why do this? Why not just mark the backup as corrupt and move on? (Why does an 
incomplete back-up freeze all backups – which you say above  I'm trying to 
understand).
{quote}

I have explained this many times already ... Restoring meta table in case of a 
backup failure is a necessary step to make future backups possible. We write 
some data during backup create, which is safe only of backup succeeds, such as 
last WAL roll timestamp per table-per RS. If backup fails, this data becomes 
corrupt w/o restoring meta table from snapshot. 

{quote}
What if its a cron job? Does this inability at moving on past failure make it 
so backup cannot be cron'd?
{quote}

Running backup repair automatically in case of a  backup failure won't hurt and 
can be incorporated into cron job

{quote}
If we weren't snapshotting/restoring the backup table, we wouldn't have to make 
a separate table to hold bulkloaded files? Is that so? (I'm not asking for a 
rewrite...).
{quote}

Yes, correct.

{quote}
I am asking questions to try and understand what is going on in here. When the 
response is terse or lean on info, I'm going to ask another question... and so 
on. As to whether snapshot/restore of the meta backup table is 'bad' or not, 
I'm still trying to understand why we would go to the extreme of offlining a 
whole table – even though rare when in error and then it seems, this offlining 
is making it so we have to add yet another table just to hold bulk loaded 
files... Pardon my being slow.
{quote}

Yes, the second table has been added long after the initial implementation was 
complete as a result of hardening bulk load support feature. You may consider 
this a s work-around, but it is pretty lightweight work-around. W/o snapshots, 
we have to make all the changes to meta table fully transactional ones. I think 
it is much harder.






> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch, HBASE-17852-v5.patch, 
> HBASE-17852-v6.patch, HBASE-17852-v7.patch, HBASE-17852-v8.patch
>
>
> Design approach rollback-via-snapshot implemented in this ticket:
> # Before backup create/delete/merge starts we take a snapshot of the backup 
> meta-table (backup system table). This procedure is lightweight because meta 
> table is small, usually should fit a single region.
> # When operation fails on a server side, we handle this failure by cleaning 
> up partial data in backup destination, followed by restoring backup 
> meta-table from a snapshot. 
> # When operation fails on a client side (abnormal termination, for example), 
> next time user will try create/merge/delete he(she) will see error message, 
> that system is in inconsistent state and repair is required, he(she) will 
> need to run backup repair tool.
> # To avoid multiple writers to the backup system table (backup client and 
> BackupObserver's) we introduce small table ONLY to keep listing of bulk 
> loaded files. All backup observers will work only with this new tables. The 
> reason: in case of a failure during backup create/delete/merge/restore, when 
> system performs automatic rollback, some data written by backup observers 
> during failed operation may be lost. This is what we try to avoid.
> # Second table keeps only bulk load related references. We do not care about 
> consistency of this table, because bulk load is idempotent operation and can 
> be repeated after failure. Partially written data in second table does not 
> affect on BackupHFileCleaner plugin, because this data (list of bulk loaded 
> files) correspond to a files which have not been loaded yet successfully and, 
> hence - are not visible to the system 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-19238) Port Hadoop's new GcTimeMonitor

2017-11-28 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269721#comment-16269721
 ] 

Andrew Purtell edited comment on HBASE-19238 at 11/28/17 11:50 PM:
---

Exposes a new Gauge metrics:
+  String GC_TIME_PERCENTAGE_KEY = "gcTimePercentage";
+  String GC_TIME_PRECENTAGE_DESC = "Percentage of time the JVM was paused in 
GC";

Configurable with these configuration parameters:
{code}
+  /** The interval over which the percentage of GC time should be calculated.
+   * A practical value would be somewhere between 30 sec and several minutes. 
*/
+  public static final String OBSERVATION_WINDOW_MS_KEY = 
"gc.time.observationWindowMs";
+  public static final long DEFAULT_OBSERVATION_WINDOW_MS = 30L * 1000;
+
+  /** sleepIntervalMs how frequently this thread should wake up to check GC 
timings.
+   * This is also a frequency with which alertHandler will be invoked if GC 
time
+   * percentage exceeds the specified limit. A practical
+   * value would likely be 500..1000 ms. */
+  public static final String SLEEP_INTERVAL_MS_KEY = "gc.time.sleepIntervalMs";
+  public static final long DEFAULT_SLEEP_INTERVAL_MS = 1000;
+
+  /** maxGcTimePercentage A GC time percentage limit (0..100) within 
observationWindowMs.
+   * Once this is exceeded, alertHandler will be invoked every sleepIntervalMs 
milliseconds
+   * until GC time percentage falls below this limit. */
+  public static final String MAX_GC_TIME_PERCENTAGE_KEY = 
"gc.time.maxGcTimePercentage";
+  public static final int DEFAULT_MAX_GC_TIME_PERCENTAGE = 50;
{code}

As is the only thing that happens is the new metric gcTimePercentage appears in 
RegionServer, REST, and Thrift server metrics sources. 

However this class can also accept an additional parameter in the constructor, 
a callback to invoke in case the maxGcTimePercentage threshold is exceeded. 

{code}
+  public interface GcTimeAlertHandler {
+void alert(GcData gcData);
+  }
{code}

At some future time we might trigger an Abortable out of the 
GcTimeAlertHandler. If the maxGcTimePercentage is set to a threshold like 95 
because we would be in a pathological condition at that point and the best 
course of action would be to terminate. (And eventually the JVM would do it for 
us, but no need to wait.)


was (Author: apurtell):
Exposes a new Gauge metrics:
+  String GC_TIME_PERCENTAGE_KEY = "gcTimePercentage";
+  String GC_TIME_PRECENTAGE_DESC = "Percentage of time the JVM was paused in 
GC";

Configurable with these configuration parameters:
{code}
+  /** The interval over which the percentage of GC time should be calculated.
+   * A practical value would be somewhere between 30 sec and several minutes. 
*/
+  public static final String OBSERVATION_WINDOW_MS_KEY = 
"gc.time.observationWindowMs";
+  public static final long DEFAULT_OBSERVATION_WINDOW_MS = 30L * 1000;
+
+  /** sleepIntervalMs how frequently this thread should wake up to check GC 
timings.
+   * This is also a frequency with which alertHandler will be invoked if GC 
time
+   * percentage exceeds the specified limit. A practical
+   * value would likely be 500..1000 ms. */
+  public static final String SLEEP_INTERVAL_MS_KEY = "gc.time.sleepIntervalMs";
+  public static final long DEFAULT_SLEEP_INTERVAL_MS = 1000;
+
+  /** maxGcTimePercentage A GC time percentage limit (0..100) within 
observationWindowMs.
+   * Once this is exceeded, alertHandler will be invoked every sleepIntervalMs 
milliseconds
+   * until GC time percentage falls below this limit. */
+  public static final String MAX_GC_TIME_PERCENTAGE_KEY = 
"gc.time.maxGcTimePercentage";
+  public static final int DEFAULT_MAX_GC_TIME_PERCENTAGE = 50;
{code}

As is the only thing that happens is the new metric gcTimePercentage appears in 
RegionServer, REST, and Thrift server metrics sources. 

However this class can also accept an additional parameter in the constructor, 
a callback to invoke in case the maxGcTimePercentage threshold is exceeded. 

{code}
+  public interface GcTimeAlertHandler {
+void alert(GcData gcData);
+  }
{code}

At some future time we might trigger an Abortable out of the 
GcTimeAlertHandler. If the maxGcTimePercentage is set to a threshold like .95 
because we would be in a pathological condition at that point and the best 
course of action would be to terminate. (And eventually the JVM would do it for 
us, but no need to wait.)

> Port Hadoop's new GcTimeMonitor
> ---
>
> Key: HBASE-19238
> URL: https://issues.apache.org/jira/browse/HBASE-19238
> Project: HBase
>  Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-19238-branch-1.patch, HBASE-19238.patch
>
>
> We borrowed Hadoop's JvmPauseMonitor some time ago. On HADOOP-14960 (Add GC

[jira] [Commented] (HBASE-19349) Introduce wrong version depencency of servlet-api jar

2017-11-28 Thread Appy (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269740#comment-16269740
 ] 

Appy commented on HBASE-19349:
--

It's just fixing missing deps in assembly.
Even if the jar doesn't get included before that change (haven't tested), that 
change itself won't be the cause, it would be just surfacing the problem 
present elsewhere. 

> Introduce wrong version depencency of servlet-api jar
> -
>
> Key: HBASE-19349
> URL: https://issues.apache.org/jira/browse/HBASE-19349
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Guanghao Zhang
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
>
> Build a tarball.
> {code}
> mvn -DskipTests clean install && mvn -DskipTests package assembly:single
> tar zxvf hbase-2.0.0-beta-1-SNAPSHOT-bin.tar.gz
> {code}
> Then I found there is a servlet-api-2.5.jar in the lib directory. The right 
> depencency should be javax.servlet-api-3.1.0.jar.
> Start a distributed cluster with this tarball. And got exception when access 
> Master/RS info jsp.
> {code}
> 2017-11-27,10:02:05,066 WARN org.eclipse.jetty.server.HttpChannel: /
> java.lang.NoSuchMethodError: 
> javax.servlet.http.HttpServletRequest.isAsyncSupported()Z
> at 
> org.eclipse.jetty.server.ResourceService.sendData(ResourceService.java:689)
> at 
> org.eclipse.jetty.server.ResourceService.doGet(ResourceService.java:294)
> at 
> org.eclipse.jetty.servlet.DefaultServlet.doGet(DefaultServlet.java:458)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at 
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
> at 
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1374)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> {code}
> Try mvn depencency:tree but didn't find why servlet-api-2.5.jar was 
> introduced.
> I download hbase-2.0.0-alpha4-bin.tar.gz and didn't find servlet-api-2.5.jar. 
> And build a tar from hbase-2.0.0-alpha4-src.tar.gz and didn't find 
> servlet-api-2.5.jar, too. So this may be introduced by recently commits. And 
> should fix this when release 2.0.0-beta1.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

2017-11-28 Thread Vladimir Rodionov (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269733#comment-16269733
 ] 

Vladimir Rodionov commented on HBASE-17852:
---

{quote}
What of my review comments are addressed in latest patch?
{quote}

Can you post your comments on RB, Stack? 

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch, HBASE-17852-v5.patch, 
> HBASE-17852-v6.patch, HBASE-17852-v7.patch, HBASE-17852-v8.patch
>
>
> Design approach rollback-via-snapshot implemented in this ticket:
> # Before backup create/delete/merge starts we take a snapshot of the backup 
> meta-table (backup system table). This procedure is lightweight because meta 
> table is small, usually should fit a single region.
> # When operation fails on a server side, we handle this failure by cleaning 
> up partial data in backup destination, followed by restoring backup 
> meta-table from a snapshot. 
> # When operation fails on a client side (abnormal termination, for example), 
> next time user will try create/merge/delete he(she) will see error message, 
> that system is in inconsistent state and repair is required, he(she) will 
> need to run backup repair tool.
> # To avoid multiple writers to the backup system table (backup client and 
> BackupObserver's) we introduce small table ONLY to keep listing of bulk 
> loaded files. All backup observers will work only with this new tables. The 
> reason: in case of a failure during backup create/delete/merge/restore, when 
> system performs automatic rollback, some data written by backup observers 
> during failed operation may be lost. This is what we try to avoid.
> # Second table keeps only bulk load related references. We do not care about 
> consistency of this table, because bulk load is idempotent operation and can 
> be repeated after failure. Partially written data in second table does not 
> affect on BackupHFileCleaner plugin, because this data (list of bulk loaded 
> files) correspond to a files which have not been loaded yet successfully and, 
> hence - are not visible to the system 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

1 2 3 >

1 - 100 of 292 matches

Mail list logo