[jira] [Updated] (HBASE-13644) Expand AC testing coverage to include all combinations of scope and permissions for region interface
[ https://issues.apache.org/jira/browse/HBASE-13644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13644: -- Labels: beginner (was: ) > Expand AC testing coverage to include all combinations of scope and > permissions for region interface > > > Key: HBASE-13644 > URL: https://issues.apache.org/jira/browse/HBASE-13644 > Project: HBase > Issue Type: Sub-task > Components: security, test >Reporter: Ashish Singhi >Priority: Major > Labels: beginner > > As of now, the tests in TestAccessController and TestAccessController2 > doesn't cover all the combinations of Scope and Permissions. Ideally, we > should have testing coverage for the entire region interface in [ACL > matrix|https://hbase.apache.org/book/appendix_acl_matrix.html]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-13644) Expand AC testing coverage to include all combinations of scope and permissions for region interface
[ https://issues.apache.org/jira/browse/HBASE-13644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi reassigned HBASE-13644: - Assignee: (was: Ashish Singhi) > Expand AC testing coverage to include all combinations of scope and > permissions for region interface > > > Key: HBASE-13644 > URL: https://issues.apache.org/jira/browse/HBASE-13644 > Project: HBase > Issue Type: Sub-task > Components: security, test >Reporter: Ashish Singhi >Priority: Major > > As of now, the tests in TestAccessController and TestAccessController2 > doesn't cover all the combinations of Scope and Permissions. Ideally, we > should have testing coverage for the entire region interface in [ACL > matrix|https://hbase.apache.org/book/appendix_acl_matrix.html]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466891#comment-16466891 ] stack commented on HBASE-20411: --- Review appreciated. Thanks. > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20539) Disable IMC; part 2
stack created HBASE-20539: - Summary: Disable IMC; part 2 Key: HBASE-20539 URL: https://issues.apache.org/jira/browse/HBASE-20539 Project: HBase Issue Type: Sub-task Components: in-memory-compaction Reporter: stack Assignee: stack Fix For: 2.0.1 Just noticed that post-flush, we are picking up a CompactingMemStore though IMC is supposed to be off. We are picking up an inMemoryCompaction of BASIC (noticed a day or so ago by [~chia7712]) and so, we fall into the default in HStore#getMemstore which is set to be a CompactingMemStore No real perf difference going by the compares I've been running but let me clean this up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20500) [rsgroup] should keep at least one server in default group
[ https://issues.apache.org/jira/browse/HBASE-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-20500: Affects Version/s: (was: 2.0.0-beta-2) 2.0.0 > [rsgroup] should keep at least one server in default group > -- > > Key: HBASE-20500 > URL: https://issues.apache.org/jira/browse/HBASE-20500 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > Attachments: HBASE-20500-branch-2.v1.patch, > HBASE-20500-master.v1.patch, HBASE-20500-master.v2.patch, > HBASE-20500-master.v3.patch, HBASE-20500-master.v4.patch > > > we move all the servers from default group > the default group will has no servers, > then we create a new table , > it will failed case of the default group has no servers > eorr info is : > EROOR: ConstraintException:Target RSGroup must have at lease on server > > we should keep at least one server in 'default' RSGroup > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[
https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466867#comment-16466867
]
Duo Zhang commented on HBASE-20244:
---
Let’s open a umbrella issue for Hadoop 3 compatibility? And linked the known
issues there. Once the production ready Hadoop 3 is out, let’s fix these things
ASAP and release a new 2.0.x.
> NoSuchMethodException when retrieving private method
> decryptEncryptedDataEncryptionKey from DFSClient
> -
>
> Key: HBASE-20244
> URL: https://issues.apache.org/jira/browse/HBASE-20244
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt
>
>
> I was running unit test against hadoop 3.0.1 RC and saw the following in test
> output:
> {code}
> ERROR [RS-EventLoopGroup-3-3]
> asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly
> initialize access to HDFS internals. Please update your WAL Provider to not
> make use of the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
> at java.lang.Class.getDeclaredMethod(Class.java:2130)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> {code}
> The private method was moved by HDFS-12574 to HdfsKMSUtil with different
> signature.
> To accommodate the above method movement, it seems we need to call the
> following method of DFSClient :
> {code}
> public KeyProvider getKeyProvider() throws IOException {
> {code}
> Since the new decryptEncryptedDataEncryptionKey method has this signature:
> {code}
> static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
> feInfo, KeyProvider keyProvider) throws IOException {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[
https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466863#comment-16466863
]
Josh Elser commented on HBASE-20244:
Wow. Ok. A simple "Yes, I disagree with you" would've been more than
sufficient. Apparently this is a touchy subject.
Just so you know, I will be revisiting this when a "production ready" Hadoop 3
version is out. This is slated to be 3.1.1 or 3.1.2. This can be kicked down
the road, but not very far.
> NoSuchMethodException when retrieving private method
> decryptEncryptedDataEncryptionKey from DFSClient
> -
>
> Key: HBASE-20244
> URL: https://issues.apache.org/jira/browse/HBASE-20244
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt
>
>
> I was running unit test against hadoop 3.0.1 RC and saw the following in test
> output:
> {code}
> ERROR [RS-EventLoopGroup-3-3]
> asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly
> initialize access to HDFS internals. Please update your WAL Provider to not
> make use of the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
> at java.lang.Class.getDeclaredMethod(Class.java:2130)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> {code}
> The private method was moved by HDFS-12574 to HdfsKMSUtil with different
> signature.
> To accommodate the above method movement, it seems we need to call the
> following method of DFSClient :
> {code}
> public KeyProvider getKeyProvider() throws IOException {
> {code}
> Since the new decryptEncryptedDataEncryptionKey method has this signature:
> {code}
> static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
> feInfo, KeyProvider keyProvider) throws IOException {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-20004: -- Status: Patch Available (was: In Progress) > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch, > HBASE-20004.v1.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[
https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466851#comment-16466851
]
Hudson commented on HBASE-20536:
Results for branch branch-2
[build #708 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/708/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/708//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/708//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/708//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Make TestRegionServerAccounting stable and it should not use absolute number
>
>
> Key: HBASE-20536
> URL: https://issues.apache.org/jira/browse/HBASE-20536
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20536.master.001.patch
>
>
> TestRegionServerAccounting failed on our internal jenkin job as we config Xmx
> to 10G. We should modify the absolute number to relative value.
> {code:java}
> new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L),
> 0);{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: HBASE-20411.branch-2.0.011.patch > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466830#comment-16466830 ] stack commented on HBASE-20411: --- Running on cluster with this patch, there is before [^2.pe.write.32026.lock.svg] , where the only lock that shows is the MutableSegment increment lock and then after, [^2.pe.write.ameliorate.106553.lock.svg] , where we have a bunch of different locks showing up. Here is how this MutableSegment lock shows in !jmc6.write_time_locks.png! > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: 2.pe.write.ameliorate.106553.lock.svg > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: 2.pe.write.32026.lock.svg > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: jmc6.write_time_locks.png > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, 41901.lock.svg, > HBASE-20411-atomiclong-longadder.patch, HBASE-20411.branch-2.0.001.patch, > HBASE-20411.branch-2.0.002.patch, HBASE-20411.branch-2.0.003.patch, > HBASE-20411.branch-2.0.004.patch, HBASE-20411.branch-2.0.005.patch, > HBASE-20411.branch-2.0.006.patch, HBASE-20411.branch-2.0.007.patch, > HBASE-20411.branch-2.0.008.patch, HBASE-20411.branch-2.0.009.patch, > HBASE-20411.branch-2.0.010.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466786#comment-16466786 ] Hudson commented on HBASE-20505: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1110 (See [https://builds.apache.org/job/HBase-1.2-IT/1110/]) HBASE-20505 PE should support multi column family read and write cases (apurtell: rev ef9fd29cad44437d3db185931c50598621bc125a) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * (edit) hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestRegionReplicaPerf.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20535) Check the UT TestSaslFanOutOneBlockAsyncDFSOutput which is always flaky
[ https://issues.apache.org/jira/browse/HBASE-20535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu resolved HBASE-20535. -- Resolution: Duplicate Close this, because [~stack] had handled it. > Check the UT TestSaslFanOutOneBlockAsyncDFSOutput which is always flaky > --- > > Key: HBASE-20535 > URL: https://issues.apache.org/jira/browse/HBASE-20535 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > > 1. > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html > 2. https://builds.apache.org/job/HBASE-Flaky-Tests//30949 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466780#comment-16466780 ] Hudson commented on HBASE-20505: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #403 (See [https://builds.apache.org/job/HBase-1.3-IT/403/]) HBASE-20505 PE should support multi column family read and write cases (apurtell: rev d5f3938520ecef6b2369ba762296b18f7266b1a7) * (edit) hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestRegionReplicaPerf.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[
https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466779#comment-16466779
]
Duo Zhang commented on HBASE-20244:
---
So I suggest we add this into our ref guide:
HBase-2.0.0 has been tested with hadoop-2.7.x, and hadoop-3.0.0. The hbase
binary is shipped with hadoop-2.7.4, but it is also OK to run this binary on
HDFS 2.8+, including 3.0.x and 3.1.x. If you want to HBase to use the dfs
client other than 2.7.4, for example, you want HBase to write file with EC, you
need to build HBase by yourself with other versions of hadoop. Notice that the
default config for HBase can only work with hadoop-3.0.0, if you want to use
other versions please set the 'hbase.wal.provider' to 'filesystem' in
hbase-site.xml. And please do not enable EC for the WAL directory. Hadoop3 is
not production ready so the support is not perfect, bug report is appreciated.
> NoSuchMethodException when retrieving private method
> decryptEncryptedDataEncryptionKey from DFSClient
> -
>
> Key: HBASE-20244
> URL: https://issues.apache.org/jira/browse/HBASE-20244
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt
>
>
> I was running unit test against hadoop 3.0.1 RC and saw the following in test
> output:
> {code}
> ERROR [RS-EventLoopGroup-3-3]
> asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly
> initialize access to HDFS internals. Please update your WAL Provider to not
> make use of the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
> at java.lang.Class.getDeclaredMethod(Class.java:2130)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> {code}
> The private method was moved by HDFS-12574 to HdfsKMSUtil with different
> signature.
> To accommodate the above method movement, it seems we need to call the
> following method of DFSClient :
> {code}
> public KeyProvider getKeyProvider() throws IOException {
> {code}
> Since the new decryptEncryptedDataEncryptionKey method has this signature:
> {code}
> static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
> feInfo, KeyProvider keyProvider) throws IOException {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-20505. Resolution: Fixed Pushed to 1.2 and up > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[
https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466752#comment-16466752
]
Duo Zhang edited comment on HBASE-20244 at 5/8/18 2:27 AM:
---
And there are also other problems for WAL on hadoop3, not only this one. If
user sets EC on the WAL directory, then we will fail to write WAL because the
EC output does not support hflush/hsync. We need to use an API which only
introduced in Hadoop 3 to create a normal output stream. This also effect
FSHLog, not only AsyncFSWAL.
In general, before we have a production ready hadoop3 release, I do not think
we need to make the Hadoop3 support perfect.
was (Author: apache9):
And there are also other problems for WAL on hadoop3, not only this one. If
user sets EC on the WAL directory, then we will fail to write WAL because the
EC output does not support hflush/haunches. We need to use an API which only
introduced in Hadoop 3 to create a normal output stream. This also effect
FSHLog, not only AsyncFSWAL.
In general, before we have a production ready hadoop3 release, I do not think
we need to make the Hadoop3 support perfect.
> NoSuchMethodException when retrieving private method
> decryptEncryptedDataEncryptionKey from DFSClient
> -
>
> Key: HBASE-20244
> URL: https://issues.apache.org/jira/browse/HBASE-20244
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt
>
>
> I was running unit test against hadoop 3.0.1 RC and saw the following in test
> output:
> {code}
> ERROR [RS-EventLoopGroup-3-3]
> asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly
> initialize access to HDFS internals. Please update your WAL Provider to not
> make use of the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
> at java.lang.Class.getDeclaredMethod(Class.java:2130)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> {code}
> The private method was moved by HDFS-12574 to HdfsKMSUtil with different
> signature.
> To accommodate the above method movement, it seems we need to call the
> following method of DFSClient :
> {code}
> public KeyProvider getKeyProvider() throws IOException {
> {code}
> Since the new decryptEncryptedDataEncryptionKey method has this signature:
> {code}
> static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
> feInfo, KeyProvider keyProvider) throws IOException {
> {code}
--
This message
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[
https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466752#comment-16466752
]
Duo Zhang commented on HBASE-20244:
---
And there are also other problems for WAL on hadoop3, not only this one. If
user sets EC on the WAL directory, then we will fail to write WAL because the
EC output does not support hflush/haunches. We need to use an API which only
introduced in Hadoop 3 to create a normal output stream. This also effect
FSHLog, not only AsyncFSWAL.
In general, before we have a production ready hadoop3 release, I do not think
we need to make the Hadoop3 support perfect.
> NoSuchMethodException when retrieving private method
> decryptEncryptedDataEncryptionKey from DFSClient
> -
>
> Key: HBASE-20244
> URL: https://issues.apache.org/jira/browse/HBASE-20244
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt
>
>
> I was running unit test against hadoop 3.0.1 RC and saw the following in test
> output:
> {code}
> ERROR [RS-EventLoopGroup-3-3]
> asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly
> initialize access to HDFS internals. Please update your WAL Provider to not
> make use of the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
> at java.lang.Class.getDeclaredMethod(Class.java:2130)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> {code}
> The private method was moved by HDFS-12574 to HdfsKMSUtil with different
> signature.
> To accommodate the above method movement, it seems we need to call the
> following method of DFSClient :
> {code}
> public KeyProvider getKeyProvider() throws IOException {
> {code}
> Since the new decryptEncryptedDataEncryptionKey method has this signature:
> {code}
> static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
> feInfo, KeyProvider keyProvider) throws IOException {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[
https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466747#comment-16466747
]
Duo Zhang commented on HBASE-20244:
---
And if you really want to use Hadoop 3.0.1 or other hadoop3 versions, you can
specify the WAL implementation in the configuration file by yourself. I stand
my point that we do not need to change our configs for a not production ready
version. The bug here is easy to fix, the patch has already been uploaded, but
I just do not want to waste time here because finally we will not add a
unstable version into our support matrix.
> NoSuchMethodException when retrieving private method
> decryptEncryptedDataEncryptionKey from DFSClient
> -
>
> Key: HBASE-20244
> URL: https://issues.apache.org/jira/browse/HBASE-20244
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt
>
>
> I was running unit test against hadoop 3.0.1 RC and saw the following in test
> output:
> {code}
> ERROR [RS-EventLoopGroup-3-3]
> asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly
> initialize access to HDFS internals. Please update your WAL Provider to not
> make use of the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
> at java.lang.Class.getDeclaredMethod(Class.java:2130)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> {code}
> The private method was moved by HDFS-12574 to HdfsKMSUtil with different
> signature.
> To accommodate the above method movement, it seems we need to call the
> following method of DFSClient :
> {code}
> public KeyProvider getKeyProvider() throws IOException {
> {code}
> Since the new decryptEncryptedDataEncryptionKey method has this signature:
> {code}
> static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
> feInfo, KeyProvider keyProvider) throws IOException {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[
https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466744#comment-16466744
]
Duo Zhang commented on HBASE-20244:
---
You can use Hadoop 3.0.0.
> NoSuchMethodException when retrieving private method
> decryptEncryptedDataEncryptionKey from DFSClient
> -
>
> Key: HBASE-20244
> URL: https://issues.apache.org/jira/browse/HBASE-20244
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt
>
>
> I was running unit test against hadoop 3.0.1 RC and saw the following in test
> output:
> {code}
> ERROR [RS-EventLoopGroup-3-3]
> asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly
> initialize access to HDFS internals. Please update your WAL Provider to not
> make use of the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
> at java.lang.Class.getDeclaredMethod(Class.java:2130)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> {code}
> The private method was moved by HDFS-12574 to HdfsKMSUtil with different
> signature.
> To accommodate the above method movement, it seems we need to call the
> following method of DFSClient :
> {code}
> public KeyProvider getKeyProvider() throws IOException {
> {code}
> Since the new decryptEncryptedDataEncryptionKey method has this signature:
> {code}
> static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
> feInfo, KeyProvider keyProvider) throws IOException {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[
https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466740#comment-16466740
]
Josh Elser commented on HBASE-20244:
{quote}Users need to build it by themselves, so I think adding something in the
refguide is enough? Especially that Hadoop 3 is not production ready.
{quote}
I think I just fundamentally disagree with you here. We expect our users to be
trying out Hadoop3, regardless of whether it's production ready or not. That's
not a justification for us to ship a known-broken feature as "on-by-default" in
my book.
Was trying to find a happy medium by suggesting that we change what the default
WAL impl is for only Hadoop3 and not for Hadoop2... Can you tell me how this
strikes you, please?
> NoSuchMethodException when retrieving private method
> decryptEncryptedDataEncryptionKey from DFSClient
> -
>
> Key: HBASE-20244
> URL: https://issues.apache.org/jira/browse/HBASE-20244
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt
>
>
> I was running unit test against hadoop 3.0.1 RC and saw the following in test
> output:
> {code}
> ERROR [RS-EventLoopGroup-3-3]
> asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly
> initialize access to HDFS internals. Please update your WAL Provider to not
> make use of the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
> at java.lang.Class.getDeclaredMethod(Class.java:2130)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> {code}
> The private method was moved by HDFS-12574 to HdfsKMSUtil with different
> signature.
> To accommodate the above method movement, it seems we need to call the
> following method of DFSClient :
> {code}
> public KeyProvider getKeyProvider() throws IOException {
> {code}
> Since the new decryptEncryptedDataEncryptionKey method has this signature:
> {code}
> static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
> feInfo, KeyProvider keyProvider) throws IOException {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20521) TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail
[ https://issues.apache.org/jira/browse/HBASE-20521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466698#comment-16466698 ] Michael Jin commented on HBASE-20521: - Ted, Mike, thanks! > TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script > run fail > --- > > Key: HBASE-20521 > URL: https://issues.apache.org/jira/browse/HBASE-20521 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0 > Environment: spark 2.2.1, hbase 2.0.0 >Reporter: Michael Jin >Assignee: Michael Jin >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20521.master.001.patch, > HBASE-20521.master.002.patch > > > HBASE-20295 fix null point exception of "conf" member variable, add > "context.getConfiguration()" in case when "conf" object was not been properly > initialized, and put it into the first priority checking sequence, this code > change affect user call "setConf" explicitly initialize "conf" object in > TableOutputFormat object, proposal to change checking sequence, use "conf" > object from "getConf" method first . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[
https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466690#comment-16466690
]
Guanghao Zhang commented on HBASE-20536:
Pushed to master and branch-2.
> Make TestRegionServerAccounting stable and it should not use absolute number
>
>
> Key: HBASE-20536
> URL: https://issues.apache.org/jira/browse/HBASE-20536
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20536.master.001.patch
>
>
> TestRegionServerAccounting failed on our internal jenkin job as we config Xmx
> to 10G. We should modify the absolute number to relative value.
> {code:java}
> new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L),
> 0);{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[
https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-20536:
---
Resolution: Fixed
Fix Version/s: 2.1.0
3.0.0
Status: Resolved (was: Patch Available)
> Make TestRegionServerAccounting stable and it should not use absolute number
>
>
> Key: HBASE-20536
> URL: https://issues.apache.org/jira/browse/HBASE-20536
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20536.master.001.patch
>
>
> TestRegionServerAccounting failed on our internal jenkin job as we config Xmx
> to 10G. We should modify the absolute number to relative value.
> {code:java}
> new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L),
> 0);{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19722) Implement a meta query statistics metrics source
[
https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466655#comment-16466655
]
Andrew Purtell commented on HBASE-19722:
Thanks for the patch! We want this to be optional, but making it an example
seems wrong. It should live in the hbase-server module. Please move it.
Optional doesn't mean example.
I think {{hermitMode}} is a fun name but probably better just to name this
boolean something like "active"? Otherwise people might be a little confused
when looking at the code in places.
I don't think you need to test for existence when marking a meter. Ahead of
marking you always register the meter if it doesn't exist.
One interesting aspect not yet handled is removing all of the metrics from
stop(). The meta region can move around. It will be closed on one server and
reopened on another. Ideally we don't want stale metrics hanging around on the
old server. If the metrics API doesn't support this we should add it.
> That's why I prefer not to put topk calculation to this coprocessor since it
> has a performance impact on all meta table operations.
I do think we need to do the top-K in the coprocessor. Or, at least, prune
meters that are not in the top K. Otherwise the amount of heap consumed by
these metrics can grow rather large depending on the number of unique clients.
> Could we count the types of access
Ideally, we should have
- Counts for each type of operation
- Aggregate count of all accesses by client identity
- Counts for each type of operation by client identity
See my comment above about pruning meters and state for rare clients. The
combinatorics of the above means we'd have 10 or more metrics per unique client.
> Implement a meta query statistics metrics source
>
>
> Key: HBASE-19722
> URL: https://issues.apache.org/jira/browse/HBASE-19722
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-19722.patch, HBASE-19722.patch.1,
> HBASE-19722.patch.2, HBASE-19722.patch.3, HBASE-19722.patch.4,
> HBASE-19722.patch.5
>
>
> Implement a meta query statistics metrics source, created whenever a
> regionserver starts hosting meta, removed when meta hosting moves. Provide
> views on top tables by request counts, top meta rowkeys by request count, top
> clients making requests by their hostname.
> Can be implemented as a coprocessor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466605#comment-16466605 ] Andrew Purtell commented on HBASE-20505: Ok, committing shortly unless objection > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20259) Doc configs for in-memory-compaction and add detail to in-memory-compaction logging
[
https://issues.apache.org/jira/browse/HBASE-20259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20259:
--
Release Note:
Disables in-memory compaction as default (See HBASE-20464 for actually
disabling IMC as default).
Adds logging of in-memory compaction configuration on creation.
Adds a chapter to the refguide on this new feature.
was:
Disables in-memory compaction as default.
Adds logging of in-memory compaction configuration on creation.
Adds a chapter to the refguide on this new feature.
> Doc configs for in-memory-compaction and add detail to in-memory-compaction
> logging
> ---
>
> Key: HBASE-20259
> URL: https://issues.apache.org/jira/browse/HBASE-20259
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20259.branch-2.0.ADDENDUM.patch,
> HBASE-20259.master.001.patch, HBASE-20259.master.002.patch,
> HBASE-20259.master.003.patch
>
>
> I set {{hbase.systemtables.compacting.memstore.type}} to NONE but it seems
> like in-memory is still on. My table looks like this:
> {code}
> Table ycsb is ENABLED
> ycsb
> COLUMN FAMILIES DESCRIPTION
> {NAME => 'family', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER =
> > 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false',
> > CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false',
> > COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
> {code}
> Looks like table doesn't have it on either (IN_MEMORY_COMPACTION doesn't show
> in the above).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466584#comment-16466584 ] stack commented on HBASE-20505: --- Please [~apurtell] I've moved to PE for testing perf lately. This stuff and [~ram_krish]'s change help. > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[
https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466564#comment-16466564
]
Andrew Purtell commented on HBASE-20505:
Rebased patches.
Tested manually on a cluster.
Used {{./bin/hbase pe --nomapred --rows=100 --families=10 randomWrite 1}}
to write 1M rows with 10 families
Verified expected 10 family schema with the shell.
Then, {{./bin/hbase pe --nomapred --rows=100 --families=1 scanRange1000 1}} to
read ranges of 1000 rows over one of the families.
Then, {{./bin/hbase pe --nomapred --rows=100 --families=10 scanRange1000 1}} to
read ranges of 1000 rows over all ten of the families. Bytes per result in scan
metrics is the expected ~10x of the single family run.
Mind if I put this in branch-2.0 along with the rest [~stack] ?
> PE should support multi column family read and write cases
> --
>
> Key: HBASE-20505
> URL: https://issues.apache.org/jira/browse/HBASE-20505
> Project: HBase
> Issue Type: Test
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch
>
>
> PerformanceEvaluation has a --columns parameter but this adjusts the number
> of distinct column qualifiers to write (and, with --addColumns, to add to the
> scan), not the number of column families.
> We need something like a new --families parameter that will increase the
> number of column families defined in the test table schema, written to, and
> included in gets and scans. Default is 1, current behavior.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Fix Version/s: 1.4.5 2.0.1 1.3.3 > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Attachment: (was: HBASE-20505-branch-1.patch) > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Attachment: (was: HBASE-20505.patch) > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Attachment: HBASE-20505.patch HBASE-20505-branch-1.patch > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[
https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466548#comment-16466548
]
Hudson commented on HBASE-20538:
Results for branch branch-2.0
[build #271 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/271/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/271//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/271//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/271//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException:
> Rejected by the jceks.key.serialFilter or jdk.serialFilter property
>
>
> Key: HBASE-20538
> URL: https://issues.apache.org/jira/browse/HBASE-20538
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments:
> 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch
>
>
> Infra must have updated our JDK over the weekend. The test
> TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since.
> [~gabor.bota] ran into it already over in HDFS-13494 and provided nice
> pointers as to cause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[
https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466537#comment-16466537
]
Hudson commented on HBASE-20523:
Results for branch branch-1.2
[build #324 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/324/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/324//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/324//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/324//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> PE tool should support configuring client side buffering sizes
> --
>
> Key: HBASE-20523
> URL: https://issues.apache.org/jira/browse/HBASE-20523
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5
>
> Attachments: HBASE-20523.patch, HBASE-20523_1.patch,
> HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch,
> HBASE-20523_branch-1.patch
>
>
> The client side buffering size impacts the write load and the write
> performance. Hence for testing purpose it is better we allow client side
> buffering to be configurable in PE. Already YCSB has such facility.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20521) TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail
[ https://issues.apache.org/jira/browse/HBASE-20521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466525#comment-16466525 ] Ted Yu commented on HBASE-20521: Mike - take your time, please merge tomorrow. > TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script > run fail > --- > > Key: HBASE-20521 > URL: https://issues.apache.org/jira/browse/HBASE-20521 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0 > Environment: spark 2.2.1, hbase 2.0.0 >Reporter: Michael Jin >Assignee: Michael Jin >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20521.master.001.patch, > HBASE-20521.master.002.patch > > > HBASE-20295 fix null point exception of "conf" member variable, add > "context.getConfiguration()" in case when "conf" object was not been properly > initialized, and put it into the first priority checking sequence, this code > change affect user call "setConf" explicitly initialize "conf" object in > TableOutputFormat object, proposal to change checking sequence, use "conf" > object from "getConf" method first . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[
https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466519#comment-16466519
]
Hudson commented on HBASE-20538:
Results for branch branch-2
[build #707 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/707/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/707//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/707//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/707//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException:
> Rejected by the jceks.key.serialFilter or jdk.serialFilter property
>
>
> Key: HBASE-20538
> URL: https://issues.apache.org/jira/browse/HBASE-20538
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments:
> 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch
>
>
> Infra must have updated our JDK over the weekend. The test
> TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since.
> [~gabor.bota] ran into it already over in HDFS-13494 and provided nice
> pointers as to cause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20521) TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail
[ https://issues.apache.org/jira/browse/HBASE-20521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466515#comment-16466515 ] Mike Drob commented on HBASE-20521: --- minor nit: you lost a space between if and opening parenthesis in TableOutputFormat. I (or somebody else) can fix this on commit. I ran the patch and also tested it against my pig script and it looks like things work. Ran the unit test against a version that only uses getConf() and doesn't read the context conf and it failed as expected. So we're good on that side. [~tedyu] - if you have no further comments on RB feel free to push, otherwise I'll pick this up tomorrow. Would like to see it in master, branch-2, and branch-2.0 > TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script > run fail > --- > > Key: HBASE-20521 > URL: https://issues.apache.org/jira/browse/HBASE-20521 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0 > Environment: spark 2.2.1, hbase 2.0.0 >Reporter: Michael Jin >Assignee: Michael Jin >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20521.master.001.patch, > HBASE-20521.master.002.patch > > > HBASE-20295 fix null point exception of "conf" member variable, add > "context.getConfiguration()" in case when "conf" object was not been properly > initialized, and put it into the first priority checking sequence, this code > change affect user call "setConf" explicitly initialize "conf" object in > TableOutputFormat object, proposal to change checking sequence, use "conf" > object from "getConf" method first . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: HBASE-20411.branch-2.0.010.patch > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, 41901.lock.svg, > HBASE-20411-atomiclong-longadder.patch, HBASE-20411.branch-2.0.001.patch, > HBASE-20411.branch-2.0.002.patch, HBASE-20411.branch-2.0.003.patch, > HBASE-20411.branch-2.0.004.patch, HBASE-20411.branch-2.0.005.patch, > HBASE-20411.branch-2.0.006.patch, HBASE-20411.branch-2.0.007.patch, > HBASE-20411.branch-2.0.008.patch, HBASE-20411.branch-2.0.009.patch, > HBASE-20411.branch-2.0.010.patch > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: HBASE-20411.branch-2.0.009.patch > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, 41901.lock.svg, > HBASE-20411-atomiclong-longadder.patch, HBASE-20411.branch-2.0.001.patch, > HBASE-20411.branch-2.0.002.patch, HBASE-20411.branch-2.0.003.patch, > HBASE-20411.branch-2.0.004.patch, HBASE-20411.branch-2.0.005.patch, > HBASE-20411.branch-2.0.006.patch, HBASE-20411.branch-2.0.007.patch, > HBASE-20411.branch-2.0.008.patch, HBASE-20411.branch-2.0.009.patch, > HBASE-20411.branch-2.0.010.patch > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[
https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466454#comment-16466454
]
Hudson commented on HBASE-20523:
Results for branch branch-1.4
[build #313 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/313/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/313//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/313//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/313//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> PE tool should support configuring client side buffering sizes
> --
>
> Key: HBASE-20523
> URL: https://issues.apache.org/jira/browse/HBASE-20523
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5
>
> Attachments: HBASE-20523.patch, HBASE-20523_1.patch,
> HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch,
> HBASE-20523_branch-1.patch
>
>
> The client side buffering size impacts the write load and the write
> performance. Hence for testing purpose it is better we allow client side
> buffering to be configurable in PE. Already YCSB has such facility.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466289#comment-16466289 ] stack commented on HBASE-20538: --- Thanks for reminder [~mdrob] > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20537) The download link is not available in the downloads webpage
[ https://issues.apache.org/jira/browse/HBASE-20537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-20537. --- Resolution: Fixed Assignee: stack Pushed small change adding 'src' and or 'bin' text. Thanks [~iemejia]. > The download link is not available in the downloads webpage > --- > > Key: HBASE-20537 > URL: https://issues.apache.org/jira/browse/HBASE-20537 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.0 >Reporter: Ismaël Mejía >Assignee: stack >Priority: Major > > When you open https://hbase.apache.org/downloads.html > There is not a direct link to download both the binary or source > distributions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20257) hbase-spark should not depend on com.google.code.findbugs.jsr305
[
https://issues.apache.org/jira/browse/HBASE-20257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466182#comment-16466182
]
Ted Yu commented on HBASE-20257:
Ping [~busbey]
> hbase-spark should not depend on com.google.code.findbugs.jsr305
>
>
> Key: HBASE-20257
> URL: https://issues.apache.org/jira/browse/HBASE-20257
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20257.v01.patch, HBASE-20257.v02.patch,
> HBASE-20257.v03.patch, HBASE-20257.v04.patch
>
>
> The following can be observed in the build output of master branch:
> {code}
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed
> with message:
> We don't allow the JSR305 jar from the Findbugs project, see HBASE-16321.
> Found Banned Dependency: com.google.code.findbugs:jsr305:jar:1.3.9
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
> {code}
> Here is related snippet from hbase-spark/pom.xml:
> {code}
>
> com.google.code.findbugs
> jsr305
> {code}
> Dependency on jsr305 should be dropped.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20537) The download link is not available in the downloads webpage
[ https://issues.apache.org/jira/browse/HBASE-20537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466180#comment-16466180 ] stack commented on HBASE-20537: --- Funny, the little boxes will work... but yeah, should have more than this. Let me fix. > The download link is not available in the downloads webpage > --- > > Key: HBASE-20537 > URL: https://issues.apache.org/jira/browse/HBASE-20537 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.0 >Reporter: Ismaël Mejía >Priority: Major > > When you open https://hbase.apache.org/downloads.html > There is not a direct link to download both the binary or source > distributions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466178#comment-16466178 ] stack commented on HBASE-20538: --- Disabled the test for now till dig in more (Need to fix HDFS-keymaking side?). Pushed to branch-2.0+. Doesn't seem to be issue in branch-1. > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20538: -- Fix Version/s: 2.0.1 2.1.0 3.0.0 > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20538: -- Attachment: 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466173#comment-16466173 ] Mike Drob commented on HBASE-20538: --- Our peter mentioned this during the 2.0 release vote as well > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20538: -- Priority: Critical (was: Major) > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[
https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466161#comment-16466161
]
stack commented on HBASE-20538:
---
Here is exception:
{code}
java.lang.reflect.InvocationTargetException
at
org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput.createEncryptionZone(TestSaslFanOutOneBlockAsyncDFSOutput.java:201)
at
org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput.setUp(TestSaslFanOutOneBlockAsyncDFSOutput.java:229)
Caused by: org.apache.hadoop.ipc.RemoteException:
Can't recover key for test_key from keystore
file:/testptch/hbase/hbase-server/target/test-data/75e90585-3745-4205-a94b-405265c6c96f/test.jks
at
org.apache.hadoop.crypto.key.JavaKeyStoreProvider.getMetadata(JavaKeyStoreProvider.java:424)
at
org.apache.hadoop.crypto.key.KeyProviderExtension.getMetadata(KeyProviderExtension.java:100)
at
org.apache.hadoop.hdfs.server.namenode.FSDirEncryptionZoneOp.ensureKeyIsInitialized(FSDirEncryptionZoneOp.java:124)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.createEncryptionZone(FSNamesystem.java:7002)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.createEncryptionZone(NameNodeRpcServer.java:2036)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.createEncryptionZone(ClientNamenodeProtocolServerSideTranslatorPB.java:1448)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
Caused by: java.security.UnrecoverableKeyException: Rejected by the
jceks.key.serialFilter or jdk.serialFilter property
at com.sun.crypto.provider.KeyProtector.unseal(KeyProtector.java:352)
at
com.sun.crypto.provider.JceKeyStore.engineGetKey(JceKeyStore.java:136)
at java.security.KeyStore.getKey(KeyStore.java:1023)
at
org.apache.hadoop.crypto.key.JavaKeyStoreProvider.getMetadata(JavaKeyStoreProvider.java:410)
... 14 more
at
org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput.createEncryptionZone(TestSaslFanOutOneBlockAsyncDFSOutput.java:201)
at
org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput.setUp(TestSaslFanOutOneBlockAsyncDFSOutput.java:229)
{code}
> TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException:
> Rejected by the jceks.key.serialFilter or jdk.serialFilter property
>
>
> Key: HBASE-20538
> URL: https://issues.apache.org/jira/browse/HBASE-20538
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
>
> Infra must have updated our JDK over the weekend. The test
> TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since.
> [~gabor.bota] ran into it already over in HDFS-13494 and provided nice
> pointers as to cause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
stack created HBASE-20538: - Summary: TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property Key: HBASE-20538 URL: https://issues.apache.org/jira/browse/HBASE-20538 Project: HBase Issue Type: Bug Reporter: stack Infra must have updated our JDK over the weekend. The test TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. [~gabor.bota] ran into it already over in HDFS-13494 and provided nice pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20537) The download link is not available in the downloads webpage
[ https://issues.apache.org/jira/browse/HBASE-20537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466097#comment-16466097 ] stack commented on HBASE-20537: --- duh > The download link is not available in the downloads webpage > --- > > Key: HBASE-20537 > URL: https://issues.apache.org/jira/browse/HBASE-20537 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.0 >Reporter: Ismaël Mejía >Priority: Major > > When you open https://hbase.apache.org/downloads.html > There is not a direct link to download both the binary or source > distributions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[
https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466057#comment-16466057
]
Hudson commented on HBASE-20523:
Results for branch branch-2.0
[build #270 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/270/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/270//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/270//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/270//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> PE tool should support configuring client side buffering sizes
> --
>
> Key: HBASE-20523
> URL: https://issues.apache.org/jira/browse/HBASE-20523
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5
>
> Attachments: HBASE-20523.patch, HBASE-20523_1.patch,
> HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch,
> HBASE-20523_branch-1.patch
>
>
> The client side buffering size impacts the write load and the write
> performance. Hence for testing purpose it is better we allow client side
> buffering to be configurable in PE. Already YCSB has such facility.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[
https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466038#comment-16466038
]
Hudson commented on HBASE-20523:
Results for branch branch-2
[build #706 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/706/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/706//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/706//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/706//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> PE tool should support configuring client side buffering sizes
> --
>
> Key: HBASE-20523
> URL: https://issues.apache.org/jira/browse/HBASE-20523
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5
>
> Attachments: HBASE-20523.patch, HBASE-20523_1.patch,
> HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch,
> HBASE-20523_branch-1.patch
>
>
> The client side buffering size impacts the write load and the write
> performance. Hence for testing purpose it is better we allow client side
> buffering to be configurable in PE. Already YCSB has such facility.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20500) [rsgroup] should keep at least one server in default group
[ https://issues.apache.org/jira/browse/HBASE-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20500: --- Status: Open (was: Patch Available) > [rsgroup] should keep at least one server in default group > -- > > Key: HBASE-20500 > URL: https://issues.apache.org/jira/browse/HBASE-20500 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0-beta-2 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > Attachments: HBASE-20500-branch-2.v1.patch, > HBASE-20500-master.v1.patch, HBASE-20500-master.v2.patch, > HBASE-20500-master.v3.patch, HBASE-20500-master.v4.patch > > > we move all the servers from default group > the default group will has no servers, > then we create a new table , > it will failed case of the default group has no servers > eorr info is : > EROOR: ConstraintException:Target RSGroup must have at lease on server > > we should keep at least one server in 'default' RSGroup > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465881#comment-16465881 ] Ashish Singhi commented on HBASE-20004: --- Attached a patch making the option of enabling and disabling the OPTIONS method as configurable. By default OPTIONS method will not be added in constraint mapping and will be added in the other places. Please review. > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch, > HBASE-20004.v1.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-20004: -- Attachment: HBASE-20004.v1.patch > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch, > HBASE-20004.v1.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20259) Doc configs for in-memory-compaction and add detail to in-memory-compaction logging
[
https://issues.apache.org/jira/browse/HBASE-20259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465874#comment-16465874
]
Chia-Ping Tsai commented on HBASE-20259:
The release notes says "Disables in-memory compaction as default." But the
default policy still is BASIC.
{code:java}
public static final String COMPACTING_MEMSTORE_TYPE_DEFAULT =
String.valueOf(MemoryCompactionPolicy.BASIC){code}
Could someone point out the "Disables in-memory compaction as default." for me?
Or setting BASIC be default is a kind of disabling the in-memory compaction
since basic policy only invokes the flattening and merging?
It seems to me HBASE-20464 does disable the in-memory compaction.
> Doc configs for in-memory-compaction and add detail to in-memory-compaction
> logging
> ---
>
> Key: HBASE-20259
> URL: https://issues.apache.org/jira/browse/HBASE-20259
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20259.branch-2.0.ADDENDUM.patch,
> HBASE-20259.master.001.patch, HBASE-20259.master.002.patch,
> HBASE-20259.master.003.patch
>
>
> I set {{hbase.systemtables.compacting.memstore.type}} to NONE but it seems
> like in-memory is still on. My table looks like this:
> {code}
> Table ycsb is ENABLED
> ycsb
> COLUMN FAMILIES DESCRIPTION
> {NAME => 'family', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER =
> > 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false',
> > CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false',
> > COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
> {code}
> Looks like table doesn't have it on either (IN_MEMORY_COMPACTION doesn't show
> in the above).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[
https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465857#comment-16465857
]
Hadoop QA commented on HBASE-20536:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
1s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}110m
58s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20536 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922248/HBASE-20536.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 05e7b80b7e99 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8e6ff689e8 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12737/testReport/ |
| Max. process+thread count | 4305 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12737/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Make
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[
https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465849#comment-16465849
]
Hudson commented on HBASE-20523:
Results for branch master
[build #323 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/323/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> PE tool should support configuring client side buffering sizes
> --
>
> Key: HBASE-20523
> URL: https://issues.apache.org/jira/browse/HBASE-20523
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5
>
> Attachments: HBASE-20523.patch, HBASE-20523_1.patch,
> HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch,
> HBASE-20523_branch-1.patch
>
>
> The client side buffering size impacts the write load and the write
> performance. Hence for testing purpose it is better we allow client side
> buffering to be configurable in PE. Already YCSB has such facility.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20508) TestIncrementalBackupWithBulkLoad doesn't need to be Parameterized test
[
https://issues.apache.org/jira/browse/HBASE-20508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465848#comment-16465848
]
Hudson commented on HBASE-20508:
Results for branch master
[build #323 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/323/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> TestIncrementalBackupWithBulkLoad doesn't need to be Parameterized test
> ---
>
> Key: HBASE-20508
> URL: https://issues.apache.org/jira/browse/HBASE-20508
> Project: HBase
> Issue Type: Test
> Components: backuprestore
>Reporter: Ted Yu
>Assignee: maoling
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-20508-master.v0.patch, HBASE-20508-master.v1.patch
>
>
> TestIncrementalBackupWithBulkLoad currently is Parameterized with only one
> value returned from data() method.
> In its ctor, this value is ignored.
> TestIncrementalBackupWithBulkLoad doesn't need to be Parameterized.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20537) The download link is not available in the downloads webpage
Ismaël Mejía created HBASE-20537: Summary: The download link is not available in the downloads webpage Key: HBASE-20537 URL: https://issues.apache.org/jira/browse/HBASE-20537 Project: HBase Issue Type: Bug Components: website Affects Versions: 2.0.0 Reporter: Ismaël Mejía When you open https://hbase.apache.org/downloads.html There is not a direct link to download both the binary or source distributions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[
https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465790#comment-16465790
]
Zheng Hu commented on HBASE-20475:
--
I guess we were stopping a ReplicationSourceShipper which has not finished its
initialization. so the NPE happen
{code}
worker.entryReader.interrupt();
{code}
> Fix the flaky TestReplicationDroppedTables unit test.
> -
>
> Key: HBASE-20475
> URL: https://issues.apache.org/jira/browse/HBASE-20475
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20475-addendum-v2.patch,
> HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch
>
>
> See
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[
https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465759#comment-16465759
]
Zheng Hu edited comment on HBASE-20475 at 5/7/18 10:46 AM:
---
Checked the UT & log again, the phenomenon is:
{code:java}
> Key: HBASE-20475
> URL: https://issues.apache.org/jira/browse/HBASE-20475
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20475-addendum-v2.patch,
> HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch
>
>
> See
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[
https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465759#comment-16465759
]
Zheng Hu edited comment on HBASE-20475 at 5/7/18 10:43 AM:
---
Checked the UT & log again, the phenomenon is:
{code:java}
> Key: HBASE-20475
> URL: https://issues.apache.org/jira/browse/HBASE-20475
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20475-addendum-v2.patch,
> HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch
>
>
> See
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[ https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465766#comment-16465766 ] Duo Zhang commented on HBASE-20475: --- What about the NPE posted by me above? Is there a race? > Fix the flaky TestReplicationDroppedTables unit test. > - > > Key: HBASE-20475 > URL: https://issues.apache.org/jira/browse/HBASE-20475 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20475-addendum-v2.patch, > HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[
https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465760#comment-16465760
]
Duo Zhang commented on HBASE-20536:
---
+1.
> Make TestRegionServerAccounting stable and it should not use absolute number
>
>
> Key: HBASE-20536
> URL: https://issues.apache.org/jira/browse/HBASE-20536
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-20536.master.001.patch
>
>
> TestRegionServerAccounting failed on our internal jenkin job as we config Xmx
> to 10G. We should modify the absolute number to relative value.
> {code:java}
> new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L),
> 0);{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[
https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465759#comment-16465759
]
Zheng Hu commented on HBASE-20475:
--
Checked the UT & log again, the phenomenon is:
{code}
> Key: HBASE-20475
> URL: https://issues.apache.org/jira/browse/HBASE-20475
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20475-addendum-v2.patch,
> HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch
>
>
> See
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[
https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-20536:
---
Assignee: Guanghao Zhang
Status: Patch Available (was: Open)
> Make TestRegionServerAccounting stable and it should not use absolute number
>
>
> Key: HBASE-20536
> URL: https://issues.apache.org/jira/browse/HBASE-20536
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-20536.master.001.patch
>
>
> TestRegionServerAccounting failed on our internal jenkin job as we config Xmx
> to 10G. We should modify the absolute number to relative value.
> {code:java}
> new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L),
> 0);{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[
https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-20536:
---
Attachment: HBASE-20536.master.001.patch
> Make TestRegionServerAccounting stable and it should not use absolute number
>
>
> Key: HBASE-20536
> URL: https://issues.apache.org/jira/browse/HBASE-20536
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-20536.master.001.patch
>
>
> TestRegionServerAccounting failed on our internal jenkin job as we config Xmx
> to 10G. We should modify the absolute number to relative value.
> {code:java}
> new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L),
> 0);{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465676#comment-16465676 ] Hudson commented on HBASE-20523: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1109 (See [https://builds.apache.org/job/HBase-1.2-IT/1109/]) HBASE-20523 PE tool should support configuring client side buffering (ramkrishna.s.vasudevan: rev b3a162bc11d5eb76479b45b07efc5cbd3bc6de47) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20523: --- Resolution: Fixed Fix Version/s: 1.4.5 1.2.7 1.5.0 Status: Resolved (was: Patch Available) Pushed to master, 2.0, branch-2, 1.2, 1.4 and branch-1. Thanks all for the reviews. > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[
https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465652#comment-16465652
]
Hadoop QA commented on HBASE-20523:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s{color}
| {color:red} HBASE-20523 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20523 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922244/HBASE-20523_branch-1.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12736/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> PE tool should support configuring client side buffering sizes
> --
>
> Key: HBASE-20523
> URL: https://issues.apache.org/jira/browse/HBASE-20523
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.1.0, 2.0.1
>
> Attachments: HBASE-20523.patch, HBASE-20523_1.patch,
> HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch,
> HBASE-20523_branch-1.patch
>
>
> The client side buffering size impacts the write load and the write
> performance. Hence for testing purpose it is better we allow client side
> buffering to be configurable in PE. Already YCSB has such facility.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20523: --- Attachment: HBASE-20523_branch-1.patch > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 2.0.1 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20523: --- Attachment: HBASE-20523_branch-1.2.patch > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 2.0.1 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20523: --- Attachment: HBASE-20523_branch-1.4.patch > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 2.0.1 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.4.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
Guanghao Zhang created HBASE-20536:
--
Summary: Make TestRegionServerAccounting stable and it should not
use absolute number
Key: HBASE-20536
URL: https://issues.apache.org/jira/browse/HBASE-20536
Project: HBase
Issue Type: Improvement
Reporter: Guanghao Zhang
TestRegionServerAccounting failed on our internal jenkin job as we config Xmx
to 10G. We should modify the absolute number to relative value.
{code:java}
new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L),
0);{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18812) Recategorize some of classes used as tools
[ https://issues.apache.org/jira/browse/HBASE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465629#comment-16465629 ] Guangxu Cheng commented on HBASE-18812: --- # BackupDriver # RestoreDriver # MapReduceHFileSplitterJob # Export # IndexBuilder # SampleUploader # StripeCompactionsPerformanceEvaluation # HashTable # SyncTable # VerifyReplication # ProcedureWALLoaderPerformanceEvaluation # ProcedureWALPerformanceEvaluation # ExpiredMobFileCleaner # Compressor # DumpReplicationQueues # ReplicationSyncUp # CreateSnapshot # Canary # ServerCommandLine # MasterProcedureSchedulerPerformanceEvaluation # WALPerformanceEvaluation # ZKAclReset Attach 002 patch as [~chia7712] suggestions.Changing the InterfaceAudience from Private to LP. Thanks > Recategorize some of classes used as tools > -- > > Key: HBASE-18812 > URL: https://issues.apache.org/jira/browse/HBASE-18812 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-18812.master.001.patch, > HBASE-18812.master.002.patch > > > The classes used from cmd line should be made as LimitedPrivate.TOOLS. The > candidates are shown below. > # BackupDriver > # RestoreDriver > # CreateSnapshot > # SnapshotInfo > # ExportSnapshot > # Canary > # VersionInfo > # RegionMover > # CellCounter > # CopyTable > # DumpReplicationQueues > # Export > # HashTable > # Import > # ImportTsv > # LoadIncrementalHFiles > # ReplicationSyncUp > # SyncTable > # VerifyReplication > # WALPlayer > # ZkAclReset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18812) Recategorize some of classes used as tools
[ https://issues.apache.org/jira/browse/HBASE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guangxu Cheng updated HBASE-18812: -- Attachment: HBASE-18812.master.002.patch > Recategorize some of classes used as tools > -- > > Key: HBASE-18812 > URL: https://issues.apache.org/jira/browse/HBASE-18812 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-18812.master.001.patch, > HBASE-18812.master.002.patch > > > The classes used from cmd line should be made as LimitedPrivate.TOOLS. The > candidates are shown below. > # BackupDriver > # RestoreDriver > # CreateSnapshot > # SnapshotInfo > # ExportSnapshot > # Canary > # VersionInfo > # RegionMover > # CellCounter > # CopyTable > # DumpReplicationQueues > # Export > # HashTable > # Import > # ImportTsv > # LoadIncrementalHFiles > # ReplicationSyncUp > # SyncTable > # VerifyReplication > # WALPlayer > # ZkAclReset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20521) TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail
[ https://issues.apache.org/jira/browse/HBASE-20521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465586#comment-16465586 ] Michael Jin commented on HBASE-20521: - [~mdrob], any update about the attach? > TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script > run fail > --- > > Key: HBASE-20521 > URL: https://issues.apache.org/jira/browse/HBASE-20521 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0 > Environment: spark 2.2.1, hbase 2.0.0 >Reporter: Michael Jin >Assignee: Michael Jin >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20521.master.001.patch, > HBASE-20521.master.002.patch > > > HBASE-20295 fix null point exception of "conf" member variable, add > "context.getConfiguration()" in case when "conf" object was not been properly > initialized, and put it into the first priority checking sequence, this code > change affect user call "setConf" explicitly initialize "conf" object in > TableOutputFormat object, proposal to change checking sequence, use "conf" > object from "getConf" method first . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[
https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465571#comment-16465571
]
Hadoop QA commented on HBASE-20411:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
27s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
37s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 33s{color}
| {color:red} hbase-server generated 6 new + 182 unchanged - 6 fixed = 188
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
13s{color} | {color:red} hbase-server: The patch generated 8 new + 534
unchanged - 2 fixed = 542 total (was 536) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
1s{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed =
1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} hbase-server generated 0 new + 0 unchanged - 2 fixed
= 0 total (was 2) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}149m 16s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}184m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Dead store to snapshotSize in
org.apache.hadoop.hbase.regionserver.DefaultMemStore.getFlushableSize() At
DefaultMemStore.java:org.apache.hadoop.hbase.regionserver.DefaultMemStore.getFlushableSize()
At DefaultMemStore.java:[line 102] |
| Failed junit tests | hadoop.hbase.regionserver.TestHStore |
| | hadoop.hbase.regionserver.TestHRegionReplayEvents |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:369877d |
| JIRA Issue | HBASE-20411 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922215/HBASE-20411.branch-2.0.008.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 37e0141dbf79 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11
12:16:42 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
