[jira] [Commented] (HBASE-11039) [VisibilityController] Integration test for labeled data set mixing and filtered excise
[ https://issues.apache.org/jira/browse/HBASE-11039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060376#comment-14060376 ] ramkrishna.s.vasudevan commented on HBASE-11039: Opened https://issues.apache.org/jira/browse/HBASE-11509 > [VisibilityController] Integration test for labeled data set mixing and > filtered excise > --- > > Key: HBASE-11039 > URL: https://issues.apache.org/jira/browse/HBASE-11039 > Project: HBase > Issue Type: Test >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 0.98.4 > > Attachments: HBASE-11039_ITBLL_v1.patch, HBASE-11039_v3_0.98.patch, > HBASE-11039_v4_0.98.patch, HBASE-11039_v5_0.98.patch > > > Create an integration test for the VisibilityController that: > 1. Create several tables of test data > 2. Assign a set of auths to each table. Label all entries in the table with > appropriate visibility expressions. Insure that some data in every table > overlaps with data in other tables at common row/family/qualifier > coordinates. Generate data like ITBLL so we can verify all data present later. > 3. Mix the data from the different tables into a new common table > 4. Verify for each set of auths defined in step #2 that all entries found in > the source table can be found in the common table. Like the ITBLL > verification step but done N times for each set of auths defined in step #2. > 5. Choose one of the source tables. Get its set of auths. Perform a deletion > with visibility expression from the common table using those auths. > 6. Verify that no data in the common table with the auth set chosen in #5 > remains. A simple row count with the set of auths chosen in #5 that should > return 0. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11509) Forward port HBASE-11039 to trunk and branch-1 after HBASE-11489
ramkrishna.s.vasudevan created HBASE-11509: -- Summary: Forward port HBASE-11039 to trunk and branch-1 after HBASE-11489 Key: HBASE-11509 URL: https://issues.apache.org/jira/browse/HBASE-11509 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 1.0.0 The visibility related IT test case that verifies delete behaviour has to be ported to branch-1 and master after HBASE-11489 is fixed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11039) [VisibilityController] Integration test for labeled data set mixing and filtered excise
[ https://issues.apache.org/jira/browse/HBASE-11039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-11039. Resolution: Fixed Hadoop Flags: Reviewed Thanks for the reviews and verifications Andy. Thanks to Anoop for the discussions. Marking fixed versions as 0.98.4 here. > [VisibilityController] Integration test for labeled data set mixing and > filtered excise > --- > > Key: HBASE-11039 > URL: https://issues.apache.org/jira/browse/HBASE-11039 > Project: HBase > Issue Type: Test >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 0.98.4 > > Attachments: HBASE-11039_ITBLL_v1.patch, HBASE-11039_v3_0.98.patch, > HBASE-11039_v4_0.98.patch, HBASE-11039_v5_0.98.patch > > > Create an integration test for the VisibilityController that: > 1. Create several tables of test data > 2. Assign a set of auths to each table. Label all entries in the table with > appropriate visibility expressions. Insure that some data in every table > overlaps with data in other tables at common row/family/qualifier > coordinates. Generate data like ITBLL so we can verify all data present later. > 3. Mix the data from the different tables into a new common table > 4. Verify for each set of auths defined in step #2 that all entries found in > the source table can be found in the common table. Like the ITBLL > verification step but done N times for each set of auths defined in step #2. > 5. Choose one of the source tables. Get its set of auths. Perform a deletion > with visibility expression from the common table using those auths. > 6. Verify that no data in the common table with the auth set chosen in #5 > remains. A simple row count with the set of auths chosen in #5 that should > return 0. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11039) [VisibilityController] Integration test for labeled data set mixing and filtered excise
[ https://issues.apache.org/jira/browse/HBASE-11039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11039: --- Fix Version/s: (was: 1.0.0) (was: 0.99.0) Committed to 0.98. Will commit to trunk and 1.0 after HBASE-11489 is resolved. Will open a follow up JIRA for the same. Or if needed we can commit the code changes to trunk and if HBASE-11489 is resolved this would automatically work with mvn verify. Anyway with a cluster the exisitng patch would work in the trunk also. > [VisibilityController] Integration test for labeled data set mixing and > filtered excise > --- > > Key: HBASE-11039 > URL: https://issues.apache.org/jira/browse/HBASE-11039 > Project: HBase > Issue Type: Test >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 0.98.4 > > Attachments: HBASE-11039_ITBLL_v1.patch, HBASE-11039_v3_0.98.patch, > HBASE-11039_v4_0.98.patch, HBASE-11039_v5_0.98.patch > > > Create an integration test for the VisibilityController that: > 1. Create several tables of test data > 2. Assign a set of auths to each table. Label all entries in the table with > appropriate visibility expressions. Insure that some data in every table > overlaps with data in other tables at common row/family/qualifier > coordinates. Generate data like ITBLL so we can verify all data present later. > 3. Mix the data from the different tables into a new common table > 4. Verify for each set of auths defined in step #2 that all entries found in > the source table can be found in the common table. Like the ITBLL > verification step but done N times for each set of auths defined in step #2. > 5. Choose one of the source tables. Get its set of auths. Perform a deletion > with visibility expression from the common table using those auths. > 6. Verify that no data in the common table with the auth set chosen in #5 > remains. A simple row count with the set of auths chosen in #5 that should > return 0. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11506) IntegrationTestWithCellVisibilityLoadAndVerify allow User to be passed as arg
[
https://issues.apache.org/jira/browse/HBASE-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060371#comment-14060371
]
Hadoop QA commented on HBASE-11506:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655499/HBASE-11506_2.patch
against trunk revision .
ATTACHMENT ID: 12655499
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10051//console
This message is automatically generated.
> IntegrationTestWithCellVisibilityLoadAndVerify allow User to be passed as arg
> -
>
> Key: HBASE-11506
> URL: https://issues.apache.org/jira/browse/HBASE-11506
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-11506.patch, HBASE-11506_1.patch,
> HBASE-11506_2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11506) IntegrationTestWithCellVisibilityLoadAndVerify allow User to be passed as arg
[ https://issues.apache.org/jira/browse/HBASE-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11506: --- Status: Patch Available (was: Open) > IntegrationTestWithCellVisibilityLoadAndVerify allow User to be passed as arg > - > > Key: HBASE-11506 > URL: https://issues.apache.org/jira/browse/HBASE-11506 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-11506.patch, HBASE-11506_1.patch, > HBASE-11506_2.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11506) IntegrationTestWithCellVisibilityLoadAndVerify allow User to be passed as arg
[ https://issues.apache.org/jira/browse/HBASE-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11506: --- Attachment: HBASE-11506_2.patch Updated patch addressing the comments. This is ready for commit. > IntegrationTestWithCellVisibilityLoadAndVerify allow User to be passed as arg > - > > Key: HBASE-11506 > URL: https://issues.apache.org/jira/browse/HBASE-11506 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-11506.patch, HBASE-11506_1.patch, > HBASE-11506_2.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-11493) Autorestart option is not working because of stale znode "shutdown"
[
https://issues.apache.org/jira/browse/HBASE-11493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060359#comment-14060359
]
rajeshbabu edited comment on HBASE-11493 at 7/14/14 5:50 AM:
-
committed to 0.98+ branches.
Thanks for patch [~nijel]
Thanks for review Andrew,Ted.
was (Author: rajesh23):
committed to 0.98+ branches. Thanks for review Andrew,Ted.
> Autorestart option is not working because of stale znode "shutdown"
>
>
> Key: HBASE-11493
> URL: https://issues.apache.org/jira/browse/HBASE-11493
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.3
>Reporter: Nishan Shetty
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: HBASE-11493.patch, HBASE-11493.v1.patch
>
>
> In hbase-daemon.sh autorestart option the znode used is "shutdown"
> {code}
> zshutdown=`$bin/hbase org.apache.hadoop.hbase.util.HBaseConfTool
> zookeeper.znode.state`
> if [ "$zshutdown" == "null" ]; then zshutdown="shutdown"; fi
> zFullShutdown=$zparent/$zshutdown
> {code}
> the node shutdown is not available now and is changed to running.
> *Since this node is not available, script does not restart the node and it
> exists.*
> So autorestart scripts needs to be changed to make use of "running" znode.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-11497) Expose RpcScheduling implementations as LimitedPrivate interfaces
[ https://issues.apache.org/jira/browse/HBASE-11497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-11497. Resolution: Fixed Hadoop Flags: Reviewed > Expose RpcScheduling implementations as LimitedPrivate interfaces > - > > Key: HBASE-11497 > URL: https://issues.apache.org/jira/browse/HBASE-11497 > Project: HBase > Issue Type: Improvement > Components: io, regionserver, Usability >Affects Versions: 0.99.0, 0.98.4 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.99.0, 0.98.4, 2.0.0 > > Attachments: HBASE-11497-0.98.patch, HBASE-11497.patch, > hbase-11497-0.98-v0.patch > > > In PHOENIX-938 we are attempting to resolve cross-RS deadlocks in indexing by > adding custom RPC handlers (so regular puts/reads don't interfere with index > updates). However, we've run into a couple of snags where the interfaces > change, making it a bit more difficult to support interoperability between > minor versions as the underlying RPC handling changed (for the better, but > still different :). > This would just mark those interfaces Public, Evolving, so we still have some > flexibility, but don't break existing usage. > Note, this kind of thing will come up for any client who is doing custom RPC > handling - beyond the recently added flexibility - but wants to stay in line > with the current HBase implementation (rather than building their own RPC > handling mechanisms). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11493) Autorestart option is not working because of stale znode "shutdown"
[
https://issues.apache.org/jira/browse/HBASE-11493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060359#comment-14060359
]
rajeshbabu commented on HBASE-11493:
committed to 0.98+ branches. Thanks for review Andrew,Ted.
> Autorestart option is not working because of stale znode "shutdown"
>
>
> Key: HBASE-11493
> URL: https://issues.apache.org/jira/browse/HBASE-11493
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.3
>Reporter: Nishan Shetty
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: HBASE-11493.patch, HBASE-11493.v1.patch
>
>
> In hbase-daemon.sh autorestart option the znode used is "shutdown"
> {code}
> zshutdown=`$bin/hbase org.apache.hadoop.hbase.util.HBaseConfTool
> zookeeper.znode.state`
> if [ "$zshutdown" == "null" ]; then zshutdown="shutdown"; fi
> zFullShutdown=$zparent/$zshutdown
> {code}
> the node shutdown is not available now and is changed to running.
> *Since this node is not available, script does not restart the node and it
> exists.*
> So autorestart scripts needs to be changed to make use of "running" znode.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11497) Expose RpcScheduling implementations as LimitedPrivate interfaces
[ https://issues.apache.org/jira/browse/HBASE-11497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060358#comment-14060358 ] Andrew Purtell commented on HBASE-11497: Thanks stack. Going to commit now. > Expose RpcScheduling implementations as LimitedPrivate interfaces > - > > Key: HBASE-11497 > URL: https://issues.apache.org/jira/browse/HBASE-11497 > Project: HBase > Issue Type: Improvement > Components: io, regionserver, Usability >Affects Versions: 0.99.0, 0.98.4 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.99.0, 0.98.4, 2.0.0 > > Attachments: HBASE-11497-0.98.patch, HBASE-11497.patch, > hbase-11497-0.98-v0.patch > > > In PHOENIX-938 we are attempting to resolve cross-RS deadlocks in indexing by > adding custom RPC handlers (so regular puts/reads don't interfere with index > updates). However, we've run into a couple of snags where the interfaces > change, making it a bit more difficult to support interoperability between > minor versions as the underlying RPC handling changed (for the better, but > still different :). > This would just mark those interfaces Public, Evolving, so we still have some > flexibility, but don't break existing usage. > Note, this kind of thing will come up for any client who is doing custom RPC > handling - beyond the recently added flexibility - but wants to stay in line > with the current HBase implementation (rather than building their own RPC > handling mechanisms). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060357#comment-14060357 ] Jerry He commented on HBASE-11148: -- Ping [~jmhsieh]. Have you got a chance to review the patch? Any additional comment? > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 0.99.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11442) ReplicationSourceManager doesn't cleanup the queues for recovered sources
[ https://issues.apache.org/jira/browse/HBASE-11442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060353#comment-14060353 ] Andrew Purtell commented on HBASE-11442: Skimmed the patch, +1 for 0.98 > ReplicationSourceManager doesn't cleanup the queues for recovered sources > - > > Key: HBASE-11442 > URL: https://issues.apache.org/jira/browse/HBASE-11442 > Project: HBase > Issue Type: Bug > Components: Replication >Reporter: Virag Kothari >Assignee: Virag Kothari > Attachments: HBASE-11442.patch, HBASE-11442_2.patch > > > Currently, ReplicationSourceManager only cleanups the queues for recovered > sources when the queue is being closed. This can cause the already read WAL's > files to be read again when a region server doing failover also dies. This > can cause replication to possibly happen again > For e.g lets say RS1 dies with 5 files in queue and RS2 is doing the > failover. Now, lets say RS2 dies after going thru 3 files in queue and RS3 is > doing the failover. In this case, RS3 will again read those 3 files as they > were not removed from the queue. (Though it will read the first file from the > set pos. in ZK) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11497) Expose RpcScheduling implementations as LimitedPrivate interfaces
[ https://issues.apache.org/jira/browse/HBASE-11497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060352#comment-14060352 ] stack commented on HBASE-11497: --- +1 nice. > Expose RpcScheduling implementations as LimitedPrivate interfaces > - > > Key: HBASE-11497 > URL: https://issues.apache.org/jira/browse/HBASE-11497 > Project: HBase > Issue Type: Improvement > Components: io, regionserver, Usability >Affects Versions: 0.99.0, 0.98.4 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.99.0, 0.98.4, 2.0.0 > > Attachments: HBASE-11497-0.98.patch, HBASE-11497.patch, > hbase-11497-0.98-v0.patch > > > In PHOENIX-938 we are attempting to resolve cross-RS deadlocks in indexing by > adding custom RPC handlers (so regular puts/reads don't interfere with index > updates). However, we've run into a couple of snags where the interfaces > change, making it a bit more difficult to support interoperability between > minor versions as the underlying RPC handling changed (for the better, but > still different :). > This would just mark those interfaces Public, Evolving, so we still have some > flexibility, but don't break existing usage. > Note, this kind of thing will come up for any client who is doing custom RPC > handling - beyond the recently added flexibility - but wants to stay in line > with the current HBase implementation (rather than building their own RPC > handling mechanisms). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060351#comment-14060351
]
Andrew Purtell commented on HBASE-11492:
bq. I'm working on a version for patch v2 that also includes backwards
compatibility for 0.98, will attach soon.
Just want to highlight again the 0.98 patch is not a backport of the master
patch. I used the master patch as a template but rather than changing "ipc.*"
to "hbase.ipc*", first we look for the "hbase.ipc.*" version, if not present we
look for the "ipc.*" version, and if not present we use the default value.
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060348#comment-14060348
]
Andrew Purtell commented on HBASE-11492:
[~nkeywal], when I was doing a substitution for "ipc.* for the 0.98 patch I
found a few additional places that I don't think the master patch changed.
Please have a look. Overzealous in 0.98? Or a few more substitutions needed for
master?
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060347#comment-14060347
]
Andrew Purtell commented on HBASE-11492:
bq. folks might have configured these options already - might be an issue in
0.98 as well.
The 0.98 patch applies the old configuration settings unless the new ones are
present. Not sure how it will be an issue. Can you elaborate?
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10930) Change Filters and GetClosestRowBeforeTracker to work with Cells
[
https://issues.apache.org/jira/browse/HBASE-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060345#comment-14060345
]
stack commented on HBASE-10930:
---
Looks great.
We giving up functionality here?
public Cell getNextCellHint(Cell currentKV) throws IOException {
-// Old filters based off of this class will override KeyValue
getNextKeyHint(KeyValue).
-// Thus to maintain compatibility we need to call the old version.
-return getNextKeyHint(KeyValueUtil.ensureKeyValue(currentKV));
+return null;
Can we change name of internal members from transformKV to tranformCell?
-return transform(KeyValueUtil.ensureKeyValue(v));
+if (!CellComparator.equals(v, referenceKV)) {
+ throw new IllegalStateException("Reference Cell: " + this.referenceKV +
" does not match: "
+ + v);
+}
+return this.transformedKV;
Ditto for referenceKV?
Change this from kv to c?
public ReturnCode filterKeyValue(Cell kv) {
Ditto here:
public Cell getNextCellHint(Cell currentKV) {
I suggest this because you've done all this nice clean up... it is a shame to
leave these little references to kv around when simple to change.
On other hand [~ram_krish], the above comments are nits. +1 on commit. Change
some of the above if want but if you don't I'm good w/ the patch as is. Can do
the nits in subsequent work. Good stuff.
> Change Filters and GetClosestRowBeforeTracker to work with Cells
>
>
> Key: HBASE-10930
> URL: https://issues.apache.org/jira/browse/HBASE-10930
> Project: HBase
> Issue Type: Sub-task
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-10930.patch, HBASE-10930_1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11039) [VisibilityController] Integration test for labeled data set mixing and filtered excise
[ https://issues.apache.org/jira/browse/HBASE-11039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060343#comment-14060343 ] Andrew Purtell commented on HBASE-11039: bq. Works with "mvn verify" also. Sets "dfs.permissions" to false. +1 > [VisibilityController] Integration test for labeled data set mixing and > filtered excise > --- > > Key: HBASE-11039 > URL: https://issues.apache.org/jira/browse/HBASE-11039 > Project: HBase > Issue Type: Test >Affects Versions: 0.98.1 >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 0.99.0, 1.0.0, 0.98.4 > > Attachments: HBASE-11039_ITBLL_v1.patch, HBASE-11039_v3_0.98.patch, > HBASE-11039_v4_0.98.patch, HBASE-11039_v5_0.98.patch > > > Create an integration test for the VisibilityController that: > 1. Create several tables of test data > 2. Assign a set of auths to each table. Label all entries in the table with > appropriate visibility expressions. Insure that some data in every table > overlaps with data in other tables at common row/family/qualifier > coordinates. Generate data like ITBLL so we can verify all data present later. > 3. Mix the data from the different tables into a new common table > 4. Verify for each set of auths defined in step #2 that all entries found in > the source table can be found in the common table. Like the ITBLL > verification step but done N times for each set of auths defined in step #2. > 5. Choose one of the source tables. Get its set of auths. Perform a deletion > with visibility expression from the common table using those auths. > 6. Verify that no data in the common table with the auth set chosen in #5 > remains. A simple row count with the set of auths chosen in #5 that should > return 0. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[
https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060338#comment-14060338
]
stack commented on HBASE-11480:
---
[~cuijianwei] This is 0.94 only? I dont' see it in master.
> ClientScanner might not close the HConnection created in construction
> -
>
> Key: HBASE-11480
> URL: https://issues.apache.org/jira/browse/HBASE-11480
> Project: HBase
> Issue Type: Improvement
> Components: Client, Scanners
>Affects Versions: 0.96.2, 0.94.20, 0.98.3
>Reporter: cuijianwei
>Priority: Trivial
>
> ClientScanner will create HConnection in its construction:
> {code}
> public ClientScanner(final Configuration conf, final Scan scan,
> final byte[] tableName) throws IOException {
> this(conf, scan, tableName, HConnectionManager.getConnection(conf));
> }
> {code}
> However, this connection won't be closed in ClientScanner.close(). Is it
> better to deprecate this construction?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060335#comment-14060335 ] Misty Stanley-Jones commented on HBASE-11400: - @stack I think that HBASE-11459 might have collided with my work and I rewrote a lot of that stuff so it's a little hard to parse whether everything from it is there. Can you look it over? I can do a more detailed comparison and make sure I got everything tomorrow if not. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400-5.patch, > HBASE-11400.patch, data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[
https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060334#comment-14060334
]
Hadoop QA commented on HBASE-11400:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655493/HBASE-11400-5.patch
against trunk revision .
ATTACHMENT ID: 12655493
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10050//console
This message is automatically generated.
> Edit, consolidate, and update Compression and data encoding docs
>
>
> Key: HBASE-11400
> URL: https://issues.apache.org/jira/browse/HBASE-11400
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
>Priority: Minor
> Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch,
> HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400-5.patch,
> HBASE-11400.patch, data_block_diff_encoding.png, data_block_no_encoding.png,
> data_block_prefix_encoding.png
>
>
> Current docs are here: http://hbase.apache.org/book.html#compression.test
> It could use some editing and expansion.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-8473) add note to ref guide about snapshots and ec2 reverse dns requirements.
[
https://issues.apache.org/jira/browse/HBASE-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060333#comment-14060333
]
Misty Stanley-Jones commented on HBASE-8473:
Thanks, I saw the other section but didn't de-duplicate for now because it
needs more re-working. At some point I will stop re-working things and start
working on the new guide. Almost to that point.
> add note to ref guide about snapshots and ec2 reverse dns requirements.
> ---
>
> Key: HBASE-8473
> URL: https://issues.apache.org/jira/browse/HBASE-8473
> Project: HBase
> Issue Type: Bug
> Components: documentation, snapshots
>Affects Versions: 0.98.0, 0.94.6.1, 0.95.0
>Reporter: Jonathan Hsieh
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-8473.patch
>
>
> From IRC from mighty Jeremy Carroll.
> {code}
> 17:10 jmhsieh: I think I found the root cuase. All my region
> servers reach the barrier, but it does not continue.
> 17:11 jmhsieh: All RS have this in their logs: 2013-05-01
> 00:04:56,356 DEBUG org.apache.hadoop.hbase.procedure.Subprocedure:
> Subprocedure 'backup1' coordinator notified of 'acquire', waiting on
> 'reached' or 'abort' from coordinator.
> 17:11 jmhsieh: Then the coordinator (Master) never sends
> anything. They just sit until the timeout.
> 17:12 jmhsieh: So basically 'reached' is never obtained.
> Then abort it set, and it fails.
> ...
> 17:24 jmhsieh: Found the bug. The hostnames dont match the
> master due to DNS resolution
> 17:25 jmhsieh: The barrier aquired is putting in the local
> hostname from the regionservers. In EC2 (Where reverse DNS does not work
> well), the master hands the internal name to the client.
> 17:25 jmhsieh:
> https://s3.amazonaws.com/uploads.hipchat.com/23947/185789/au94meik0h3y5ii/Screen%20Shot%202013-04-30%20at%2017.25.50.png
>
> 17:26 jmhsieh: So it's waiting for something like
> 'ip-10-155-208-202.ec2.internal,60020,1367366580066' zNode to show up, but
> instead 'hbasemetaclustera-d1b0a484,60020,1367366580066,' is being inserted.
> Barrier is not reached
> 17:27 jmhsieh: Reason being in our environment the master
> does not have a reverse DNS entry. So we get stuff like this on RegionServer
> startup in our logs.
> 17:27 jmhsieh: 2013-05-01 00:03:00,614 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us hostname
> to use. Was=hbasemetaclustera-d1b0a484, Now=ip-10-155-208-202.ec2.internal
> 17:54 jmhsieh: That was it. Verified. Now that Reverse DNS
> is working, snapshots are working. Now how to figure out how to get Reverse
> DNS working on Route53. I wished there was something like 'slave.host.name'
> inside of Hadoop for this. Looking at source code.
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-8473) add note to ref guide about snapshots and ec2 reverse dns requirements.
[
https://issues.apache.org/jira/browse/HBASE-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8473:
-
Resolution: Fixed
Fix Version/s: 2.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to master. Thanks [~misty]
> add note to ref guide about snapshots and ec2 reverse dns requirements.
> ---
>
> Key: HBASE-8473
> URL: https://issues.apache.org/jira/browse/HBASE-8473
> Project: HBase
> Issue Type: Bug
> Components: documentation, snapshots
>Affects Versions: 0.98.0, 0.94.6.1, 0.95.0
>Reporter: Jonathan Hsieh
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-8473.patch
>
>
> From IRC from mighty Jeremy Carroll.
> {code}
> 17:10 jmhsieh: I think I found the root cuase. All my region
> servers reach the barrier, but it does not continue.
> 17:11 jmhsieh: All RS have this in their logs: 2013-05-01
> 00:04:56,356 DEBUG org.apache.hadoop.hbase.procedure.Subprocedure:
> Subprocedure 'backup1' coordinator notified of 'acquire', waiting on
> 'reached' or 'abort' from coordinator.
> 17:11 jmhsieh: Then the coordinator (Master) never sends
> anything. They just sit until the timeout.
> 17:12 jmhsieh: So basically 'reached' is never obtained.
> Then abort it set, and it fails.
> ...
> 17:24 jmhsieh: Found the bug. The hostnames dont match the
> master due to DNS resolution
> 17:25 jmhsieh: The barrier aquired is putting in the local
> hostname from the regionservers. In EC2 (Where reverse DNS does not work
> well), the master hands the internal name to the client.
> 17:25 jmhsieh:
> https://s3.amazonaws.com/uploads.hipchat.com/23947/185789/au94meik0h3y5ii/Screen%20Shot%202013-04-30%20at%2017.25.50.png
>
> 17:26 jmhsieh: So it's waiting for something like
> 'ip-10-155-208-202.ec2.internal,60020,1367366580066' zNode to show up, but
> instead 'hbasemetaclustera-d1b0a484,60020,1367366580066,' is being inserted.
> Barrier is not reached
> 17:27 jmhsieh: Reason being in our environment the master
> does not have a reverse DNS entry. So we get stuff like this on RegionServer
> startup in our logs.
> 17:27 jmhsieh: 2013-05-01 00:03:00,614 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us hostname
> to use. Was=hbasemetaclustera-d1b0a484, Now=ip-10-155-208-202.ec2.internal
> 17:54 jmhsieh: That was it. Verified. Now that Reverse DNS
> is working, snapshots are working. Now how to figure out how to get Reverse
> DNS working on Route53. I wished there was something like 'slave.host.name'
> inside of Hadoop for this. Looking at source code.
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-5.patch I put back the LZ4 stuff. [~stack], the images should work fine in HTML and HTML single. Please test my changes to pom.xml. I think they might break other images. If so, the images need to be copied from site/images/ to docbkx/images/. I know there is a maintenance cost but it's better for the long term and makes it easier to add more images in future. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400-5.patch, > HBASE-11400.patch, data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8473) add note to ref guide about snapshots and ec2 reverse dns requirements.
[
https://issues.apache.org/jira/browse/HBASE-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060326#comment-14060326
]
stack commented on HBASE-8473:
--
Patch is good. I could apply. We mention reverse dns as an old requirement at
"2.1.2.2. DNS". Should this section link to it? Might be ok if it didn't? I
can just commit. Troubleshooting is a good place for this info at least for
starters. Let me just commit.
> add note to ref guide about snapshots and ec2 reverse dns requirements.
> ---
>
> Key: HBASE-8473
> URL: https://issues.apache.org/jira/browse/HBASE-8473
> Project: HBase
> Issue Type: Bug
> Components: documentation, snapshots
>Affects Versions: 0.98.0, 0.94.6.1, 0.95.0
>Reporter: Jonathan Hsieh
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-8473.patch
>
>
> From IRC from mighty Jeremy Carroll.
> {code}
> 17:10 jmhsieh: I think I found the root cuase. All my region
> servers reach the barrier, but it does not continue.
> 17:11 jmhsieh: All RS have this in their logs: 2013-05-01
> 00:04:56,356 DEBUG org.apache.hadoop.hbase.procedure.Subprocedure:
> Subprocedure 'backup1' coordinator notified of 'acquire', waiting on
> 'reached' or 'abort' from coordinator.
> 17:11 jmhsieh: Then the coordinator (Master) never sends
> anything. They just sit until the timeout.
> 17:12 jmhsieh: So basically 'reached' is never obtained.
> Then abort it set, and it fails.
> ...
> 17:24 jmhsieh: Found the bug. The hostnames dont match the
> master due to DNS resolution
> 17:25 jmhsieh: The barrier aquired is putting in the local
> hostname from the regionservers. In EC2 (Where reverse DNS does not work
> well), the master hands the internal name to the client.
> 17:25 jmhsieh:
> https://s3.amazonaws.com/uploads.hipchat.com/23947/185789/au94meik0h3y5ii/Screen%20Shot%202013-04-30%20at%2017.25.50.png
>
> 17:26 jmhsieh: So it's waiting for something like
> 'ip-10-155-208-202.ec2.internal,60020,1367366580066' zNode to show up, but
> instead 'hbasemetaclustera-d1b0a484,60020,1367366580066,' is being inserted.
> Barrier is not reached
> 17:27 jmhsieh: Reason being in our environment the master
> does not have a reverse DNS entry. So we get stuff like this on RegionServer
> startup in our logs.
> 17:27 jmhsieh: 2013-05-01 00:03:00,614 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us hostname
> to use. Was=hbasemetaclustera-d1b0a484, Now=ip-10-155-208-202.ec2.internal
> 17:54 jmhsieh: That was it. Verified. Now that Reverse DNS
> is working, snapshots are working. Now how to figure out how to get Reverse
> DNS working on Route53. I wished there was something like 'slave.host.name'
> inside of Hadoop for this. Looking at source code.
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060323#comment-14060323 ] stack commented on HBASE-11400: --- [~misty] rebase? I'm talking about what HBASE-11459 added. That is still there after your patch? > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch, > data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11499) AsyncProcess.buildDetailedErrorMessage concatenates strings using + in a loop
[
https://issues.apache.org/jira/browse/HBASE-11499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11499:
--
Resolution: Fixed
Fix Version/s: 2.0.0
Assignee: Mike Drob
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to master. Thanks for the patch [~mdrob]
> AsyncProcess.buildDetailedErrorMessage concatenates strings using + in a loop
> -
>
> Key: HBASE-11499
> URL: https://issues.apache.org/jira/browse/HBASE-11499
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Trivial
> Labels: findbugs
> Fix For: 2.0.0
>
> Attachments: HBASE-11499.patch.txt, HBASE-11499.v2.patch.txt
>
>
> The method is building a string using lots of concatenation, including in a
> loop, which will be {{O(n^2)}} because a new array will be allocated each
> time. Would be more efficient to use a StringBuilder for this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11508) Document changes to IPC config parameters from HBASE-11492
Misty Stanley-Jones created HBASE-11508: --- Summary: Document changes to IPC config parameters from HBASE-11492 Key: HBASE-11508 URL: https://issues.apache.org/jira/browse/HBASE-11508 Project: HBase Issue Type: Sub-task Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10800) Use CellComparator instead of KVComparator
[ https://issues.apache.org/jira/browse/HBASE-10800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060302#comment-14060302 ] stack commented on HBASE-10800: --- bq. Hence this compareOnlyKeyPortion. But tags and mvcc are part of key? No? bq. They flag is to denote the types of compare that we may have to do as per the above comment. To be clear, the flag is a little ugly. A method that spells out what the compare does would be better than a single method differentiated by a boolean flag IMO. Thanks for working on this [~ram_krish] This is an important one to get behind us. > Use CellComparator instead of KVComparator > -- > > Key: HBASE-10800 > URL: https://issues.apache.org/jira/browse/HBASE-10800 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-10800_1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060299#comment-14060299 ] Misty Stanley-Jones commented on HBASE-11400: - Hmm. How have I done that? Let me explore and see. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch, > data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060295#comment-14060295 ] stack commented on HBASE-11400: --- The images changes look good misty. They work for the single-page book and multipage? I think you are throwing away some recent additions I made around LZ4 and instruction on use of native libs? i.e. how to make sure you have native lib pointers in expected locations? > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch, > data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[
https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060294#comment-14060294
]
stack commented on HBASE-11507:
---
bq. I disagree...
... is usually followed by argument/justification.
Add this at least the below [~gustavoanatoly] so that the unusual developer who
happens to stumble upon the zero copy byte string optimization (totally
by-passing our use of ByteStringer elsewhere throughout the codebase) is
suitably discouraged
{code}
diff --git
a/hbase-protocol/src/main/java/com/google/protobuf/HBaseZeroCopyByteString.java
b/hbase-protocol/src/main/java/com/google/protobuf/HBaseZeroCopyByteString.java
index f7e9c7f..7f59c9c 100644
---
a/hbase-protocol/src/main/java/com/google/protobuf/HBaseZeroCopyByteString.java
+++
b/hbase-protocol/src/main/java/com/google/protobuf/HBaseZeroCopyByteString.java
@@ -26,7 +26,10 @@ package com.google.protobuf; // This is a lie.
* from C buffer to JVM buffer).
*
* @since 0.96.1
+ * @deprecated Do not use directly. Instead go via
org.apache.hadoop.hbase.util.ByteStringer
+ * class instead. See HBASE-8
*/
+@Deprecated
public final class HBaseZeroCopyByteString extends LiteralByteString {
// Gotten from AsyncHBase code base with permission.
/** Private constructor so this class cannot be instantiated. */
{code}
Thanks.
> Enhance test-patch.sh to check for direct calls to
> HBaseZeroCopyByteString.wrap()
> -
>
> Key: HBASE-11507
> URL: https://issues.apache.org/jira/browse/HBASE-11507
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
>Priority: Trivial
> Attachments: HBASE-11507.patch
>
>
> HBaseZeroCopyByteString.wrap() should not be called directly.
> ByteStringer.wrap() should be called instead.
> This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to
> test-patch.sh.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11474) [Thrift2] support authentication/impersonation
[ https://issues.apache.org/jira/browse/HBASE-11474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060268#comment-14060268 ] Hudson commented on HBASE-11474: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #372 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/372/]) HBASE-11474 [Thrift2] support authentication/impersonation (jxiang: rev 9cf54eee53e184b78ff05b0c6d2485f062c8cb83) * hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithLabels.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftHBaseServiceHandler.java * hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandler.java > [Thrift2] support authentication/impersonation > -- > > Key: HBASE-11474 > URL: https://issues.apache.org/jira/browse/HBASE-11474 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: hbase-11474.patch, hbase-11474_1.patch > > > This is to do the same as HBASE-11349 for Thrift2. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11474) [Thrift2] support authentication/impersonation
[ https://issues.apache.org/jira/browse/HBASE-11474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060266#comment-14060266 ] Hudson commented on HBASE-11474: FAILURE: Integrated in HBase-1.0 #34 (See [https://builds.apache.org/job/HBase-1.0/34/]) HBASE-11474 [Thrift2] support authentication/impersonation (jxiang: rev 757ecdb14f75169b1c1a02e7b36fd9c463a7b8f7) * hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithLabels.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftHBaseServiceHandler.java * hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java > [Thrift2] support authentication/impersonation > -- > > Key: HBASE-11474 > URL: https://issues.apache.org/jira/browse/HBASE-11474 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: hbase-11474.patch, hbase-11474_1.patch > > > This is to do the same as HBASE-11349 for Thrift2. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11474) [Thrift2] support authentication/impersonation
[ https://issues.apache.org/jira/browse/HBASE-11474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060264#comment-14060264 ] Hudson commented on HBASE-11474: SUCCESS: Integrated in HBase-TRUNK #5295 (See [https://builds.apache.org/job/HBase-TRUNK/5295/]) HBASE-11474 [Thrift2] support authentication/impersonation (jxiang: rev fc573634041242c322c29cff722eb0d74d3ab20f) * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java * hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandler.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java * hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithLabels.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftHBaseServiceHandler.java > [Thrift2] support authentication/impersonation > -- > > Key: HBASE-11474 > URL: https://issues.apache.org/jira/browse/HBASE-11474 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: hbase-11474.patch, hbase-11474_1.patch > > > This is to do the same as HBASE-11349 for Thrift2. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11474) [Thrift2] support authentication/impersonation
[ https://issues.apache.org/jira/browse/HBASE-11474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060259#comment-14060259 ] Hudson commented on HBASE-11474: FAILURE: Integrated in HBase-0.98 #392 (See [https://builds.apache.org/job/HBase-0.98/392/]) HBASE-11474 [Thrift2] support authentication/impersonation (jxiang: rev 9cf54eee53e184b78ff05b0c6d2485f062c8cb83) * hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithLabels.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftHBaseServiceHandler.java * hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java > [Thrift2] support authentication/impersonation > -- > > Key: HBASE-11474 > URL: https://issues.apache.org/jira/browse/HBASE-11474 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: hbase-11474.patch, hbase-11474_1.patch > > > This is to do the same as HBASE-11349 for Thrift2. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[
https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060246#comment-14060246
]
Hadoop QA commented on HBASE-11400:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655480/HBASE-11400-4.patch
against trunk revision .
ATTACHMENT ID: 12655480
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10049//console
This message is automatically generated.
> Edit, consolidate, and update Compression and data encoding docs
>
>
> Key: HBASE-11400
> URL: https://issues.apache.org/jira/browse/HBASE-11400
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
>Priority: Minor
> Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch,
> HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch,
> data_block_diff_encoding.png, data_block_no_encoding.png,
> data_block_prefix_encoding.png
>
>
> Current docs are here: http://hbase.apache.org/book.html#compression.test
> It could use some editing and expansion.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-4.patch Regenerated version 4 of the patch because I had mistakenly removed the modifications to pom.xml for image support. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch, > data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: (was: HBASE-11400-4.patch) > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400.patch, data_block_diff_encoding.png, > data_block_no_encoding.png, data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[
https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060242#comment-14060242
]
Hadoop QA commented on HBASE-11400:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655476/HBASE-11400-4.patch
against trunk revision .
ATTACHMENT ID: 12655476
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10048//console
This message is automatically generated.
> Edit, consolidate, and update Compression and data encoding docs
>
>
> Key: HBASE-11400
> URL: https://issues.apache.org/jira/browse/HBASE-11400
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
>Priority: Minor
> Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch,
> HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch,
> data_block_diff_encoding.png, data_block_no_encoding.png,
> data_block_prefix_encoding.png
>
>
> Current docs are here: http://hbase.apache.org/book.html#compression.test
> It could use some editing and expansion.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: data_block_diff_encoding.png data_block_no_encoding.png data_block_prefix_encoding.png HBASE-11400-4.patch Got permission from Matteo to use his images, and also got some clarification on why an image is not needed for Fast Diff and is difficult to do for Prefix Tree. The attached images belong in the src/main/docbkx/images/ directory if that is not clear from the diff itself. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch, > data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060239#comment-14060239 ] Misty Stanley-Jones commented on HBASE-11502: - Yes it might be cleaner if you do it in the relevant JIRA than here. > Track down broken images in Ref Guide > - > > Key: HBASE-11502 > URL: https://issues.apache.org/jira/browse/HBASE-11502 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > I realized that image support was broken in the Ref Guide. I fixed it by > making some changes to the pom.xml (in HBASE-11400). I need to track down > what images are broken besides the new ones I added, find out where they are, > and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[
https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060238#comment-14060238
]
Hadoop QA commented on HBASE-11400:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655475/HBASE-11400-3.patch
against trunk revision .
ATTACHMENT ID: 12655475
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10047//console
This message is automatically generated.
> Edit, consolidate, and update Compression and data encoding docs
>
>
> Key: HBASE-11400
> URL: https://issues.apache.org/jira/browse/HBASE-11400
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
>Priority: Minor
> Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch,
> HBASE-11400-3.patch, HBASE-11400.patch
>
>
> Current docs are here: http://hbase.apache.org/book.html#compression.test
> It could use some editing and expansion.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-3.patch Added a section to talk about implications of https://issues.apache.org/jira/browse/HBASE-6370. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400.patch > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060222#comment-14060222
]
Hadoop QA commented on HBASE-11492:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655470/11492.v2.patch
against trunk revision .
ATTACHMENT ID: 12655470
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileGetting(TestHRegion.java:3504)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10046//console
This message is automatically generated.
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sl
[jira] [Commented] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[ https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060221#comment-14060221 ] Ted Yu commented on HBASE-11507: I disagree that the effort by Gustavo of preventing regression w.r.t. HBASE-8 is trivial. > Enhance test-patch.sh to check for direct calls to > HBaseZeroCopyByteString.wrap() > - > > Key: HBASE-11507 > URL: https://issues.apache.org/jira/browse/HBASE-11507 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly >Priority: Trivial > Attachments: HBASE-11507.patch > > > HBaseZeroCopyByteString.wrap() should not be called directly. > ByteStringer.wrap() should be called instead. > This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to > test-patch.sh. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[
https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060220#comment-14060220
]
Hadoop QA commented on HBASE-11507:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655469/HBASE-11507.patch
against trunk revision .
ATTACHMENT ID: 12655469
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
warnings=`$GREP 'new TreeMaphttps://builds.apache.org/job/PreCommit-HBASE-Build/10045//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10045//console
This message is automatically generated.
> Enhance test-patch.sh to check for direct calls to
> HBaseZeroCopyByteString.wrap()
> -
>
> Key: HBASE-11507
> URL: https://issues.apache.org/jira/browse/HBASE-11507
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
>Priority: Trivial
> Attachments: HBASE-11507.patch
>
>
> HBaseZeroCopyByteString.wrap() should not be called directly.
> ByteStringer.wrap() should be called instead.
> This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to
> test-patch.sh.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[ https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060218#comment-14060218 ] stack commented on HBASE-11507: --- [~gustavoanatoly] Anything that helps with the hbase 1.0 project would be appreciated. See HBASE-10856 for its list of items to do and linked issues. If that list has nought of interest just say and can dig up something else. Write me offline if you'd like input or help, np. bq. What a weird issue 'weird' is wrong in above. The patch attached could be of some, rare 'utility'. Grepping 'HBaseZeroCopyByteString\.wrap()' seems wrong though given the wrap takes args and there is also zeroCopyGetBytes I'd think we'd at least start by adding warning javadoc on the objectional class itself discouraging its use with why If out to build yourself a track record [~gustavoanatoly], yeah, take on something more substantial. Good on you. > Enhance test-patch.sh to check for direct calls to > HBaseZeroCopyByteString.wrap() > - > > Key: HBASE-11507 > URL: https://issues.apache.org/jira/browse/HBASE-11507 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly >Priority: Trivial > Attachments: HBASE-11507.patch > > > HBaseZeroCopyByteString.wrap() should not be called directly. > ByteStringer.wrap() should be called instead. > This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to > test-patch.sh. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-11492:
Attachment: 11492.v2.patch
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-11492:
Status: Open (was: Patch Available)
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060217#comment-14060217
]
Nicolas Liochon commented on HBASE-11492:
-
bq. So here by we rename all the configs ipc.* to hbase.ipc.* Great.!
Yeah, small issue big consequences :-)
+1 for 0.98 as well.
I'm re-triggering hadoop qa for trunk.
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-11492:
Status: Patch Available (was: Open)
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[ https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060216#comment-14060216 ] Gustavo Anatoly commented on HBASE-11507: - Hi, [~stack]. I would like to work with some substance and import. Do you have some task? Thanks. > Enhance test-patch.sh to check for direct calls to > HBaseZeroCopyByteString.wrap() > - > > Key: HBASE-11507 > URL: https://issues.apache.org/jira/browse/HBASE-11507 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly >Priority: Trivial > Attachments: HBASE-11507.patch > > > HBaseZeroCopyByteString.wrap() should not be called directly. > ByteStringer.wrap() should be called instead. > This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to > test-patch.sh. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[ https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Anatoly updated HBASE-11507: Attachment: HBASE-11507.patch > Enhance test-patch.sh to check for direct calls to > HBaseZeroCopyByteString.wrap() > - > > Key: HBASE-11507 > URL: https://issues.apache.org/jira/browse/HBASE-11507 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly >Priority: Trivial > Attachments: HBASE-11507.patch > > > HBaseZeroCopyByteString.wrap() should not be called directly. > ByteStringer.wrap() should be called instead. > This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to > test-patch.sh. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[ https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11507: -- Priority: Trivial (was: Major) What a weird issue. Marking trivial. [~gustavoanatoly], if you want to work on something with some substance and import, give me a shout and I'll set you right. > Enhance test-patch.sh to check for direct calls to > HBaseZeroCopyByteString.wrap() > - > > Key: HBASE-11507 > URL: https://issues.apache.org/jira/browse/HBASE-11507 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly >Priority: Trivial > > HBaseZeroCopyByteString.wrap() should not be called directly. > ByteStringer.wrap() should be called instead. > This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to > test-patch.sh. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[ https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Anatoly updated HBASE-11507: Attachment: (was: HBASE-11507.patch) > Enhance test-patch.sh to check for direct calls to > HBaseZeroCopyByteString.wrap() > - > > Key: HBASE-11507 > URL: https://issues.apache.org/jira/browse/HBASE-11507 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly > > HBaseZeroCopyByteString.wrap() should not be called directly. > ByteStringer.wrap() should be called instead. > This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to > test-patch.sh. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[ https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060210#comment-14060210 ] Ted Yu commented on HBASE-11507: QA only accepts level 1 patch. Please regenerate patch. > Enhance test-patch.sh to check for direct calls to > HBaseZeroCopyByteString.wrap() > - > > Key: HBASE-11507 > URL: https://issues.apache.org/jira/browse/HBASE-11507 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Attachments: HBASE-11507.patch > > > HBaseZeroCopyByteString.wrap() should not be called directly. > ByteStringer.wrap() should be called instead. > This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to > test-patch.sh. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[
https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060209#comment-14060209
]
Hadoop QA commented on HBASE-11507:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655467/HBASE-11507.patch
against trunk revision .
ATTACHMENT ID: 12655467
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10044//console
This message is automatically generated.
> Enhance test-patch.sh to check for direct calls to
> HBaseZeroCopyByteString.wrap()
> -
>
> Key: HBASE-11507
> URL: https://issues.apache.org/jira/browse/HBASE-11507
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
> Attachments: HBASE-11507.patch
>
>
> HBaseZeroCopyByteString.wrap() should not be called directly.
> ByteStringer.wrap() should be called instead.
> This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to
> test-patch.sh.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[ https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Anatoly updated HBASE-11507: Attachment: HBASE-11507.patch > Enhance test-patch.sh to check for direct calls to > HBaseZeroCopyByteString.wrap() > - > > Key: HBASE-11507 > URL: https://issues.apache.org/jira/browse/HBASE-11507 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Attachments: HBASE-11507.patch > > > HBaseZeroCopyByteString.wrap() should not be called directly. > ByteStringer.wrap() should be called instead. > This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to > test-patch.sh. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11507) Enhance test-patch.sh to check for direct calls to HBaseZeroCopyByteString.wrap()
[ https://issues.apache.org/jira/browse/HBASE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Anatoly updated HBASE-11507: Status: Patch Available (was: Open) > Enhance test-patch.sh to check for direct calls to > HBaseZeroCopyByteString.wrap() > - > > Key: HBASE-11507 > URL: https://issues.apache.org/jira/browse/HBASE-11507 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Attachments: HBASE-11507.patch > > > HBaseZeroCopyByteString.wrap() should not be called directly. > ByteStringer.wrap() should be called instead. > This task adds check for direct calls to HBaseZeroCopyByteString.wrap() to > test-patch.sh. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11437) Modify cell tag handling code to treat the length as unsigned
[ https://issues.apache.org/jira/browse/HBASE-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060174#comment-14060174 ] Hudson commented on HBASE-11437: SUCCESS: Integrated in HBase-TRUNK #5294 (See [https://builds.apache.org/job/HBase-TRUNK/5294/]) HBASE-11437 Modify cell tag handling code to treat the length as unsigned. (Anoop) (anoopsamjohn: rev 213e565bced01e8d4ffe5bb3b76d2a7026533aec) * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV3.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV3.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java * hbase-common/src/test/java/org/apache/hadoop/hbase/io/TestTagCompressionContext.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/TagCompressionContext.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java * hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java * hbase-common/src/main/java/org/apache/hadoop/hbase/Cell.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java * hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeArrayScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityUtils.java * hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestByteRangeWithKVSerialization.java * hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeCell.java * hbase-common/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV3.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTags.java * hbase-client/src/test/java/org/apache/hadoop/hbase/ipc/TestPayloadCarryingRpcController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java > Modify cell tag handling code to treat the length as unsigned > - > > Key: HBASE-11437 > URL: https://issues.apache.org/jira/browse/HBASE-11437 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 0.98.4, 2.0.0 > > Attachments: HBASE-11437.patch, HBASE-11437.patch, HBASE-11437.patch, > HBASE-11437_0.98.patch, HBASE-11437_0.98_V2.patch, HBASE-11437_trunk.patch > > > We store each tag's length and total tags length with 2 bytes in KeyValue > buffer, HFiles etc and we treat these lengths as short through out the code. > So the max length can be Short.MAX_VALUE. We can treat these lengths as > unsigned and +ve always. So we can actually treat these lengths as int and > store with 2 bytes so that the max length can reach 65535. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11437) Modify cell tag handling code to treat the length as unsigned
[ https://issues.apache.org/jira/browse/HBASE-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060064#comment-14060064 ] Hudson commented on HBASE-11437: FAILURE: Integrated in HBase-0.98 #391 (See [https://builds.apache.org/job/HBase-0.98/391/]) HBASE-11437 Modify cell tag handling code to treat the length as unsigned. (Anoop) (anoopsamjohn: rev f2d16e0a8cf6fc349216ffcc9c2b6eabd3a99c0d) * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV3.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityUtils.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeCell.java * hbase-common/src/test/java/org/apache/hadoop/hbase/codec/TestCellCodecWithTags.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityScanDeleteTracker.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * hbase-common/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeArrayScanner.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/KeyValueCompression.java * hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWithTags.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SecureWALCellCodec.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java * hbase-client/src/test/java/org/apache/hadoop/hbase/ipc/TestPayloadCarryingRpcController.java * hbase-common/src/test/java/org/apache/hadoop/hbase/codec/TestKeyValueCodecWithTags.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTags.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestPrefixTreeEncoding.java * hbase-common/src/main/java/org/apache/hadoop/hbase/codec/CellCodecWithTags.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/TagCompressionContext.java * hbase-common/src/main/java/org/apache/hadoop/hbase/Cell.java * hbase-common/src/test/java/org/apache/hadoop/hbase/io/TestTagCompressionContext.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftUtilities.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV3.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV3.java > Modify cell tag handling code to treat the length as unsigned > - > > Key: HBASE-11437 > URL: https://issues.apache.org/jira/browse/HBASE-11437 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 0.98.4, 2.0.0 > > Attachments: HBASE-11437.patch, HBASE-11437.patch, HBASE-11437.patch, > HBASE-11437_0.98.patch, HBASE-11437_0.98_V2.patch, HBASE-11437_trunk.patch > > > We store each tag's length and total tags length with 2 bytes in KeyValue > buffer, HFiles etc and we treat these lengths as short through out the code. > So the max length can be Short.MAX_VALUE. We can treat these lengths as > unsigned and +ve always. So we can actually treat these lengths as int and > store with 2 bytes so that the max length can reach 65535. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11437) Modify cell tag handling code to treat the length as unsigned
[ https://issues.apache.org/jira/browse/HBASE-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060062#comment-14060062 ] Hudson commented on HBASE-11437: SUCCESS: Integrated in HBase-1.0 #33 (See [https://builds.apache.org/job/HBase-1.0/33/]) HBASE-11437 Modify cell tag handling code to treat the length as unsigned. (Anoop) (anoopsamjohn: rev 42a7be1032d72791b7df2ea4d987fe35e6c6af19) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * hbase-common/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeCell.java * hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeArrayScanner.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV3.java * hbase-common/src/test/java/org/apache/hadoop/hbase/io/TestTagCompressionContext.java * hbase-client/src/test/java/org/apache/hadoop/hbase/ipc/TestPayloadCarryingRpcController.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV3.java * hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestByteRangeWithKVSerialization.java * hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java * hbase-common/src/main/java/org/apache/hadoop/hbase/Cell.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityUtils.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTags.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV3.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/TagCompressionContext.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java > Modify cell tag handling code to treat the length as unsigned > - > > Key: HBASE-11437 > URL: https://issues.apache.org/jira/browse/HBASE-11437 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 0.98.4, 2.0.0 > > Attachments: HBASE-11437.patch, HBASE-11437.patch, HBASE-11437.patch, > HBASE-11437_0.98.patch, HBASE-11437_0.98_V2.patch, HBASE-11437_trunk.patch > > > We store each tag's length and total tags length with 2 bytes in KeyValue > buffer, HFiles etc and we treat these lengths as short through out the code. > So the max length can be Short.MAX_VALUE. We can treat these lengths as > unsigned and +ve always. So we can actually treat these lengths as int and > store with 2 bytes so that the max length can reach 65535. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11437) Modify cell tag handling code to treat the length as unsigned
[ https://issues.apache.org/jira/browse/HBASE-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060061#comment-14060061 ] Hudson commented on HBASE-11437: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #371 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/371/]) HBASE-11437 Modify cell tag handling code to treat the length as unsigned. (Anoop) (anoopsamjohn: rev f2d16e0a8cf6fc349216ffcc9c2b6eabd3a99c0d) * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV3.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java * hbase-common/src/test/java/org/apache/hadoop/hbase/codec/TestKeyValueCodecWithTags.java * hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeArrayScanner.java * hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestPrefixTreeEncoding.java * hbase-common/src/main/java/org/apache/hadoop/hbase/codec/CellCodecWithTags.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV3.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityUtils.java * hbase-common/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java * hbase-common/src/test/java/org/apache/hadoop/hbase/io/TestTagCompressionContext.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityScanDeleteTracker.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/TagCompressionContext.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/KeyValueCompression.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelFilter.java * hbase-client/src/test/java/org/apache/hadoop/hbase/ipc/TestPayloadCarryingRpcController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTags.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java * hbase-common/src/main/java/org/apache/hadoop/hbase/Cell.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV3.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeCell.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SecureWALCellCodec.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWithTags.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java * hbase-common/src/test/java/org/apache/hadoop/hbase/codec/TestCellCodecWithTags.java * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftUtilities.java > Modify cell tag handling code to treat the length as unsigned > - > > Key: HBASE-11437 > URL: https://issues.apache.org/jira/browse/HBASE-11437 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 0.98.4, 2.0.0 > > Attachments: HBASE-11437.patch, HBASE-11437.patch, HBASE-11437.patch, > HBASE-11437_0.98.patch, HBASE-11437_0.98_V2.patch, HBASE-11437_trunk.patch > > > We store each tag's length and total tags length with 2 bytes in KeyValue > buffer, HFiles etc and we treat these lengths as short through out the code. > So the max length can be Short.MAX_VALUE. We can treat these lengths as > unsigned and +ve always. So we can actually treat these lengths as int and > store with 2 bytes so that the max length can reach 65535. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11360) SnapshotFileCache refresh logic based on modified directory time might be insufficient
[ https://issues.apache.org/jira/browse/HBASE-11360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060057#comment-14060057 ] Lars Hofhansl commented on HBASE-11360: --- Where are we with this. Is somebody volunteering for a trunk patch. Won't be completely trivial as the logic is different in trunk. (I can see if I have time, but I'm on vacation still... Not sure when I'll get to it) > SnapshotFileCache refresh logic based on modified directory time might be > insufficient > -- > > Key: HBASE-11360 > URL: https://issues.apache.org/jira/browse/HBASE-11360 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.19 >Reporter: churro morales > Fix For: 0.99.0, 0.98.5, 0.94.22 > > Attachments: 11360-0.98.txt, HBASE-11360-0.94.patch, > HBASE-11360-0.96.patch > > > Right now we decide whether to refresh the cache based on the lastModified > timestamp of all the snapshots and those "running" snapshots which is located > in the /hbase/.hbase-snapshot/.tmp/ directory > We ran a ExportSnapshot job which takes around 7 minutes between creating the > directory and copying all the files. > Thus the modified time for the > /hbase/.hbase-snapshot/.tmp directory was 7 minutes earlier than the modified > time of the > /hbase/.hbase-snapshot/.tmp/ directory > Thus the cache refresh happens and doesn't pick up all the files but thinks > its up to date as the modified time of the .tmp directory never changes. > This is a bug as when the export job starts the cache never contains the > files for the "running" snapshot and will fail. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060056#comment-14060056
]
Lars Hofhansl commented on HBASE-11492:
---
Let's not change this in 0.94, as folks might have configured these options
already - might be an issue in 0.98 as well.
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
