date:20141006

[jira] [Commented] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store

2014-10-06 Thread Lars Hofhansl (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160034#comment-14160034
 ] 

Lars Hofhansl commented on HBASE-12148:
---

Is this correct:
{code}
+  long minimumTimestamp = INITIAL_MINIMUM_TIMESTAMP;
...
+if (timestamp  this.minimumTimestamp) {
+  synchronized (this) {
+if (!init(timestamp)) {
...
+} else if (timestamp  this.maximumTimestamp) {
+  synchronized (this) {
+if (!init(timestamp)) {
...
{code}
?

{{this.minimumTimestamp}} might be in flight of being changed.

Your reasoning is that minimumTimestamp will only get smaller and 
maximumTimestamp only bigger... Hence the check here will not actually miss a 
change? (i.e. if the new value is not smaller than the current minTS it is 
definitely not smaller than any new version of minTS...?)

That might work, but it's risky. On 32bit systems an update to long is not 
atomic in Java, so a reader here might see a partially updated long.

I think you cannot get around declaring minimumTimestamp and maximumTimestamp 
volatile, that will add some of the overhead back in.

[~stack], [~apurtell].


 Remove TimeRangeTracker as point of contention when many threads writing a 
 Store
 

 Key: HBASE-12148
 URL: https://issues.apache.org/jira/browse/HBASE-12148
 Project: HBase
  Issue Type: Sub-task
  Components: Performance
Affects Versions: 2.0.0, 0.99.1
Reporter: stack
Assignee: stack
 Fix For: 2.0.0, 0.98.7, 0.99.1

 Attachments: 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, Screen 
 Shot 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12075) Preemptive Fast Fail

2014-10-06 Thread Lars Hofhansl (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160038#comment-14160038
 ] 

Lars Hofhansl commented on HBASE-12075:
---

From the description this is an implementation of the circuit breaker pattern, 
right?
Anecdotically we had that implemented here as well, but found it problematic 
for various reason and have since replaced with a resource counter/limiter. 
I.e. via a simple semaphore and an acquire/release protocol we simply limit the 
number of threads that use a resource (HTable, HConnection, PhoenixConnection) 
to a number that is acceptable to us.

CircuitBreaker was problematic for various reasons:
# needed to absolutely sure this is a non-recoverable problem
# what if only a few region servers have issues (a) (now need to group 
exception by region server) in order to decide we need to fail other connection
# what if only a few region servers have issues (b) - cluster is not down, yet, 
client threads will hang
# apps created grouping constructs over HTable/HConnection (Phoenix in our 
case), now the circuit breaker got in the way at the wrong times, we need to 
pull it up higher
# (there were more issues, these are off the top of my head)


 Preemptive Fast Fail
 

 Key: HBASE-12075
 URL: https://issues.apache.org/jira/browse/HBASE-12075
 Project: HBase
  Issue Type: Sub-task
  Components: Client
Affects Versions: 0.99.0, 2.0.0, 0.98.6.1
Reporter: Manukranth Kolloju
Assignee: Manukranth Kolloju
 Attachments: 0001-Add-a-test-case-for-Preemptive-Fast-Fail.patch, 
 0001-HBASE-12075-Implement-Preemptive-Fast-Fail.patch, 
 0001-HBASE-12075-Implement-Preemptive-Fast-Fail.patch, 
 0001-Implement-Preemptive-Fast-Fail.patch, 
 0001-Implement-Preemptive-Fast-Fail.patch, 
 0001-Implement-Preemptive-Fast-Fail.patch, 
 0001-Implement-Preemptive-Fast-Fail.patch, 
 0001-Implement-Preemptive-Fast-Fail.patch


 In multi threaded clients, we use a feature developed on 0.89-fb branch 
 called Preemptive Fast Fail. This allows the client threads which would 
 potentially fail, fail fast. The idea behind this feature is that we allow, 
 among the hundreds of client threads, one thread to try and establish 
 connection with the regionserver and if that succeeds, we mark it as a live 
 node again. Meanwhile, other threads which are trying to establish connection 
 to the same server would ideally go into the timeouts which is effectively 
 unfruitful. We can in those cases return appropriate exceptions to those 
 clients instead of letting them retry.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12176) WALCellCodec Encoders support for non-KeyValue Cells

2014-10-06 Thread Anoop Sam John (JIRA)

Anoop Sam John created HBASE-12176:
--

 Summary: WALCellCodec Encoders support for non-KeyValue Cells
 Key: HBASE-12176
 URL: https://issues.apache.org/jira/browse/HBASE-12176
 Project: HBase
  Issue Type: Improvement
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 2.0.0, 0.99.1


Now in all Encoder returned by WALCellCodec and SecureWALCellCodec it is 
assumed that it will get only KeyValues. If not it just throws IOE. Now we 
support Cells through out in write path as well and these Encoders should 
support non KV cells as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12176) WALCellCodec Encoders support for non-KeyValue Cells

2014-10-06 Thread Anoop Sam John (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-12176:
---
Status: Patch Available  (was: Open)

 WALCellCodec Encoders support for non-KeyValue Cells
 

 Key: HBASE-12176
 URL: https://issues.apache.org/jira/browse/HBASE-12176
 Project: HBase
  Issue Type: Improvement
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 2.0.0, 0.99.1

 Attachments: HBASE-12176.patch


 Now in all Encoder returned by WALCellCodec and SecureWALCellCodec it is 
 assumed that it will get only KeyValues. If not it just throws IOE. Now we 
 support Cells through out in write path as well and these Encoders should 
 support non KV cells as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12176) WALCellCodec Encoders support for non-KeyValue Cells

2014-10-06 Thread Anoop Sam John (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-12176:
---
Attachment: HBASE-12176.patch

 WALCellCodec Encoders support for non-KeyValue Cells
 

 Key: HBASE-12176
 URL: https://issues.apache.org/jira/browse/HBASE-12176
 Project: HBase
  Issue Type: Improvement
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 2.0.0, 0.99.1

 Attachments: HBASE-12176.patch


 Now in all Encoder returned by WALCellCodec and SecureWALCellCodec it is 
 assumed that it will get only KeyValues. If not it just throws IOE. Now we 
 support Cells through out in write path as well and these Encoders should 
 support non KV cells as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12176) WALCellCodec Encoders support for non-KeyValue Cells

2014-10-06 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160099#comment-14160099
]

Hadoop QA commented on HBASE-12176:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12673061/HBASE-12176.patch
against trunk revision .
ATTACHMENT ID: 12673061

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:red}-1 core tests{color}. The patch failed these unit tests:

org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11221//console

This message is automatically generated.

WALCellCodec Encoders support for non-KeyValue Cells

Key: HBASE-12176
URL: https://issues.apache.org/jira/browse/HBASE-12176
Project: HBase
Issue Type: Improvement
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 2.0.0, 0.99.1

Attachments: HBASE-12176.patch

Now in all Encoder returned by WALCellCodec and SecureWALCellCodec it is
assumed that it will get only KeyValues. If not it just throws IOE. Now we
support Cells through out in write path as well and these Encoders should
support non KV cells as well.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12176) WALCellCodec Encoders support for non-KeyValue Cells

2014-10-06 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160105#comment-14160105
 ] 

Ted Yu commented on HBASE-12176:


+1

 WALCellCodec Encoders support for non-KeyValue Cells
 

 Key: HBASE-12176
 URL: https://issues.apache.org/jira/browse/HBASE-12176
 Project: HBase
  Issue Type: Improvement
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 2.0.0, 0.99.1

 Attachments: HBASE-12176.patch


 Now in all Encoder returned by WALCellCodec and SecureWALCellCodec it is 
 assumed that it will get only KeyValues. If not it just throws IOE. Now we 
 support Cells through out in write path as well and these Encoders should 
 support non KV cells as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (HBASE-11639) [Visibility controller] Replicate the visibility of Cells as strings

2014-10-06 Thread ramkrishna.s.vasudevan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-11639 started by ramkrishna.s.vasudevan.
--
 [Visibility controller] Replicate the visibility of Cells as strings
 

 Key: HBASE-11639
 URL: https://issues.apache.org/jira/browse/HBASE-11639
 Project: HBase
  Issue Type: Improvement
  Components: Replication, security
Affects Versions: 0.98.4
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
  Labels: VisibilityLabels
 Fix For: 2.0.0, 0.98.7, 0.99.1


 This issue is aimed at persisting the visibility labels as strings in the WAL 
 rather than Label ordinals.  This would help in replicating the label 
 ordinals to the replication cluster as strings directly and also that after 
 HBASE-11553 would help because the replication cluster could have an 
 implementation as string based visibility labels.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11835) Wrong managenement of non expected calls in the client

2014-10-06 Thread Nicolas Liochon (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160384#comment-14160384
 ] 

Nicolas Liochon commented on HBASE-11835:
-

The failures are very likely unrelated. I plan to commit this this week if 
nobody disagrees.

 Wrong managenement of non expected calls in the client
 --

 Key: HBASE-11835
 URL: https://issues.apache.org/jira/browse/HBASE-11835
 Project: HBase
  Issue Type: Bug
  Components: Client
Affects Versions: 1.0.0, 2.0.0, 0.98.6
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
 Fix For: 2.0.0, 0.99.1

 Attachments: 11835.rebase.patch, rpcClient.patch


 If a call is purged or canceled we try to skip the reply from the server, but 
 we read the wrong number of bytes so we corrupt the tcp channel. It's hidden 
 as it triggers retry and so on, but it's bad for performances obviously.
 It happens with cell blocks.
 [~ram_krish_86], [~saint@gmail.com], you know this part better than me, 
 do you agree with the analysis and the patch?
 The changes in rpcServer are not fully related: as the client close the 
 connections in such situation, I observed  both ClosedChannelException and 
 CancelledKeyException. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

1 2 >

1 - 100 of 153 matches

Mail list logo