[
https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094013#comment-16094013
]
Appy commented on HBASE-14070:
------------------------------
While reviewing one of the patches, a question popped in my mind:
We are passing master clock to RS in open/close region request and RS clock to
master in the responses. And they both update their own time on receiving these
request/response.
(x= don't care)
Say RS hlc time is (10,x) and master time is (20,x). Master sends request to RS
which then updates its own time to (20,x).
While RS's physical java clock (which is backing up physical component of hlc
clock) will still take 10 sec to catch up, we'll keep incrementing logical
component. That means, in worst case, our logical clock window should be big
enough to support all the events that can happen in max skew time.
The problem is, that doesn't seem to be the case. Our logical window is 1M
events (20bits) and max skew time is 30 sec, that results in 33k max write
qps, which is quite low. We can easily see 150k update qps per beefy server
with 1k values.
Even 22 bits won't be enough. We'll need minimum of 23 bits and 20 sec max skew
time to get to ~420k events per second (worst case).
> Hybrid Logical Clocks for HBase
> -------------------------------
>
> Key: HBASE-14070
> URL: https://issues.apache.org/jira/browse/HBASE-14070
> Project: HBase
> Issue Type: New Feature
> Reporter: Enis Soztutar
> Assignee: Amit Patel
> Attachments: HBASE-14070.master.001.patch,
> HybridLogicalClocksforHBaseandPhoenix.docx,
> HybridLogicalClocksforHBaseandPhoenix.pdf
>
>
> HBase and Phoenix uses systems physical clock (PT) to give timestamps to
> events (read and writes). This works mostly when the system clock is strictly
> monotonically increasing and there is no cross-dependency between servers
> clocks. However we know that leap seconds, general clock skew and clock drift
> are in fact real.
> This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of
> hybrid physical clock + a logical clock. HLC is best of both worlds where it
> keeps causality relationship similar to logical clocks, but still is
> compatible with NTP based physical system clock. HLC can be represented in
> 64bits.
> A design document is attached and also can be found here:
> https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit#
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
