Look at the pretty HLC timestamps in meta after a run:
hbase:meta
column=table:state, timestamp=1576463246879096833, value=\x08\x00
hbase:namespace
column=table:state, timestamp=1576460369910562817, value=\x08\x00
hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5.
column=info:regioninfo, timestamp=1576462593304821761,
value={ENCODED => 6d365281d5ca8ee0bc0fa12cdf33ecb5, NAME =>
'hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5.',
STARTKEY => '', ENDKEY => ''}
hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5.
column=info:seqnumDuringOpen, timestamp=1576462593304821761,
value=\x00\x00\x00\x00\x00\x00\x00\x15
hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5.
column=info:server, timestamp=1576462593304821761, value=
ve0536.halxg.cloudera.com:16020
hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5.
column=info:serverstartcode, timestamp=1576462593304821761,
value=1503431013602
hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5.
column=info:sn, timestamp=1576462593050017795, value=
ve0536.halxg.cloudera.com,16020,1503431013602
hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5.
column=info:state, timestamp=1576462593304821761, value=OPEN
St.Ack
On Tue, Aug 22, 2017 at 1:26 PM, Stack <[email protected]> wrote:
> Nice work Amit. Idea of merging sounds good to me. Our metadata system
> will run w/ a new rigor as HLC allows us leave behind a whole class of
> issues.
>
> I ran the HLC branch on small cluster for a while w/ chaos. The ITBLL job
> passed so basically works I'd say. +1 from me on merge.
>
> St.Ack
>
> On Mon, Aug 21, 2017 at 1:29 PM, Amit Patel <[email protected]>
> wrote:
>
>> Hi everyone,
>>
>> I’d like to begin the discussion of merging the current HLC work with
>> master. Note that the current work is not ready to merge yet since there
>> are a few remaining issues (see below). For reference, HBASE-14070
>> <https://issues.apache.org/jira/browse/HBASE-14070> proposes using hybrid
>> logical clocks, which combine both logical and physical clocks to capture
>> not only causality relationship between events, but also physical time at
>> which events occur. HLC can be enabled on a per-table basis and at the
>> moment just the meta table has HLC enabled by default (the other tables
>> default to use the system clock). An initial design document created by
>> Enis Soztutar can be found here
>> <https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LD
>> T4e_bXy8P9h6kWC05Bhw/edit#>.
>> The more recent status document created by Sai Teja Ranuva can be found
>> here
>> <https://docs.google.com/document/d/1n32DUKoL3LSoKQ1_NuF8TbQ
>> kZiZCwFhbHZ7JXRjg3oA/edit#heading=h.881hwl2sdbvx>.
>>
>>
>> Changes
>>
>> The new classes that would be introduced include: a Clock interface with
>> three clock implementations mentioned above and enums for the clock and
>> timestamp type. The three clock implementations are hybrid logical,
>> system
>> monotonic, and system clocks and each generate 64-bit timestamps.
>> Timestamps from the system monotonic and system clock represent physical
>> time and are directly backwards compatible.
>>
>> Users can set the clock type of a table during table creation. A table’s
>> clock type cannot be changed after the table has been created. For tables
>> whose clock type is HLC, users would not be permitted to set their own
>> timestamps for mutations. Certain events such as region assignment,
>> unassignment, and recovery update the clocks upon receiving timestamps
>> externally.
>>
>> Benefits
>>
>>
>> -
>>
>> Hybrid logical clocks (and even system monotonic clocks) can solve a
>> variety of long standing issues related to time highlighted here
>> <https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4L
>> DT4e_bXy8P9h6kWC05Bhw/edit#heading=h.msabldfs8oly>.
>>
>> -
>>
>> Future work with hybrid logical clocks would include enabling them with
>> user tables. Ideally we would want HLC to be used everywhere. Further
>> work
>> such as global point-in-time snapshots can leverage HLC
>>
>>
>> Remaining issues
>>
>> There are still a few remaining issues to close out before actually
>> merging, but I wanted to at least start the discussion. Currently I am
>> working on fixing remaining tests that are either failing or timing out in
>> the public branch <https://builds.apache.org/job/HBASE-14070.HLC/> as
>> well
>> as doing local performance tests, and cluster tests. Some of the remaining
>> issues we are tracking are:
>>
>>
>> -
>>
>> HBASE-18542 (Perf numbers)
>> <https://issues.apache.org/jira/browse/HBASE-18542>
>> -
>>
>> HBASE-18508 (Fixing unit tests)
>> <https://issues.apache.org/jira/browse/HBASE-18508>
>> -
>>
>> HBASE-18432 (Clock skew)
>> <https://issues.apache.org/jira/browse/HBASE-18432>
>>
>> Some brainstorming/discussions we also are tracking:
>>
>> -
>>
>> (HBASE-18643) Adding transaction id to Result
>> <https://issues.apache.org/jira/browse/HBASE-18643>
>> -
>>
>> (HBASE-18642) Deprecate setting of timestamp in client for HLC
>> <https://issues.apache.org/jira/browse/HBASE-18642>
>>
>
>
