Hi everyone, I’d like to begin the discussion of merging the current HLC work with master. Note that the current work is not ready to merge yet since there are a few remaining issues (see below). For reference, HBASE-14070 <https://issues.apache.org/jira/browse/HBASE-14070> proposes using hybrid logical clocks, which combine both logical and physical clocks to capture not only causality relationship between events, but also physical time at which events occur. HLC can be enabled on a per-table basis and at the moment just the meta table has HLC enabled by default (the other tables default to use the system clock). An initial design document created by Enis Soztutar can be found here <https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit#>. The more recent status document created by Sai Teja Ranuva can be found here <https://docs.google.com/document/d/1n32DUKoL3LSoKQ1_NuF8TbQkZiZCwFhbHZ7JXRjg3oA/edit#heading=h.881hwl2sdbvx>.
Changes The new classes that would be introduced include: a Clock interface with three clock implementations mentioned above and enums for the clock and timestamp type. The three clock implementations are hybrid logical, system monotonic, and system clocks and each generate 64-bit timestamps. Timestamps from the system monotonic and system clock represent physical time and are directly backwards compatible. Users can set the clock type of a table during table creation. A table’s clock type cannot be changed after the table has been created. For tables whose clock type is HLC, users would not be permitted to set their own timestamps for mutations. Certain events such as region assignment, unassignment, and recovery update the clocks upon receiving timestamps externally. Benefits - Hybrid logical clocks (and even system monotonic clocks) can solve a variety of long standing issues related to time highlighted here <https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit#heading=h.msabldfs8oly>. - Future work with hybrid logical clocks would include enabling them with user tables. Ideally we would want HLC to be used everywhere. Further work such as global point-in-time snapshots can leverage HLC Remaining issues There are still a few remaining issues to close out before actually merging, but I wanted to at least start the discussion. Currently I am working on fixing remaining tests that are either failing or timing out in the public branch <https://builds.apache.org/job/HBASE-14070.HLC/> as well as doing local performance tests, and cluster tests. Some of the remaining issues we are tracking are: - HBASE-18542 (Perf numbers) <https://issues.apache.org/jira/browse/HBASE-18542> - HBASE-18508 (Fixing unit tests) <https://issues.apache.org/jira/browse/HBASE-18508> - HBASE-18432 (Clock skew) <https://issues.apache.org/jira/browse/HBASE-18432> Some brainstorming/discussions we also are tracking: - (HBASE-18643) Adding transaction id to Result <https://issues.apache.org/jira/browse/HBASE-18643> - (HBASE-18642) Deprecate setting of timestamp in client for HLC <https://issues.apache.org/jira/browse/HBASE-18642>
