HLC will never be able to match the throughput for the system clock of course. I was expecting the perf to be in the 2-3x slower range, not more than an order of magnitude difference. The current perf is definitely acceptable for meta updates, but if we are recommending HLC for user tables, we have to also do end-to-end testing to demonstrate that perf is not affected by HLC.
Enis On Fri, Aug 25, 2017 at 2:00 PM, Apekshit Sharma <a...@cloudera.com> wrote: > It's true that hlc clock is very slower than system, but is it really a > bottleneck? I mean 7mil ops/sec with 16 threads doesn't sound bottleneck > because: > - It's like 70 times of any RS' throughput (assuming 100k write > throughput) > - It's certainly enough for meta operations. > What say? > > But i agree that we should try to make it better and file a jira for > followup. > > > On Fri, Aug 25, 2017 at 11:53 AM, Enis Söztutar <enis....@gmail.com> > wrote: > > > Indeed it seems like a perf improvement in the code HLC clock is needed > > before the merge. Do you mind creating a sub ticket for this. > > > > Enis > > > > On Fri, Aug 25, 2017 at 10:54 AM, Amit Patel <amit.pa...@cloudera.com> > > wrote: > > > > > Just as an update with regard to performance: getting the time from or > > > updating the hybrid logical clock is quite expensive (see clocks > > > microbenchmark > > > <https://issues.apache.org/jira/browse/HBASE-18542? > > > focusedCommentId=16141932&page=com.atlassian.jira. > > > plugin.system.issuetabpanels:comment-tabpanel#comment-16141932>) > > > to the extent that it would be a blocker for this work. There is > however > > > certainly room for improvement given that the current implementation > uses > > > synchronized blocks for both Clock#now and Clock#update. > > > > > > On Wed, Aug 23, 2017 at 6:18 PM, Enis Söztutar <e...@apache.org> > wrote: > > > > > > > Thanks Amit for continuing on the work. > > > > > > > > I am biased, but of course would like to see this merged back to > master > > > > (and possibly make it to 2.0) once the issues are ironed out. We can > > > > eliminate a very large class of problems with a mathematically sound > > > > strategy. > > > > > > > > Enis > > > > > > > > On Tue, Aug 22, 2017 at 1:28 PM, Stack <st...@duboce.net> wrote: > > > > > > > > > Look at the pretty HLC timestamps in meta after a run: > > > > > > > > > > hbase:meta > > > > > column=table:state, timestamp=1576463246879096833, > > > > value=\x08\x00 > > > > > hbase:namespace > > > > > column=table:state, timestamp=1576460369910562817, > > > > > value=\x08\x00 > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5. > > > > > column=info:regioninfo, timestamp=1576462593304821761, > > > > > value={ENCODED => 6d365281d5ca8ee0bc0fa12cdf33ecb5, NAME => > > > > > 'hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec > b5.', > > > > > STARTKEY => '', ENDKEY => ''} > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5. > > > > > column=info:seqnumDuringOpen, > > timestamp=1576462593304821761, > > > > > value=\x00\x00\x00\x00\x00\x00\x00\x15 > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5. > > > > > column=info:server, timestamp=1576462593304821761, value= > > > > > ve0536.halxg.cloudera.com:16020 > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5. > > > > > column=info:serverstartcode, > timestamp=1576462593304821761, > > > > > value=1503431013602 > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5. > > > > > column=info:sn, timestamp=1576462593050017795, value= > > > > > ve0536.halxg.cloudera.com,16020,1503431013602 > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ecb5. > > > > > column=info:state, timestamp=1576462593304821761, > > value=OPEN > > > > > > > > > > St.Ack > > > > > > > > > > > > > > > > > > > > On Tue, Aug 22, 2017 at 1:26 PM, Stack <st...@duboce.net> wrote: > > > > > > > > > > > Nice work Amit. Idea of merging sounds good to me. Our metadata > > > system > > > > > > will run w/ a new rigor as HLC allows us leave behind a whole > class > > > of > > > > > > issues. > > > > > > > > > > > > I ran the HLC branch on small cluster for a while w/ chaos. The > > ITBLL > > > > job > > > > > > passed so basically works I'd say. +1 from me on merge. > > > > > > > > > > > > St.Ack > > > > > > > > > > > > On Mon, Aug 21, 2017 at 1:29 PM, Amit Patel < > > amit.pa...@cloudera.com > > > > > > > > > > wrote: > > > > > > > > > > > >> Hi everyone, > > > > > >> > > > > > >> I’d like to begin the discussion of merging the current HLC work > > > with > > > > > >> master. Note that the current work is not ready to merge yet > since > > > > there > > > > > >> are a few remaining issues (see below). For reference, > HBASE-14070 > > > > > >> <https://issues.apache.org/jira/browse/HBASE-14070> proposes > > using > > > > > hybrid > > > > > >> logical clocks, which combine both logical and physical clocks > to > > > > > capture > > > > > >> not only causality relationship between events, but also > physical > > > time > > > > > at > > > > > >> which events occur. HLC can be enabled on a per-table basis and > at > > > the > > > > > >> moment just the meta table has HLC enabled by default (the other > > > > tables > > > > > >> default to use the system clock). An initial design document > > created > > > > by > > > > > >> Enis Soztutar can be found here > > > > > >> <https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LD > > > > > >> T4e_bXy8P9h6kWC05Bhw/edit#>. > > > > > >> The more recent status document created by Sai Teja Ranuva can > be > > > > found > > > > > >> here > > > > > >> <https://docs.google.com/document/d/1n32DUKoL3LSoKQ1_NuF8TbQ > > > > > >> kZiZCwFhbHZ7JXRjg3oA/edit#heading=h.881hwl2sdbvx>. > > > > > >> > > > > > >> > > > > > >> Changes > > > > > >> > > > > > >> The new classes that would be introduced include: a Clock > > interface > > > > with > > > > > >> three clock implementations mentioned above and enums for the > > clock > > > > and > > > > > >> timestamp type. The three clock implementations are hybrid > > logical, > > > > > >> system > > > > > >> monotonic, and system clocks and each generate 64-bit > timestamps. > > > > > >> Timestamps from the system monotonic and system clock represent > > > > physical > > > > > >> time and are directly backwards compatible. > > > > > >> > > > > > >> Users can set the clock type of a table during table creation. A > > > > table’s > > > > > >> clock type cannot be changed after the table has been created. > For > > > > > tables > > > > > >> whose clock type is HLC, users would not be permitted to set > their > > > own > > > > > >> timestamps for mutations. Certain events such as region > > assignment, > > > > > >> unassignment, and recovery update the clocks upon receiving > > > timestamps > > > > > >> externally. > > > > > >> > > > > > >> Benefits > > > > > >> > > > > > >> > > > > > >> - > > > > > >> > > > > > >> Hybrid logical clocks (and even system monotonic clocks) can > > > solve > > > > a > > > > > >> variety of long standing issues related to time highlighted > > here > > > > > >> <https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4L > > > > > >> DT4e_bXy8P9h6kWC05Bhw/edit#heading=h.msabldfs8oly>. > > > > > >> > > > > > >> - > > > > > >> > > > > > >> Future work with hybrid logical clocks would include enabling > > > them > > > > > with > > > > > >> user tables. Ideally we would want HLC to be used everywhere. > > > > Further > > > > > >> work > > > > > >> such as global point-in-time snapshots can leverage HLC > > > > > >> > > > > > >> > > > > > >> Remaining issues > > > > > >> > > > > > >> There are still a few remaining issues to close out before > > actually > > > > > >> merging, but I wanted to at least start the discussion. > Currently > > I > > > am > > > > > >> working on fixing remaining tests that are either failing or > > timing > > > > out > > > > > in > > > > > >> the public branch <https://builds.apache.org/ > job/HBASE-14070.HLC/ > > > > > > as > > > > > >> well > > > > > >> as doing local performance tests, and cluster tests. Some of the > > > > > remaining > > > > > >> issues we are tracking are: > > > > > >> > > > > > >> > > > > > >> - > > > > > >> > > > > > >> HBASE-18542 (Perf numbers) > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18542> > > > > > >> - > > > > > >> > > > > > >> HBASE-18508 (Fixing unit tests) > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18508> > > > > > >> - > > > > > >> > > > > > >> HBASE-18432 (Clock skew) > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18432> > > > > > >> > > > > > >> Some brainstorming/discussions we also are tracking: > > > > > >> > > > > > >> - > > > > > >> > > > > > >> (HBASE-18643) Adding transaction id to Result > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18643> > > > > > >> - > > > > > >> > > > > > >> (HBASE-18642) Deprecate setting of timestamp in client for > HLC > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18642> > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > -- Appy >
