bq. ...but if we are recommending HLC for user tables, we have to also do end-to-end testing to demonstrate that perf is not affected by HLC. Agree. I think the only way of doing it right now is running ycsb with/without HLC. If it highlights an issue, it means there is one (there can't be false -ves). But if it doesn't, it doesn't mean there isn't one (there can be false +ve). But i think it's a reasonable way of testing. Any other ideas?
On Fri, Aug 25, 2017 at 2:36 PM, Enis Söztutar <[email protected]> wrote: > HLC will never be able to match the throughput for the system clock of > course. I was expecting the perf to be in the 2-3x slower range, not more > than an order of magnitude difference. The current perf is definitely > acceptable for meta updates, but if we are recommending HLC for user > tables, we have to also do end-to-end testing to demonstrate that perf is > not affected by HLC. > > Enis > > On Fri, Aug 25, 2017 at 2:00 PM, Apekshit Sharma <[email protected]> > wrote: > > > It's true that hlc clock is very slower than system, but is it really a > > bottleneck? I mean 7mil ops/sec with 16 threads doesn't sound bottleneck > > because: > > - It's like 70 times of any RS' throughput (assuming 100k write > > throughput) > > - It's certainly enough for meta operations. > > What say? > > > > But i agree that we should try to make it better and file a jira for > > followup. > > > > > > On Fri, Aug 25, 2017 at 11:53 AM, Enis Söztutar <[email protected]> > > wrote: > > > > > Indeed it seems like a perf improvement in the code HLC clock is needed > > > before the merge. Do you mind creating a sub ticket for this. > > > > > > Enis > > > > > > On Fri, Aug 25, 2017 at 10:54 AM, Amit Patel <[email protected]> > > > wrote: > > > > > > > Just as an update with regard to performance: getting the time from > or > > > > updating the hybrid logical clock is quite expensive (see clocks > > > > microbenchmark > > > > <https://issues.apache.org/jira/browse/HBASE-18542? > > > > focusedCommentId=16141932&page=com.atlassian.jira. > > > > plugin.system.issuetabpanels:comment-tabpanel#comment-16141932>) > > > > to the extent that it would be a blocker for this work. There is > > however > > > > certainly room for improvement given that the current implementation > > uses > > > > synchronized blocks for both Clock#now and Clock#update. > > > > > > > > On Wed, Aug 23, 2017 at 6:18 PM, Enis Söztutar <[email protected]> > > wrote: > > > > > > > > > Thanks Amit for continuing on the work. > > > > > > > > > > I am biased, but of course would like to see this merged back to > > master > > > > > (and possibly make it to 2.0) once the issues are ironed out. We > can > > > > > eliminate a very large class of problems with a mathematically > sound > > > > > strategy. > > > > > > > > > > Enis > > > > > > > > > > On Tue, Aug 22, 2017 at 1:28 PM, Stack <[email protected]> wrote: > > > > > > > > > > > Look at the pretty HLC timestamps in meta after a run: > > > > > > > > > > > > hbase:meta > > > > > > column=table:state, timestamp=1576463246879096833, > > > > > value=\x08\x00 > > > > > > hbase:namespace > > > > > > column=table:state, timestamp=1576460369910562817, > > > > > > value=\x08\x00 > > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec > b5. > > > > > > column=info:regioninfo, timestamp=1576462593304821761, > > > > > > value={ENCODED => 6d365281d5ca8ee0bc0fa12cdf33ecb5, NAME => > > > > > > 'hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec > > b5.', > > > > > > STARTKEY => '', ENDKEY => ''} > > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec > b5. > > > > > > column=info:seqnumDuringOpen, > > > timestamp=1576462593304821761, > > > > > > value=\x00\x00\x00\x00\x00\x00\x00\x15 > > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec > b5. > > > > > > column=info:server, timestamp=1576462593304821761, > value= > > > > > > ve0536.halxg.cloudera.com:16020 > > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec > b5. > > > > > > column=info:serverstartcode, > > timestamp=1576462593304821761, > > > > > > value=1503431013602 > > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec > b5. > > > > > > column=info:sn, timestamp=1576462593050017795, value= > > > > > > ve0536.halxg.cloudera.com,16020,1503431013602 > > > > > > hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec > b5. > > > > > > column=info:state, timestamp=1576462593304821761, > > > value=OPEN > > > > > > > > > > > > St.Ack > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Aug 22, 2017 at 1:26 PM, Stack <[email protected]> wrote: > > > > > > > > > > > > > Nice work Amit. Idea of merging sounds good to me. Our metadata > > > > system > > > > > > > will run w/ a new rigor as HLC allows us leave behind a whole > > class > > > > of > > > > > > > issues. > > > > > > > > > > > > > > I ran the HLC branch on small cluster for a while w/ chaos. The > > > ITBLL > > > > > job > > > > > > > passed so basically works I'd say. +1 from me on merge. > > > > > > > > > > > > > > St.Ack > > > > > > > > > > > > > > On Mon, Aug 21, 2017 at 1:29 PM, Amit Patel < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > >> Hi everyone, > > > > > > >> > > > > > > >> I’d like to begin the discussion of merging the current HLC > work > > > > with > > > > > > >> master. Note that the current work is not ready to merge yet > > since > > > > > there > > > > > > >> are a few remaining issues (see below). For reference, > > HBASE-14070 > > > > > > >> <https://issues.apache.org/jira/browse/HBASE-14070> proposes > > > using > > > > > > hybrid > > > > > > >> logical clocks, which combine both logical and physical clocks > > to > > > > > > capture > > > > > > >> not only causality relationship between events, but also > > physical > > > > time > > > > > > at > > > > > > >> which events occur. HLC can be enabled on a per-table basis > and > > at > > > > the > > > > > > >> moment just the meta table has HLC enabled by default (the > other > > > > > tables > > > > > > >> default to use the system clock). An initial design document > > > created > > > > > by > > > > > > >> Enis Soztutar can be found here > > > > > > >> <https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LD > > > > > > >> T4e_bXy8P9h6kWC05Bhw/edit#>. > > > > > > >> The more recent status document created by Sai Teja Ranuva can > > be > > > > > found > > > > > > >> here > > > > > > >> <https://docs.google.com/document/d/1n32DUKoL3LSoKQ1_NuF8TbQ > > > > > > >> kZiZCwFhbHZ7JXRjg3oA/edit#heading=h.881hwl2sdbvx>. > > > > > > >> > > > > > > >> > > > > > > >> Changes > > > > > > >> > > > > > > >> The new classes that would be introduced include: a Clock > > > interface > > > > > with > > > > > > >> three clock implementations mentioned above and enums for the > > > clock > > > > > and > > > > > > >> timestamp type. The three clock implementations are hybrid > > > logical, > > > > > > >> system > > > > > > >> monotonic, and system clocks and each generate 64-bit > > timestamps. > > > > > > >> Timestamps from the system monotonic and system clock > represent > > > > > physical > > > > > > >> time and are directly backwards compatible. > > > > > > >> > > > > > > >> Users can set the clock type of a table during table > creation. A > > > > > table’s > > > > > > >> clock type cannot be changed after the table has been created. > > For > > > > > > tables > > > > > > >> whose clock type is HLC, users would not be permitted to set > > their > > > > own > > > > > > >> timestamps for mutations. Certain events such as region > > > assignment, > > > > > > >> unassignment, and recovery update the clocks upon receiving > > > > timestamps > > > > > > >> externally. > > > > > > >> > > > > > > >> Benefits > > > > > > >> > > > > > > >> > > > > > > >> - > > > > > > >> > > > > > > >> Hybrid logical clocks (and even system monotonic clocks) > can > > > > solve > > > > > a > > > > > > >> variety of long standing issues related to time highlighted > > > here > > > > > > >> <https://docs.google.com/document/d/ > 1LL2GAodiYi0waBz5ODGL4L > > > > > > >> DT4e_bXy8P9h6kWC05Bhw/edit#heading=h.msabldfs8oly>. > > > > > > >> > > > > > > >> - > > > > > > >> > > > > > > >> Future work with hybrid logical clocks would include > enabling > > > > them > > > > > > with > > > > > > >> user tables. Ideally we would want HLC to be used > everywhere. > > > > > Further > > > > > > >> work > > > > > > >> such as global point-in-time snapshots can leverage HLC > > > > > > >> > > > > > > >> > > > > > > >> Remaining issues > > > > > > >> > > > > > > >> There are still a few remaining issues to close out before > > > actually > > > > > > >> merging, but I wanted to at least start the discussion. > > Currently > > > I > > > > am > > > > > > >> working on fixing remaining tests that are either failing or > > > timing > > > > > out > > > > > > in > > > > > > >> the public branch <https://builds.apache.org/ > > job/HBASE-14070.HLC/ > > > > > > > > as > > > > > > >> well > > > > > > >> as doing local performance tests, and cluster tests. Some of > the > > > > > > remaining > > > > > > >> issues we are tracking are: > > > > > > >> > > > > > > >> > > > > > > >> - > > > > > > >> > > > > > > >> HBASE-18542 (Perf numbers) > > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18542> > > > > > > >> - > > > > > > >> > > > > > > >> HBASE-18508 (Fixing unit tests) > > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18508> > > > > > > >> - > > > > > > >> > > > > > > >> HBASE-18432 (Clock skew) > > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18432> > > > > > > >> > > > > > > >> Some brainstorming/discussions we also are tracking: > > > > > > >> > > > > > > >> - > > > > > > >> > > > > > > >> (HBASE-18643) Adding transaction id to Result > > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18643> > > > > > > >> - > > > > > > >> > > > > > > >> (HBASE-18642) Deprecate setting of timestamp in client for > > HLC > > > > > > >> <https://issues.apache.org/jira/browse/HBASE-18642> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > -- Appy > > > -- -- Appy
