>> Are the patches/improvements in it going to be open sources and possibly backported to ZooKeeper
I'll let folks from FB speak but recent activities of incoming patches is a positive sign. Also I hope ZK would still be used in prod environment at FB so it can got hammered and bugs got exposed. For infrastructure software, there is no better way of testing than testing in prod. The LAD paper does not mention ZK's role in the new system - I suspect it's still there but not on critical path (maybe just for config change notification?). >> by introducing a plugin-based data store replacing current ZkDatabase. I don't think this will make any substantial difference w.r.t performance, if we need keep same client side semantics and API compatibility. Regardless of which state machine implementation it's using, the serving layer (the processor pipelines) is under same ordering constraints. >> So maybe it was originally a use case problem which eventually escalated to a new system design. I feel the same way. Those who use ZK only for leader election and small metadata storage / change notification are very happy. Those who use ZK on their performance critical path all ends up with moving ZK out of the critical path, and questioning the "high availability" part of ZK <grin>. On Fri, Jul 20, 2018 at 1:49 AM, Andor Molnar <[email protected]> wrote: > Great article, thanks Pat. > > There're 2 things which I found particularly interesting: > > - Is Zeus going to be decod as part of the migration? Are the > patches/improvements in it going to be open sources and possibly backported > to ZooKeeper? > > - "Independent data store: ZooKeeper couples a data store with its > distribution framework." > > I believe that decoupling the data store from the coordination framework > wouldn't be too much of a hassle in ZooKeeper by introducing a plugin-based > data store replacing current ZkDatabase. In addition to that I wonder if we > could make client libraries "datastore-aware" letting them serve read > requests directly from the database. e.g. from HDFS data nodes. It might > successfully decouple the load generated by large number of read requests > from writes. > > In which case will ZooKeeper be in any aspects different from an HDFS > cluster? It adds strong consistency and ordering guarantees. > Which parts of ZooKeeper do we still need after such change, because the > distributed data store guarantees a consistent view of the data tree across > participants so what would be the gain of dealing with consensus? > > Going back to Fb's original problem in terms of data/metadata decoupling: > they wanted to distribute large files like 100MB whereas they already 5MB > files present in Zk. I think 5MB is already too much data in a single znode > and ZooKeeper is not designed to deal with such load of data distribution. > So maybe it was originally a use case problem which eventually escalated to > a new system design. > > Propagating metadata updates to millions of subscribers sounds to me more > of an IoT / pubsub problem. Which Facebook has presented a very promising > answer to. Using a tree of subscribers combined with sharding has very > strong scalability potential. > > Dealing with the distribution of large files triggered by metadata change: > bittorrent? :) > > Regards, > Andor > > > > > On Fri, Jul 20, 2018 at 8:07 AM, Patrick Hunt <[email protected]> wrote: > > > "Location-Aware Distribution (LAD), a new peer-to-peer system that > handles > > the distribution of configuration changes to millions of servers." > > https://code.fb.com/data-infrastructure/location-aware- > > distribution-configuring-servers-at-scale/ > > > > Previously Facebookâs configuration management system (called > Configerator) > > used ZooKeeper. SOSP 2015 paper: > > https://research.fb.com/publications/holistic- > configuration-management-at- > > facebook/ > > > > Patrick > > >
