lfrancke commented on a change in pull request #14: TRAINING-13: Apache ZooKeeper slides URL: https://github.com/apache/incubator-training/pull/14#discussion_r288745708
########## File path: content/ZooKeeper/src/main/asciidoc/index_en.adoc ########## @@ -0,0 +1,129 @@ +//// + + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + +//// + + +[%notitle] +== Apache ZooKeeper +:description: Brief introduction to Apache ZooKeeper +:keywords: Apache ZooKeeper + +image::http://www.apache.org/logos/res/zookeeper/zookeeper.png[] + + +== Introduction + +ZooKeeper is a distributed, highly available, scalable and strictly consistent hierarchical data store + + +== Introduction +* Current version is 3.5.5 (May 2019) +** Released after five years of development +** Major new features: Dynamic Reconfiguration, "container" znode, more later +* Other maintained branch is 3.4 with 3.4.14 (April 2019) +* One of the first tools from the Hadoop ecosystem +* Built at Yahoo!, now an Apache project + + +== Use Cases +* Foundation for many features in the Hadoop ecosystem +** HA (HBase, YARN, Hive, …) +** Coordination (HBase, ...) +* "Recipes" easily implemented using ZooKeeper: +** Group Membership, Name Service, Configuration, Barriers, Queues, Locks, Leader Election, Two-phased commit +* While ZooKeeper originated within the Hadoop ecosystem it is used heavily outside of it as well +** e.g. Solr and others + + +== Data Model +image::data-model.png[] + + +== Data Model +* Hierarchy of nodes (called znode) +** Similar to typical file systems +* Each node can "contain" other nodes as well as data +** Different from file systems where a node is *either* a file *or* a directory +* Data in a node is usually small +** In the kilobyte range + + +== Data Model +* As all updates are strictly ordered and only a single master processes writes ZooKeeper is not meant as a high-volume data store +* ZooKeeper knows several different types of nodes +* A connection to ZooKeeper is called a "session" and is also assigned an id +** Connection setup relatively expensive because it also requires a vote +* znodes can be watched (i.e. notifications on changes) + +== Node Types +* Ephemeral nodes +** Ephemeral nodes get deleted automatically once the session is closed in which it was created +** Can be used to get notified of failed servers (using watches) +* Persistent nodes +** These will not be deleted when the session ends + + +== Node Types +* Time to live (TTL) nodes (as of 3.5) +** When creating a node a TTL value (in ms) can be specified +** When the node has not been modified within the TTL and there are no children it may also be deleted automatically +* Container nodes (as of 3.5) +** When all child nodes have been deleted the container node may also be deleted automatically at some point + + +== Node Types +* Ephemeral, Persistent and TTL nodes can additionally be _sequential_ +** This will automatically append a monotonically increasing number to the end of the node name + + +== Implementation Details +* ZooKeeper uses a wire protocol based on a library called "Jute" +** Extracted from Hadoop +** Not used outside of ZooKeeper +* There are native clients for C and Java +** Other (e.g. Python) clients maintained by the community +* Apache Curator is a Java based client library that is often easier to use than the native one as it offers higher level concepts + + +== Zab Protocol +* ZooKeeper can use multiple servers in what's called an "Ensemble" +* ZooKeeper uses an algorithm called Zab to guarantee reliable delivery, total and causal order of messages in the face of unreliable networks +* In an ensemble there is at most one leader server supported by a majority of followers Review comment: Out of curiosity....what's a use-case for this feature? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
