ZOOKEEPER-3022: MAVEN MIGRATION 3.4 - Iteration 1 - docs, it Maven migration - first iteration (zookeeper-docs and zookeeper-it), branch 3.4
Author: Norbert Kalmar <[email protected]> Reviewers: Andor Molnar <[email protected]> Closes #552 from nkalmar/ZOOKEEPER-3022-1_b3.4 Project: http://git-wip-us.apache.org/repos/asf/zookeeper/repo Commit: http://git-wip-us.apache.org/repos/asf/zookeeper/commit/c1efa954 Tree: http://git-wip-us.apache.org/repos/asf/zookeeper/tree/c1efa954 Diff: http://git-wip-us.apache.org/repos/asf/zookeeper/diff/c1efa954 Branch: refs/heads/branch-3.4 Commit: c1efa954d015130a8a01be8334dcf8e7189da7fd Parents: 4a8cceb Author: Norbert Kalmar <[email protected]> Authored: Wed Jul 4 15:11:11 2018 +0200 Committer: Andor Molnar <[email protected]> Committed: Wed Jul 4 15:11:11 2018 +0200 ---------------------------------------------------------------------- build.xml | 2 +- src/docs/forrest.properties | 109 - src/docs/src/documentation/README.txt | 7 - src/docs/src/documentation/TODO.txt | 227 --- .../classes/CatalogManager.properties | 37 - src/docs/src/documentation/conf/cli.xconf | 328 --- .../content/xdocs/bookkeeperConfig.xml | 156 -- .../content/xdocs/bookkeeperOverview.xml | 419 ---- .../content/xdocs/bookkeeperProgrammer.xml | 678 ------- .../content/xdocs/bookkeeperStarted.xml | 208 -- .../content/xdocs/bookkeeperStream.xml | 331 ---- .../src/documentation/content/xdocs/index.xml | 98 - .../documentation/content/xdocs/javaExample.xml | 663 ------- .../src/documentation/content/xdocs/recipes.xml | 637 ------ .../src/documentation/content/xdocs/site.xml | 103 - .../src/documentation/content/xdocs/tabs.xml | 36 - .../content/xdocs/zookeeperAdmin.xml | 1861 ------------------ .../xdocs/zookeeperHierarchicalQuorums.xml | 75 - .../content/xdocs/zookeeperInternals.xml | 487 ----- .../content/xdocs/zookeeperJMX.xml | 236 --- .../content/xdocs/zookeeperObservers.xml | 145 -- .../content/xdocs/zookeeperOtherInfo.xml | 46 - .../content/xdocs/zookeeperOver.xml | 464 ----- .../content/xdocs/zookeeperProgrammers.xml | 1640 --------------- .../content/xdocs/zookeeperQuotas.xml | 71 - .../content/xdocs/zookeeperStarted.xml | 418 ---- .../content/xdocs/zookeeperTutorial.xml | 712 ------- .../src/documentation/resources/images/2pc.jpg | Bin 15174 -> 0 bytes .../resources/images/bk-overview.jpg | Bin 124211 -> 0 bytes .../documentation/resources/images/favicon.ico | Bin 766 -> 0 bytes .../resources/images/hadoop-logo.jpg | Bin 9443 -> 0 bytes .../resources/images/state_dia.dia | Bin 2597 -> 0 bytes .../resources/images/state_dia.jpg | Bin 51364 -> 0 bytes .../documentation/resources/images/zkarch.jpg | Bin 24535 -> 0 bytes .../resources/images/zkcomponents.jpg | Bin 30831 -> 0 bytes .../resources/images/zknamespace.jpg | Bin 35414 -> 0 bytes .../resources/images/zkperfRW-3.2.jpg | Bin 41948 -> 0 bytes .../documentation/resources/images/zkperfRW.jpg | Bin 161542 -> 0 bytes .../resources/images/zkperfreliability.jpg | Bin 69825 -> 0 bytes .../resources/images/zkservice.jpg | Bin 86790 -> 0 bytes .../resources/images/zookeeper_small.gif | Bin 4847 -> 0 bytes src/docs/src/documentation/skinconf.xml | 360 ---- src/docs/status.xml | 74 - zookeeper-docs/forrest.properties | 109 + zookeeper-docs/src/documentation/README.txt | 7 + zookeeper-docs/src/documentation/TODO.txt | 227 +++ .../classes/CatalogManager.properties | 37 + zookeeper-docs/src/documentation/conf/cli.xconf | 328 +++ .../content/xdocs/bookkeeperConfig.xml | 156 ++ .../content/xdocs/bookkeeperOverview.xml | 419 ++++ .../content/xdocs/bookkeeperProgrammer.xml | 678 +++++++ .../content/xdocs/bookkeeperStarted.xml | 208 ++ .../content/xdocs/bookkeeperStream.xml | 331 ++++ .../src/documentation/content/xdocs/index.xml | 98 + .../documentation/content/xdocs/javaExample.xml | 663 +++++++ .../src/documentation/content/xdocs/recipes.xml | 637 ++++++ .../src/documentation/content/xdocs/site.xml | 103 + .../src/documentation/content/xdocs/tabs.xml | 36 + .../content/xdocs/zookeeperAdmin.xml | 1861 ++++++++++++++++++ .../xdocs/zookeeperHierarchicalQuorums.xml | 75 + .../content/xdocs/zookeeperInternals.xml | 487 +++++ .../content/xdocs/zookeeperJMX.xml | 236 +++ .../content/xdocs/zookeeperObservers.xml | 145 ++ .../content/xdocs/zookeeperOtherInfo.xml | 46 + .../content/xdocs/zookeeperOver.xml | 464 +++++ .../content/xdocs/zookeeperProgrammers.xml | 1640 +++++++++++++++ .../content/xdocs/zookeeperQuotas.xml | 71 + .../content/xdocs/zookeeperStarted.xml | 418 ++++ .../content/xdocs/zookeeperTutorial.xml | 712 +++++++ .../src/documentation/resources/images/2pc.jpg | Bin 0 -> 15174 bytes .../resources/images/bk-overview.jpg | Bin 0 -> 124211 bytes .../documentation/resources/images/favicon.ico | Bin 0 -> 766 bytes .../resources/images/hadoop-logo.jpg | Bin 0 -> 9443 bytes .../resources/images/state_dia.dia | Bin 0 -> 2597 bytes .../resources/images/state_dia.jpg | Bin 0 -> 51364 bytes .../documentation/resources/images/zkarch.jpg | Bin 0 -> 24535 bytes .../resources/images/zkcomponents.jpg | Bin 0 -> 30831 bytes .../resources/images/zknamespace.jpg | Bin 0 -> 35414 bytes .../resources/images/zkperfRW-3.2.jpg | Bin 0 -> 41948 bytes .../documentation/resources/images/zkperfRW.jpg | Bin 0 -> 161542 bytes .../resources/images/zkperfreliability.jpg | Bin 0 -> 69825 bytes .../resources/images/zkservice.jpg | Bin 0 -> 86790 bytes .../resources/images/zookeeper_small.gif | Bin 0 -> 4847 bytes zookeeper-docs/src/documentation/skinconf.xml | 360 ++++ zookeeper-docs/status.xml | 74 + zookeeper-it/.empty | 0 86 files changed, 10627 insertions(+), 10627 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zookeeper/blob/c1efa954/build.xml ---------------------------------------------------------------------- diff --git a/build.xml b/build.xml index 8ffa2f3..640be98 100644 --- a/build.xml +++ b/build.xml @@ -133,7 +133,7 @@ xmlns:cs="antlib:com.puppycrawl.tools.checkstyle"> <property name="test.quick" value="no" /> <property name="conf.dir" value="${basedir}/conf"/> <property name="docs.dir" value="${basedir}/docs"/> - <property name="docs.src" value="${basedir}/src/docs"/> + <property name="docs.src" value="${basedir}/zookeeper-docs"/> <property name="javadoc.link.java" value="http://docs.oracle.com/javase/6/docs/api/"; /> <property name="javadoc.packages" value="org.apache.*" /> http://git-wip-us.apache.org/repos/asf/zookeeper/blob/c1efa954/src/docs/forrest.properties ---------------------------------------------------------------------- diff --git a/src/docs/forrest.properties b/src/docs/forrest.properties deleted file mode 100644 index 70cf81d..0000000 --- a/src/docs/forrest.properties +++ /dev/null @@ -1,109 +0,0 @@ -# Copyright 2002-2004 The Apache Software Foundation -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -############## -# Properties used by forrest.build.xml for building the website -# These are the defaults, un-comment them if you need to change them. -############## - -# Prints out a summary of Forrest settings for this project -#forrest.echo=true - -# Project name (used to name .war file) -#project.name=my-project - -# Specifies name of Forrest skin to use -#project.skin=tigris -#project.skin=pelt - -# comma separated list, file:// is supported -#forrest.skins.descriptors=http://forrest.apache.org/skins/skins.xml,file:///c:/myskins/skins.xml - -############## -# behavioural properties -#project.menu-scheme=tab_attributes -#project.menu-scheme=directories - -############## -# layout properties - -# Properties that can be set to override the default locations -# -# Parent properties must be set. This usually means uncommenting -# project.content-dir if any other property using it is uncommented - -#project.status=status.xml -#project.content-dir=src/documentation -project.configfile=${project.home}/src/documentation/conf/cli.xconf -#project.raw-content-dir=${project.content-dir}/content -#project.conf-dir=${project.content-dir}/conf -#project.sitemap-dir=${project.content-dir} -#project.xdocs-dir=${project.content-dir}/content/xdocs -#project.resources-dir=${project.content-dir}/resources -#project.stylesheets-dir=${project.resources-dir}/stylesheets -#project.images-dir=${project.resources-dir}/images -#project.schema-dir=${project.resources-dir}/schema -#project.skins-dir=${project.content-dir}/skins -#project.skinconf=${project.content-dir}/skinconf.xml -#project.lib-dir=${project.content-dir}/lib -#project.classes-dir=${project.content-dir}/classes -#project.translations-dir=${project.content-dir}/translations - -############## -# validation properties - -# This set of properties determine if validation is performed -# Values are inherited unless overridden. -# e.g. if forrest.validate=false then all others are false unless set to true. -forrest.validate=true -forrest.validate.xdocs=${forrest.validate} -forrest.validate.skinconf=${forrest.validate} -forrest.validate.stylesheets=${forrest.validate} -forrest.validate.skins=${forrest.validate} -forrest.validate.skins.stylesheets=${forrest.validate.skins} - -# Make Forrest work with JDK6 -forrest.validate.sitemap=false - -# *.failonerror=(true|false) - stop when an XML file is invalid -forrest.validate.failonerror=true - -# *.excludes=(pattern) - comma-separated list of path patterns to not validate -# e.g. -#forrest.validate.xdocs.excludes=samples/subdir/**, samples/faq.xml -#forrest.validate.xdocs.excludes= - - -############## -# General Forrest properties - -# The URL to start crawling from -#project.start-uri=linkmap.html -# Set logging level for messages printed to the console -# (DEBUG, INFO, WARN, ERROR, FATAL_ERROR) -#project.debuglevel=ERROR -# Max memory to allocate to Java -#forrest.maxmemory=64m -# Any other arguments to pass to the JVM. For example, to run on an X-less -# server, set to -Djava.awt.headless=true -#forrest.jvmargs= -# The bugtracking URL - the issue number will be appended -#project.bugtracking-url=http://issues.apache.org/bugzilla/show_bug.cgi?id= -#project.bugtracking-url=http://issues.apache.org/jira/browse/ -# The issues list as rss -#project.issues-rss-url= -#I18n Property only works for the "forrest run" target. -#project.i18n=true - -project.required.plugins=org.apache.forrest.plugin.output.pdf,org.apache.forrest.plugin.input.simplifiedDocbook http://git-wip-us.apache.org/repos/asf/zookeeper/blob/c1efa954/src/docs/src/documentation/README.txt ---------------------------------------------------------------------- diff --git a/src/docs/src/documentation/README.txt b/src/docs/src/documentation/README.txt deleted file mode 100644 index 9bc261b..0000000 --- a/src/docs/src/documentation/README.txt +++ /dev/null @@ -1,7 +0,0 @@ -This is the base documentation directory. - -skinconf.xml # This file customizes Forrest for your project. In it, you - # tell forrest the project name, logo, copyright info, etc - -sitemap.xmap # Optional. This sitemap is consulted before all core sitemaps. - # See http://forrest.apache.org/docs/project-sitemap.html http://git-wip-us.apache.org/repos/asf/zookeeper/blob/c1efa954/src/docs/src/documentation/TODO.txt ---------------------------------------------------------------------- diff --git a/src/docs/src/documentation/TODO.txt b/src/docs/src/documentation/TODO.txt deleted file mode 100644 index 84e7dfa..0000000 --- a/src/docs/src/documentation/TODO.txt +++ /dev/null @@ -1,227 +0,0 @@ -This is a running list of todo documentation items. Feel free -to add to the list or take on an item as you wish (in the form -of a JIRA patch of course). -------------------------------------------------------------- - -recipes.xml:110: -[maybe an illustration would be nice for each recipe?] - -recipes.xml:167: -"wait for each watch event". [how do you wait?] - -recipes.xml:457: -<remark>[tbd: myabe helpful to indicate which step this refers to?]</remark> - -zookeeperAdmin.xml:77: -because requires a majority <remark>[tbd: why?]</remark>, it is best to use... - -zookeeperAdmin.xml:112: - <screen>$yinst -i jdk-1.6.0.00_3 -br test <remark>[y! prop - replace with open equiv]</remark></screen> - -zookeeperAdmin.xml:99: -- use a maximum heap size of 3GB for a 4GB machine. <remark>[tbd: where would they do this? Environment variable, etc?]</remark> - -zookeeperAdmin.xml:120 -<screen>$ yinst install -nostart zookeeper_server <remark>[Y! prop - replace with open eq]</remark></screen> - -zookeeperAdmin.xml:171: -In Java, you can run the following command to execute simple operations:<remark> [tbd: also, maybe give some of those simple operations?] - -zookeeperAdmin.xml:194: -Running either program gives you a shell in which to execute simple file-system-like operations. <remark>[tbd: again, sample - operations?] - -zookeeperAdmin.xml:252: -If servers use different configuration files, -care must be taken to ensure that the list of servers in all of the -standard form, with legal values, etc]</remark> - -zookeeperAdmin.xml:408: -(Note: The system property has no zookeeper -prefix, and the configuration variable name is different from -the system property. Yes - it's not consistent, and it's -annoying.<remark> [tbd: is there any explanation for -this?]</remark>) - -zookeeperAdmin.xml:445: When the election algorithm is - "0" a UDP port with the same port number as the port listed in - the <emphasis role="bold">server.num</emphasis> option will be - used. <remark>[tbd: should that be <emphasis - role="bold">server.id</emphasis>? Also, why isn't server.id - documented anywhere?]</remark> - -zookeeperAdmin.xml:481: The default to this option is yes, which - means that a leader will accept client connections. - <remark>[tbd: how do you specifiy which server is the - leader?]</remark> - -zookeeperAdmin.xml:495 When the server - starts up, it determines which server it is by looking for the - file <filename>myid</filename> in the data directory.<remark> - [tdb: should we mention somewhere about creating this file, - myid, in the setup procedure?]</remark> - -zookeeperAdmin.xml:508: [tbd: is the next sentence explanation an of what the - election port or is it a description of a special case?] - </remark>If you want to test multiple servers on a single - machine, the individual choices of electionPort for each - server can be defined in each server's config files using the - line electionPort=xxxx to avoid clashes. - -zookeeperAdmin.xml:524: If followers fall too far behind a - leader, they will be dropped. <remark>[tbd: is this a correct - rewording: if followers fall beyond this limit, they are - dropped?]</remark> - -zookeeperAdmin.xml:551: ZooKeeper will not require updates - to be synced to the media. <remark>[tbd: useful because..., - dangerous because...]</remark> - -zookeeperAdmin.xml:580: Skips ACL checks. <remark>[tbd: when? where?]</remark> - -zookeeperAdmin.xml:649: <remark>[tbd: Patrick, Ben, et al: I believe the Message Broker - team does perform routine monitoring of Zookeeper. But I might be - wrong. To your knowledge, is there any monitoring of a Zookeeper - deployment that will a Zookeeper sys admin will want to do, outside of - Yahoo?]</remark> - -zookeeperAdmin.xml:755: Also, - the server lists in each Zookeeper server configuration file - should be consistent with one another. <remark>[tbd: I'm assuming - this last part is true. Is it?]</remark> - -zookeeperAdmin.xml:812: For best results, take note of the following list of good - Zookeeper practices. <remark>[tbd: I just threw this section in. Do we - have list that is is different from the "things to avoid"? If not, I can - easily remove this section.]</remark> - - -zookeeperOver.xml:162: Ephemeral nodes are useful when you - want to implement <remark>[tbd]</remark>. - -zookeeperOver.xml:174: And if the - connection between the client and one of the Zoo Keeper servers is - broken, the client will receive a local notification. These can be used - to <remark>[tbd]</remark> - -zookeeperOver.xml:215: <para>For more information on these (guarantees), and how they can be used, see - <remark>[tbd]</remark></para> - -zookeeperOver.xml:294: <para><xref linkend="fg_zkComponents" /> shows the high-level components - of the ZooKeeper service. With the exception of the request processor, - <remark>[tbd: where does the request processor live?]</remark> - -zookeeperOver.xml:298: <para><xref linkend="fg_zkComponents" /> shows the high-level components - of the ZooKeeper service. With the exception of the request processor, - each of - the servers that make up the ZooKeeper service replicates its own copy - of each of components. <remark>[tbd: I changed the wording in this - sentence from the white paper. Can someone please make sure it is still - correct?]</remark> - -zookeeperOver.xml:342: The programming interface to ZooKeeper is deliberately simple. - With it, however, you can implement higher order operations, such as - synchronizations primitives, group membership, ownership, etc. Some - distributed applications have used it to: <remark>[tbd: add uses from - white paper and video presentation.]</remark> - - -zookeeperProgrammers.xml:94: <listitem> - <para><xref linkend="ch_programStructureWithExample" /> - <remark>[tbd]</remark></para> - </listitem> - -zookeeperProgrammers.xml:115: Also, - the <ulink url="#ch_programStructureWithExample">Simple Programmming - Example</ulink> <remark>[tbd]</remark> is helpful for understand the basic - structure of a ZooKeeper client application. - -zookeeperProgrammers.xml:142: The following characters are not - allowed because <remark>[tbd: - do we need reasons?]</remark> - -zookeeperProgrammers.xml:172: If - the version it supplies doesn't match the actual version of the data, - the update will fail. (This behavior can be overridden. For more - information see... )<remark>[tbd... reference here to the section - describing the special version number -1]</remark> - -zookeeperProgrammers.xml:197: More information about watches can be - found in the section - <ulink url="recipes.html#sc_recipes_Locks"> - Zookeeper Watches</ulink>. - <remark>[tbd: fix this link] [tbd: Ben there is note from to emphasize - that "it is queued". What is "it" and is what we have here - sufficient?]</remark></para> - -zookeeperProgrammers.xml:335: it will send the session id as a part of the connection handshake. - As a security measure, the server creates a password for the session id - that any ZooKeeper server can validate. <remark>[tbd: note from Ben: - "perhaps capability is a better word." need clarification on that.] - </remark> - -zookeeperProgrammers.xml:601: <ulink - url="recipes.html#sc_recipes_Locks">Locks</ulink> - <remark>[tbd:...]</remark> in <ulink - url="recipes.html">Zookeeper Recipes</ulink>. - <remark>[tbd:..]</remark>).</para> - -zookeeperProgrammers.xml:766: <para>See INSTALL for general information about running - <emphasis role="bold">configure</emphasis>. <remark>[tbd: what - is INSTALL? a directory? a file?]</remark></para> - - - -zookeeperProgrammers.xml:813: <para>To verify that the node's been created:</para> - - <para>You should see a list of node who are children of the root node - "/".</para><remark>[tbd: document all the cli commands (I think this is ben's comment) - -zookeeperProgrammers.xml:838: <para>Refer to <xref linkend="ch_programStructureWithExample"/>for examples of usage in Java and C. - <remark>[tbd]</remark></para> - -zookeeperProgrammers.xml 847: <remark>[tbd: This is a new section. The below - is just placeholder. Eventually, a subsection on each of those operations, with a little - bit of illustrative code for each op.] </remark> - -zookeeperProgrammers.xml:915: Program Structure, with Simple Example</title> - -zookeeperProgrammers.xml:999: <term>ZooKeeper Whitepaper <remark>[tbd: find url]</remark></term> - -zookeeperProgrammers.xml:1008: <term>API Reference <remark>[tbd: find url]</remark></term> - -zookeeperProgrammers.xml:1062: [tbd]</remark></term><listitem> - <para>Any other good sources anyone can think of...</para> - </listitem> - -zookeeperStarted.xml:73: <para>[tbd: should we start w/ a word here about were to get the source, - exactly what to download, how to unpack it, and where to put it? Also, - does the user need to be in sudo, or can they be under their regular - login?]</para> - -zookeeperStarted.xml:84: <para>This should generate a JAR file called zookeeper.jar. To start - Zookeeper, compile and run zookeeper.jar. <emphasis>[tbd, some more - instruction here. Perhaps a command line? Are these two steps or - one?]</emphasis></para> - -zookeeperStarted.xml:139: <para>ZooKeeper logs messages using log4j -- more detail available in - the <ulink url="zookeeperProgrammers.html#Logging">Logging</ulink> - section of the Programmer's Guide.<remark revision="include_tbd">[tbd: - real reference needed]</remark> - -zookeeperStarted.xml:201: The C bindings exist in two variants: single - threaded and multi-threaded. These differ only in how the messaging loop - is done. <remark>[tbd: what is the messaging loop? Do we talk about it - anywyhere? is this too much info for a getting started guide?]</remark> - -zookeeperStarted.xml:217: The entry <emphasis - role="bold">syncLimit</emphasis> limits how far out of date a server can - be from a leader. [TBD: someone please verify that the previous is - true.] - -zookeeperStarted.xml:232: These are the "electionPort" numbers of the servers (as opposed to - clientPorts), that is ports for <remark>[tbd: feedback need: what are - these ports, exactly?] - -zookeeperStarted.xml:258: <remark>[tbd: what is the other config param? - (I believe two are mentioned above.)]</remark> http://git-wip-us.apache.org/repos/asf/zookeeper/blob/c1efa954/src/docs/src/documentation/classes/CatalogManager.properties ---------------------------------------------------------------------- diff --git a/src/docs/src/documentation/classes/CatalogManager.properties b/src/docs/src/documentation/classes/CatalogManager.properties deleted file mode 100644 index ac060b9..0000000 --- a/src/docs/src/documentation/classes/CatalogManager.properties +++ /dev/null @@ -1,37 +0,0 @@ -# Copyright 2002-2004 The Apache Software Foundation -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -#======================================================================= -# CatalogManager.properties -# -# This is the default properties file for Apache Forrest. -# This facilitates local configuration of application-specific catalogs. -# -# See the Apache Forrest documentation: -# http://forrest.apache.org/docs/your-project.html -# http://forrest.apache.org/docs/validation.html - -# verbosity ... level of messages for status/debug -# See forrest/src/core/context/WEB-INF/cocoon.xconf - -# catalogs ... list of additional catalogs to load -# (Note that Apache Forrest will automatically load its own default catalog -# from src/core/context/resources/schema/catalog.xcat) -# use full pathnames -# pathname separator is always semi-colon (;) regardless of operating system -# directory separator is always slash (/) regardless of operating system -# -#catalogs=/home/me/forrest/my-site/src/documentation/resources/schema/catalog.xcat -catalogs= - http://git-wip-us.apache.org/repos/asf/zookeeper/blob/c1efa954/src/docs/src/documentation/conf/cli.xconf ---------------------------------------------------------------------- diff --git a/src/docs/src/documentation/conf/cli.xconf b/src/docs/src/documentation/conf/cli.xconf deleted file mode 100644 index c671340..0000000 --- a/src/docs/src/documentation/conf/cli.xconf +++ /dev/null @@ -1,328 +0,0 @@ -<?xml version="1.0"?> -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> -<!--+ - | This is the Apache Cocoon command line configuration file. - | Here you give the command line interface details of where - | to find various aspects of your Cocoon installation. - | - | If you wish, you can also use this file to specify the URIs - | that you wish to generate. - | - | The current configuration information in this file is for - | building the Cocoon documentation. Therefore, all links here - | are relative to the build context dir, which, in the build.xml - | file, is set to ${build.context} - | - | Options: - | verbose: increase amount of information presented - | to standard output (default: false) - | follow-links: whether linked pages should also be - | generated (default: true) - | precompile-only: precompile sitemaps and XSP pages, but - | do not generate any pages (default: false) - | confirm-extensions: check the mime type for the generated page - | and adjust filename and links extensions - | to match the mime type - | (e.g. text/html->.html) - | - | Note: Whilst using an xconf file to configure the Cocoon - | Command Line gives access to more features, the use of - | command line parameters is more stable, as there are - | currently plans to improve the xconf format to allow - | greater flexibility. If you require a stable and - | consistent method for accessing the CLI, it is recommended - | that you use the command line parameters to configure - | the CLI. See documentation at: - | http://cocoon.apache.org/2.1/userdocs/offline/ - | http://wiki.apache.org/cocoon/CommandLine - | - +--> - -<cocoon verbose="true" - follow-links="true" - precompile-only="false" - confirm-extensions="false"> - - <!--+ - | The context directory is usually the webapp directory - | containing the sitemap.xmap file. - | - | The config file is the cocoon.xconf file. - | - | The work directory is used by Cocoon to store temporary - | files and cache files. - | - | The destination directory is where generated pages will - | be written (assuming the 'simple' mapper is used, see - | below) - +--> - <context-dir>.</context-dir> - <config-file>WEB-INF/cocoon.xconf</config-file> - <work-dir>../tmp/cocoon-work</work-dir> - <dest-dir>../site</dest-dir> - - <!--+ - | A checksum file can be used to store checksums for pages - | as they are generated. When the site is next generated, - | files will not be written if their checksum has not changed. - | This means that it will be easier to detect which files - | need to be uploaded to a server, using the timestamp. - | - | The default path is relative to the core webapp directory. - | An asolute path can be used. - +--> - <!-- <checksums-uri>build/work/checksums</checksums-uri>--> - - <!--+ - | Broken link reporting options: - | Report into a text file, one link per line: - | <broken-links type="text" report="filename"/> - | Report into an XML file: - | <broken-links type="xml" report="filename"/> - | Ignore broken links (default): - | <broken-links type="none"/> - | - | Two attributes to this node specify whether a page should - | be generated when an error has occurred. 'generate' specifies - | whether a page should be generated (default: true) and - | extension specifies an extension that should be appended - | to the generated page's filename (default: none) - | - | Using this, a quick scan through the destination directory - | will show broken links, by their filename extension. - +--> - <broken-links type="xml" - file="../brokenlinks.xml" - generate="false" - extension=".error" - show-referrers="true"/> - - <!--+ - | Load classes at startup. This is necessary for generating - | from sites that use SQL databases and JDBC. - | The <load-class> element can be repeated if multiple classes - | are needed. - +--> - <!-- - <load-class>org.firebirdsql.jdbc.Driver</load-class> - --> - - <!--+ - | Configures logging. - | The 'log-kit' parameter specifies the location of the log kit - | configuration file (usually called logkit.xconf. - | - | Logger specifies the logging category (for all logging prior - | to other Cocoon logging categories taking over) - | - | Available log levels are: - | DEBUG: prints all level of log messages. - | INFO: prints all level of log messages except DEBUG - | ones. - | WARN: prints all level of log messages except DEBUG - | and INFO ones. - | ERROR: prints all level of log messages except DEBUG, - | INFO and WARN ones. - | FATAL_ERROR: prints only log messages of this level - +--> - <!-- <logging log-kit="WEB-INF/logkit.xconf" logger="cli" level="ERROR" /> --> - - <!--+ - | Specifies the filename to be appended to URIs that - | refer to a directory (i.e. end with a forward slash). - +--> - <default-filename>index.html</default-filename> - - <!--+ - | Specifies a user agent string to the sitemap when - | generating the site. - | - | A generic term for a web browser is "user agent". Any - | user agent, when connecting to a web server, will provide - | a string to identify itself (e.g. as Internet Explorer or - | Mozilla). It is possible to have Cocoon serve different - | content depending upon the user agent string provided by - | the browser. If your site does this, then you may want to - | use this <user-agent> entry to provide a 'fake' user agent - | to Cocoon, so that it generates the correct version of your - | site. - | - | For most sites, this can be ignored. - +--> - <!-- - <user-agent>Cocoon Command Line Environment 2.1</user-agent> - --> - - <!--+ - | Specifies an accept string to the sitemap when generating - | the site. - | User agents can specify to an HTTP server what types of content - | (by mime-type) they are able to receive. E.g. a browser may be - | able to handle jpegs, but not pngs. The HTTP accept header - | allows the server to take the browser's capabilities into account, - | and only send back content that it can handle. - | - | For most sites, this can be ignored. - +--> - - <accept>*/*</accept> - - <!--+ - | Specifies which URIs should be included or excluded, according - | to wildcard patterns. - | - | These includes/excludes are only relevant when you are following - | links. A link URI must match an include pattern (if one is given) - | and not match an exclude pattern, if it is to be followed by - | Cocoon. It can be useful, for example, where there are links in - | your site to pages that are not generated by Cocoon, such as - | references to api-documentation. - | - | By default, all URIs are included. If both include and exclude - | patterns are specified, a URI is first checked against the - | include patterns, and then against the exclude patterns. - | - | Multiple patterns can be given, using muliple include or exclude - | nodes. - | - | The order of the elements is not significant, as only the first - | successful match of each category is used. - | - | Currently, only the complete source URI can be matched (including - | any URI prefix). Future plans include destination URI matching - | and regexp matching. If you have requirements for these, contact - | [email protected]. - +--> - - <exclude pattern="**/"/> - <exclude pattern="**apidocs**"/> - <exclude pattern="api/**"/> - - <!-- ZOOKEEPER-2364 - we build our own release notes separately --> - <exclude pattern="releasenotes.**"/> - -<!-- - This is a workaround for FOR-284 "link rewriting broken when - linking to xml source views which contain site: links". - See the explanation there and in declare-broken-site-links.xsl ---> - <exclude pattern="site:**"/> - <exclude pattern="ext:**"/> - <exclude pattern="lm:**"/> - <exclude pattern="**/site:**"/> - <exclude pattern="**/ext:**"/> - <exclude pattern="**/lm:**"/> - - <!-- Exclude tokens used in URLs to ASF mirrors (interpreted by a CGI) --> - <exclude pattern="[preferred]/**"/> - <exclude pattern="[location]"/> - - <!-- <include-links extension=".html"/>--> - - <!--+ - | <uri> nodes specify the URIs that should be generated, and - | where required, what should be done with the generated pages. - | They describe the way the URI of the generated file is created - | from the source page's URI. There are three ways that a generated - | file URI can be created: append, replace and insert. - | - | The "type" attribute specifies one of (append|replace|insert): - | - | append: - | Append the generated page's URI to the end of the source URI: - | - | <uri type="append" src-prefix="documents/" src="index.html" - | dest="build/dest/"/> - | - | This means that - | (1) the "documents/index.html" page is generated - | (2) the file will be written to "build/dest/documents/index.html" - | - | replace: - | Completely ignore the generated page's URI - just - | use the destination URI: - | - | <uri type="replace" src-prefix="documents/" src="index.html" - | dest="build/dest/docs.html"/> - | - | This means that - | (1) the "documents/index.html" page is generated - | (2) the result is written to "build/dest/docs.html" - | (3) this works only for "single" pages - and not when links - | are followed - | - | insert: - | Insert generated page's URI into the destination - | URI at the point marked with a * (example uses fictional - | zip protocol) - | - | <uri type="insert" src-prefix="documents/" src="index.html" - | dest="zip://*.zip/page.html"/> - | - | This means that - | (1) - | - | In any of these scenarios, if the dest attribute is omitted, - | the value provided globally using the <dest-dir> node will - | be used instead. - +--> - <!-- - <uri type="replace" - src-prefix="samples/" - src="hello-world/hello.html" - dest="build/dest/hello-world.html"/> - --> - - <!--+ - | <uri> nodes can be grouped together in a <uris> node. This - | enables a group of URIs to share properties. The following - | properties can be set for a group of URIs: - | * follow-links: should pages be crawled for links - | * confirm-extensions: should file extensions be checked - | for the correct mime type - | * src-prefix: all source URIs should be - | pre-pended with this prefix before - | generation. The prefix is not - | included when calculating the - | destination URI - | * dest: the base destination URI to be - | shared by all pages in this group - | * type: the method to be used to calculate - | the destination URI. See above - | section on <uri> node for details. - | - | Each <uris> node can have a name attribute. When a name - | attribute has been specified, the -n switch on the command - | line can be used to tell Cocoon to only process the URIs - | within this URI group. When no -n switch is given, all - | <uris> nodes are processed. Thus, one xconf file can be - | used to manage multiple sites. - +--> - <!-- - <uris name="mirrors" follow-links="false"> - <uri type="append" src="mirrors.html"/> - </uris> - --> - - <!--+ - | File containing URIs (plain text, one per line). - +--> - <!-- - <uri-file>uris.txt</uri-file> - --> -</cocoon> http://git-wip-us.apache.org/repos/asf/zookeeper/blob/c1efa954/src/docs/src/documentation/content/xdocs/bookkeeperConfig.xml ---------------------------------------------------------------------- diff --git a/src/docs/src/documentation/content/xdocs/bookkeeperConfig.xml b/src/docs/src/documentation/content/xdocs/bookkeeperConfig.xml deleted file mode 100644 index 7a80949..0000000 --- a/src/docs/src/documentation/content/xdocs/bookkeeperConfig.xml +++ /dev/null @@ -1,156 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!-- - Copyright 2002-2004 The Apache Software Foundation - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> -<!DOCTYPE article PUBLIC "-//OASIS//DTD Simplified DocBook XML V1.0//EN" -"http://www.oasis-open.org/docbook/xml/simple/1.0/sdocbook.dtd";> -<article id="bk_Admin"> - <title>BookKeeper Administrator's Guide</title> - - <subtitle>Setup Guide</subtitle> - - <articleinfo> - <legalnotice> - <para>Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. You may - obtain a copy of the License at <ulink - url="http://www.apache.org/licenses/LICENSE-2.0";>http://www.apache.org/licenses/LICENSE-2.0</ulink>. - </para> - - <para>Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an "AS IS" - BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied. See the License for the specific language governing permissions - and limitations under the License. - </para> - </legalnotice> - - <abstract> - <para>This document contains information about deploying, administering - and mantaining BookKeeper. It also discusses best practices and common - problems. - </para> - <para> As BookKeeper is still a prototype, this article is likely to change - significantly over time. - </para> - </abstract> - </articleinfo> - - <section id="bk_deployment"> - <title>Deployment</title> - - <para>This section contains information about deploying BookKeeper and - covers these topics:</para> - - <itemizedlist> - <listitem> - <para><xref linkend="bk_sysReq" /></para> - </listitem> - - <listitem> - <para><xref linkend="bk_runningBookies" /></para> - </listitem> - - <listitem> - <para><xref linkend="bk_zkMetadata" /></para> - </listitem> - </itemizedlist> - - <para> The first section tells you how many machines you need. The second explains how to bootstrap bookies - (BookKeeper storage servers). The third section explains how we use ZooKeeper and our requirements with - respect to ZooKeeper. - </para> - - <section id="bk_sysReq"> - <title>System requirements</title> - <para> A typical BookKeeper installation comprises a set of bookies and a set of ZooKeeper replicas. The exact number of bookies - depends on the quorum mode, desired throughput, and number of clients using this installation simultaneously. The minimum number of - bookies is three for self-verifying (stores a message authentication code along with each entry) and four for generic (does not - store a message authentication codewith each entry), and there is no upper limit on the number of bookies. Increasing the number of - bookies, in fact, enables higher throughput. - </para> - - <para> For performance, we require each server to have at least two disks. It is possible to run a bookie with a single disk, but - performance will be significantly lower in this case. Of course, it works with one disk, but performance is significantly lower. - </para> - - <para> For ZooKeeper, there is no constraint with respect to the number of replicas. Having a single machine running ZooKeeper - in standalone mode is sufficient for BookKeeper. For resilience purposes, it might be a good idea to run ZooKeeper in quorum - mode with multiple servers. Please refer to the ZooKeeper documentation for detail on how to configure ZooKeeper with multiple - replicas - </para> - </section> - - <section id="bk_runningBookies"> - <title>Running bookies</title> - <para> - To run a bookie, we execute the following command: - </para> - - <para><computeroutput> - java -cp .:./zookeeper-<version>-bookkeeper.jar:./zookeeper-<version>.jar\ - :../log4j/apache-log4j-1.2.15/log4j-1.2.15.jar -Dlog4j.configuration=log4j.properties\ - org.apache.bookkeeper.proto.BookieServer 3181 127.0.0.1:2181 /path_to_log_device/\ - /path_to_ledger_device/ - </computeroutput></para> - - <para> - The parameters are: - </para> - - <itemizedlist> - <listitem> - <para> - Port number that the bookie listens on; - </para> - </listitem> - - <listitem> - <para> - Comma separated list of ZooKeeper servers with a hostname:port format; - </para> - </listitem> - - <listitem> - <para> - Path for Log Device (stores bookie write-ahead log); - </para> - </listitem> - - <listitem> - <para> - Path for Ledger Device (stores ledger entries); - </para> - </listitem> - </itemizedlist> - - <para> - Ideally, <computeroutput>/path_to_log_device/ </computeroutput> and <computeroutput>/path_to_ledger_device/ </computeroutput> are each - in a different device. - </para> - </section> - - <section id="bk_zkMetadata"> - <title>ZooKeeper Metadata</title> - <para> - For BookKeeper, we require a ZooKeeper installation to store metadata, and to pass the list - of ZooKeeper servers as parameter to the constructor of the BookKeeper class (<computeroutput> - org.apache.bookkeeper.client,BookKeeper</computeroutput>). - To setup ZooKeeper, please check the <ulink url="index.html"> - ZooKeeper documentation</ulink>. - </para> - </section> - </section> -</article> http://git-wip-us.apache.org/repos/asf/zookeeper/blob/c1efa954/src/docs/src/documentation/content/xdocs/bookkeeperOverview.xml ---------------------------------------------------------------------- diff --git a/src/docs/src/documentation/content/xdocs/bookkeeperOverview.xml b/src/docs/src/documentation/content/xdocs/bookkeeperOverview.xml deleted file mode 100644 index cdc1878..0000000 --- a/src/docs/src/documentation/content/xdocs/bookkeeperOverview.xml +++ /dev/null @@ -1,419 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!-- - Copyright 2002-2004 The Apache Software Foundation - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> - -<!DOCTYPE article PUBLIC "-//OASIS//DTD Simplified DocBook XML V1.0//EN" -"http://www.oasis-open.org/docbook/xml/simple/1.0/sdocbook.dtd";> -<article id="bk_GettStartedGuide"> - <title>BookKeeper overview</title> - - <articleinfo> - <legalnotice> - <para>Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. You may - obtain a copy of the License at <ulink - url="http://www.apache.org/licenses/LICENSE-2.0";>http://www.apache.org/licenses/LICENSE-2.0</ulink>.</para> - - <para>Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an "AS IS" - BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied. See the License for the specific language governing permissions - and limitations under the License.</para> - </legalnotice> - - <abstract> - <para>This guide contains detailed information about using BookKeeper - for logging. It discusses the basic operations BookKeeper supports, - and how to create logs and perform basic read and write operations on these - logs.</para> - </abstract> - </articleinfo> - <section id="bk_Overview"> - <title>BookKeeper overview</title> - - <section id="bk_Intro"> - <title>BookKeeper introduction</title> - <para> - BookKeeper is a replicated service to reliably log streams of records. In BookKeeper, - servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a - "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store - ledgers, can crash, corrupt data, discard data, but as long as there are enough bookies - behaving correctly the service as a whole behaves correctly. - </para> - - <para> - The initial motivation for BookKeeper comes from the namenode of HDFS. Namenodes have to - log operations in a reliable fashion so that recovery is possible in the case of crashes. - We have found the applications for BookKeeper extend far beyond HDFS, however. Essentially, - any application that requires an append storage can replace their implementations with - BookKeeper. BookKeeper has the advantage of scaling throughput with the number of servers. - </para> - - <para> - At a high level, a bookkeeper client receives entries from a client application and stores it to - sets of bookies, and there are a few advantages in having such a service: - </para> - - <itemizedlist> - <listitem> - <para> - We can use hardware that is optimized for such a service. We currently believe that such a - system has to be optimized only for disk I/O; - </para> - </listitem> - - <listitem> - <para> - We can have a pool of servers implementing such a log system, and shared among a number of servers; - </para> - </listitem> - - <listitem> - <para> - We can have a higher degree of replication with such a pool, which makes sense if the hardware necessary for it is cheaper compared to the one the application uses. - </para> - </listitem> - </itemizedlist> - - </section> - - <section id="bk_moreDetail"> - <title>In slightly more detail...</title> - - <para> BookKeeper implements highly available logs, and it has been designed with write-ahead logging in mind. Besides high availability - due to the replicated nature of the service, it provides high throughput due to striping. As we write entries in a subset of bookies of an - ensemble and rotate writes across available quorums, we are able to increase throughput with the number of servers for both reads and writes. - Scalability is a property that is possible to achieve in this case due to the use of quorums. Other replication techniques, such as - state-machine replication, do not enable such a property. - </para> - - <para> An application first creates a ledger before writing to bookies through a local BookKeeper client instance. - Upon creating a ledger, a BookKeeper client writes metadata about the ledger to ZooKeeper. Each ledger currently - has a single writer. This writer has to execute a close ledger operation before any other client can read from it. - If the writer of a ledger does not close a ledger properly because, for example, it has crashed before having the - opportunity of closing the ledger, then the next client that tries to open a ledger executes a procedure to recover - it. As closing a ledger consists essentially of writing the last entry written to a ledger to ZooKeeper, the recovery - procedure simply finds the last entry written correctly and writes it to ZooKeeper. - </para> - - <para> - Note that currently this recovery procedure is executed automatically upon trying to open a ledger and no explicit action is necessary. - Although two clients may try to recover a ledger concurrently, only one will succeed, the first one that is able to create the close znode - for the ledger. - </para> - </section> - - <section id="bk_basicComponents"> - <title>Bookkeeper elements and concepts</title> - <para> - BookKeeper uses four basic elements: - </para> - - <itemizedlist> - <listitem> - <para> - <emphasis role="bold">Ledger</emphasis>: A ledger is a sequence of entries, and each entry is a sequence of bytes. Entries are - written sequentially to a ledger and at most once. Consequently, ledgers have an append-only semantics; - </para> - </listitem> - - <listitem> - <para> - <emphasis role="bold">BookKeeper client</emphasis>: A client runs along with a BookKeeper application, and it enables applications - to execute operations on ledgers, such as creating a ledger and writing to it; - </para> - </listitem> - - <listitem> - <para> - <emphasis role="bold">Bookie</emphasis>: A bookie is a BookKeeper storage server. Bookies store the content of ledgers. For any given - ledger L, we call an <emphasis>ensemble</emphasis> the group of bookies storing the content of L. For performance, we store on - each bookie of an ensemble only a fragment of a ledger. That is, we stripe when writing entries to a ledger such that - each entry is written to sub-group of bookies of the ensemble. - </para> - </listitem> - - <listitem> - <para> - <emphasis role="bold">Metadata storage service</emphasis>: BookKeeper requires a metadata storage service to store information related - to ledgers and available bookies. We currently use ZooKeeper for such a task. - </para> - </listitem> - </itemizedlist> - </section> - - <section id="bk_initialDesign"> - <title>Bookkeeper initial design</title> - <para> - A set of bookies implements BookKeeper, and we use a quorum-based protocol to replicate data across the bookies. - There are basically two operations to an existing ledger: read and append. Here is the complete API list - (mode detail <ulink url="bookkeeperProgrammer.html"> - here</ulink>): - </para> - - <itemizedlist> - <listitem> - <para> - Create ledger: creates a new empty ledger; - </para> - </listitem> - - <listitem> - <para> - Open ledger: opens an existing ledger for reading; - </para> - </listitem> - - <listitem> - <para> - Add entry: adds a record to a ledger either synchronously or asynchronously; - </para> - </listitem> - - <listitem> - <para> - Read entries: reads a sequence of entries from a ledger either synchronously or asynchronously - </para> - </listitem> - </itemizedlist> - - <para> - There is only a single client that can write to a ledger. Once that ledger is closed or the client fails, - no more entries can be added. (We take advantage of this behavior to provide our strong guarantees.) - There will not be gaps in the ledger. Fingers get broken, people get roughed up or end up in prison when - books are manipulated, so there is no deleting or changing of entries. - </para> - - <figure> - <title>BookKeeper Overview</title> - - <mediaobject> - <imageobject> - <imagedata fileref="images/bk-overview.jpg" width="3in" depth="3in" contentwidth="3in" contentdepth="3in" scalefit="0"/> - </imageobject> - </mediaobject> - </figure> - - <para> - A simple use of BooKeeper is to implement a write-ahead transaction log. A server maintains an in-memory data structure - (with periodic snapshots for example) and logs changes to that structure before it applies the change. The application - server creates a ledger at startup and store the ledger id and password in a well known place (ZooKeeper maybe). When - it needs to make a change, the server adds an entry with the change information to a ledger and apply the change when - BookKeeper adds the entry successfully. The server can even use asyncAddEntry to queue up many changes for high change - throughput. BooKeeper meticulously logs the changes in order and call the completion functions in order. - </para> - - <para> - When the application server dies, a backup server will come online, get the last snapshot and then it will open the - ledger of the old server and read all the entries from the time the snapshot was taken. (Since it doesn't know the - last entry number it will use MAX_INTEGER). Once all the entries have been processed, it will close the ledger and - start a new one for its use. - </para> - - <para> - A client library takes care of communicating with bookies and managing entry numbers. An entry has the following fields: - </para> - - <table frame='all'><title>Entry fields</title> - <tgroup cols='3' align='left' colsep='1' rowsep='1'> - <colspec colname='Field'/> - <colspec colname='Type'/> - <colspec colname='Description'/> - <colspec colnum='5' colname='c5'/> - <thead> - <row> - <entry>Field</entry> - <entry>Type</entry> - <entry>Description</entry> - </row> - </thead> - <tfoot> - <row> - <entry>Ledger number</entry> - <entry>long</entry> - <entry>The id of the ledger of this entry</entry> - </row> - <row> - <entry>Entry number</entry> - <entry>long</entry> - <entry>The id of this entry</entry> - </row> - </tfoot> - <tbody> - <row> - <entry>last confirmed (<emphasis>LC</emphasis>)</entry> - <entry>long</entry> - <entry>id of the last recorded entry</entry> - </row> - <row> - <entry>data</entry> - <entry>byte[]</entry> - <entry>the entry data (supplied by application)</entry> - </row> - <row> - <entry>authentication code</entry> - <entry>byte[]</entry> - <entry>Message authentication code that includes all other fields of the entry</entry> - </row> - - </tbody> - </tgroup> - </table> - - <para> - The client library generates a ledger entry. None of the fields are modified by the bookies and only the first three - fields are interpreted by the bookies. - </para> - - <para> - To add to a ledger, the client generates the entry above using the ledger number. The entry number will be one more - than the last entry generated. The <emphasis>LC</emphasis> field contains the last entry that has been successfully recorded by BookKeeper. - If the client writes entries one at a time, <emphasis>LC</emphasis> is the last entry id. But, if the client is using asyncAddEntry, there - may be many entries in flight. An entry is considered recorded when both of the following conditions are met: - </para> - - <itemizedlist> - <listitem> - <para> - the entry has been accepted by a quorum of bookies - </para> - </listitem> - - <listitem> - <para> - all entries with a lower entry id have been accepted by a quorum of bookies - </para> - </listitem> - </itemizedlist> - - <para> - <emphasis>LC</emphasis> seems mysterious right now, but it is too early to explain how we use it; just smile and move on. - </para> - - <para> - Once all the other fields have been field in, the client generates an authentication code with all of the previous fields. - The entry is then sent to a quorum of bookies to be recorded. Any failures will result in the entry being sent to a new - quorum of bookies. - </para> - - <para> - To read, the client library initially contacts a bookie and starts requesting entries. If an entry is missing or - invalid (a bad MAC for example), the client will make a request to a different bookie. By using quorum writes, - as long as enough bookies are up we are guaranteed to eventually be able to read an entry. - </para> - - </section> - - <section id="bk_metadata"> - <title>Bookkeeper metadata management</title> - - <para> - There are some meta data that needs to be made available to BookKeeper clients: - </para> - - <itemizedlist> - <listitem> - <para> - The available bookies; - </para> - </listitem> - - <listitem> - <para> - The list of ledgers; - </para> - </listitem> - - <listitem> - <para> - The list of bookies that have been used for a given ledger; - </para> - </listitem> - - <listitem> - <para> - The last entry of a ledger; - </para> - </listitem> - </itemizedlist> - - <para> - We maintain this information in ZooKeeper. Bookies use ephemeral nodes to indicate their availability. Clients - use znodes to track ledger creation and deletion and also to know the end of the ledger and the bookies that - were used to store the ledger. Bookies also watch the ledger list so that they can cleanup ledgers that get deleted. - </para> - - </section> - - <section id="bk_closingOut"> - <title>Closing out ledgers</title> - - <para> - The process of closing out the ledger and finding the last ledger is difficult due to the durability guarantees of BookKeeper: - </para> - - <itemizedlist> - <listitem> - <para> - If an entry has been successfully recorded, it must be readable. - </para> - </listitem> - - <listitem> - <para> - If an entry is read once, it must always be available to be read. - </para> - </listitem> - </itemizedlist> - - <para> - If the ledger was closed gracefully, ZooKeeper will have the last entry and everything will work well. But, if the - BookKeeper client that was writing the ledger dies, there is some recovery that needs to take place. - </para> - - <para> - The problematic entries are the ones at the end of the ledger. There can be entries in flight when a BookKeeper client - dies. If the entry only gets to one bookie, the entry should not be readable since the entry will disappear if that bookie - fails. If the entry is only on one bookie, that doesn't mean that the entry has not been recorded successfully; the other - bookies that recorded the entry might have failed. - </para> - - <para> - The trick to making everything work is to have a correct idea of a last entry. We do it in roughly three steps: - </para> - <orderedlist> - <listitem> - <para> - Find the entry with the highest last recorded entry, <emphasis>LC</emphasis>; - </para> - </listitem> - - <listitem> - <para> - Find the highest consecutively recorded entry, <emphasis>LR</emphasis>; - </para> - </listitem> - - <listitem> - <para> - Make sure that all entries between <emphasis>LC</emphasis> and <emphasis>LR</emphasis> are on a quorum of bookies; - </para> - </listitem> - - </orderedlist> - </section> - </section> -</article> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/zookeeper/blob/c1efa954/src/docs/src/documentation/content/xdocs/bookkeeperProgrammer.xml ---------------------------------------------------------------------- diff --git a/src/docs/src/documentation/content/xdocs/bookkeeperProgrammer.xml b/src/docs/src/documentation/content/xdocs/bookkeeperProgrammer.xml deleted file mode 100644 index 5f330e1..0000000 --- a/src/docs/src/documentation/content/xdocs/bookkeeperProgrammer.xml +++ /dev/null @@ -1,678 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!-- - Copyright 2002-2004 The Apache Software Foundation - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> - -<!DOCTYPE article PUBLIC "-//OASIS//DTD Simplified DocBook XML V1.0//EN" -"http://www.oasis-open.org/docbook/xml/simple/1.0/sdocbook.dtd";> -<article id="bk_GettStartedGuide"> - <title>BookKeeper Getting Started Guide</title> - - <articleinfo> - <legalnotice> - <para>Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. You may - obtain a copy of the License at <ulink - url="http://www.apache.org/licenses/LICENSE-2.0";>http://www.apache.org/licenses/LICENSE-2.0</ulink>.</para> - - <para>Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an "AS IS" - BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied. See the License for the specific language governing permissions - and limitations under the License.</para> - </legalnotice> - - <abstract> - <para>This guide contains detailed information about using BookKeeper - for logging. It discusses the basic operations BookKeeper supports, - and how to create logs and perform basic read and write operations on these - logs.</para> - </abstract> - </articleinfo> - <section id="bk_GettingStarted"> - <title>Programming with BookKeeper</title> - - <itemizedlist> - <listitem> - <para><xref linkend="bk_instance" /></para> - </listitem> - - <listitem> - <para><xref linkend="bk_createLedger" /></para> - </listitem> - - <listitem> - <para><xref linkend="bk_writeLedger" /></para> - </listitem> - - <listitem> - <para><xref linkend="bk_closeLedger" /></para> - </listitem> - - <listitem> - <para><xref linkend="bk_openLedger" /></para> - </listitem> - - <listitem> - <para><xref linkend="bk_readLedger" /></para> - </listitem> - - <listitem> - <para><xref linkend="bk_deleteLedger" /></para> - </listitem> - - </itemizedlist> - - <section id="bk_instance"> - <title> Instantiating BookKeeper.</title> - <para> - The first step to use BookKeeper is to instantiate a BookKeeper object: - </para> - <para> - <computeroutput> - org.apache.bookkeeper.BookKeeper - </computeroutput> - </para> - - <para> - There are three BookKeeper constructors: - </para> - - <para> - <computeroutput> - public BookKeeper(String servers) - throws KeeperException, IOException - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>servers</computeroutput> is a comma-separated list of ZooKeeper servers. - </para> - </listitem> - </itemizedlist> - - <para> - <computeroutput> - public BookKeeper(ZooKeeper zk) - throws InterruptedException, KeeperException - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>zk</computeroutput> is a ZooKeeper object. This constructor is useful when - the application also using ZooKeeper and wants to have a single instance of ZooKeeper. - </para> - </listitem> - </itemizedlist> - - - <para> - <computeroutput> - public BookKeeper(ZooKeeper zk, ClientSocketChannelFactory channelFactory) - throws InterruptedException, KeeperException - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>zk</computeroutput> is a ZooKeeper object. This constructor is useful when - the application also using ZooKeeper and wants to have a single instance of ZooKeeper. - </para> - </listitem> - - <listitem> - <para> - <computeroutput>channelFactory</computeroutput> is a netty channel object - (<computeroutput>org.jboss.netty.channel.socket</computeroutput>). - </para> - </listitem> - </itemizedlist> - - - - </section> - - <section id="bk_createLedger"> - <title> Creating a ledger. </title> - - <para> Before writing entries to BookKeeper, it is necessary to create a ledger. - With the current BookKeeper API, it is possible to create a ledger both synchronously - or asynchronously. The following methods belong - to <computeroutput>org.apache.bookkeeper.client.BookKeeper</computeroutput>. - </para> - - <para> - <emphasis role="bold">Synchronous call:</emphasis> - </para> - - <para> - <computeroutput> - public LedgerHandle createLedger(int ensSize, int qSize, DigestType type, byte passwd[]) - throws KeeperException, InterruptedException, - IOException, BKException - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>ensSize</computeroutput> is the number of bookies (ensemble size); - </para> - </listitem> - - <listitem> - <para> - <computeroutput>qSize</computeroutput> is the write quorum size; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>type</computeroutput> is the type of digest used with entries: either MAC or CRC32. - </para> - </listitem> - - <listitem> - <para> - <computeroutput>passwd</computeroutput> is a password that authorizes the client to write to the - ledger being created. - </para> - </listitem> - </itemizedlist> - - <para> - All further operations on a ledger are invoked through the <computeroutput>LedgerHandle</computeroutput> - object returned. - </para> - - <para> - As a convenience, we provide a <computeroutput>createLedger</computeroutput> with default parameters (3,2,VERIFIABLE), - and the only two input parameters it requires are a digest type and a password. - </para> - - <para> - <emphasis role="bold">Asynchronous call:</emphasis> - </para> - - <para> - <computeroutput> - public void asyncCreateLedger(int ensSize, - int qSize, - DigestType type, - byte passwd[], - CreateCallback cb, - Object ctx - ) - </computeroutput> - </para> - - <para> - The parameters are the same of the synchronous version, with the - exception of <computeroutput>cb</computeroutput> and <computeroutput>ctx</computeroutput>. <computeroutput>CreateCallback</computeroutput> - is an interface in <computeroutput>org.apache.bookkeeper.client.AsyncCallback</computeroutput>, and - a class implementing it has to implement a method called <computeroutput>createComplete</computeroutput> - that has the following signature: - </para> - - <para> - <computeroutput> - void createComplete(int rc, LedgerHandle lh, Object ctx); - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>rc</computeroutput> is a return code (please refer to <computeroutput>org.apache.bookeeper.client.BKException</computeroutput> for a list); - </para> - </listitem> - - <listitem> - <para> - <computeroutput>lh</computeroutput> is a <computeroutput>LedgerHandle</computeroutput> object to manipulate a ledger; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>ctx</computeroutput> is a control object for accountability purposes. It can be essentially any object the application is happy with. - </para> - </listitem> - </itemizedlist> - - <para> - The <computeroutput>ctx</computeroutput> object passed as a parameter to the call to create a ledger - is the one same returned in the callback. - </para> - </section> - - <section id="bk_writeLedger"> - <title> Adding entries to a ledger. </title> - <para> - Once we have a ledger handle <computeroutput>lh</computeroutput> obtained through a call to create a ledger, we - can start writing entries. As with creating ledgers, we can write both synchronously and - asynchronously. The following methods belong - to <computeroutput>org.apache.bookkeeper.client.LedgerHandle</computeroutput>. - </para> - - <para> - <emphasis role="bold">Synchronous call:</emphasis> - </para> - - <para> - <computeroutput> - public long addEntry(byte[] data) - throws InterruptedException - </computeroutput> - </para> - - <para> - where: - </para> - - <itemizedlist> - <listitem> - <para> - <computeroutput>data</computeroutput> is a byte array; - </para> - </listitem> - </itemizedlist> - - <para> - A call to <computeroutput>addEntry</computeroutput> returns the status of the operation (please refer to <computeroutput>org.apache.bookeeper.client.BKDefs</computeroutput> for a list); - </para> - - <para> - <emphasis role="bold">Asynchronous call:</emphasis> - </para> - - <para> - <computeroutput> - public void asyncAddEntry(byte[] data, AddCallback cb, Object ctx) - </computeroutput> - </para> - - <para> - It also takes a byte array as the sequence of bytes to be stored as an entry. Additionaly, it takes - a callback object <computeroutput>cb</computeroutput> and a control object <computeroutput>ctx</computeroutput>. The callback object must implement - the <computeroutput>AddCallback</computeroutput> interface in <computeroutput>org.apache.bookkeeper.client.AsyncCallback</computeroutput>, and - a class implementing it has to implement a method called <computeroutput>addComplete</computeroutput> - that has the following signature: - </para> - - <para> - <computeroutput> - void addComplete(int rc, LedgerHandle lh, long entryId, Object ctx); - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>rc</computeroutput> is a return code (please refer to <computeroutput>org.apache.bookeeper.client.BKDefs</computeroutput> for a list); - </para> - </listitem> - - <listitem> - <para> - <computeroutput>lh</computeroutput> is a <computeroutput>LedgerHandle</computeroutput> object to manipulate a ledger; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>entryId</computeroutput> is the identifier of entry associated with this request; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>ctx</computeroutput> is control object used for accountability purposes. It can be any object the application is happy with. - </para> - </listitem> - </itemizedlist> - </section> - - <section id="bk_closeLedger"> - <title> Closing a ledger. </title> - <para> - Once a client is done writing, it closes the ledger. The following methods belong - to <computeroutput>org.apache.bookkeeper.client.LedgerHandle</computeroutput>. - </para> - <para> - <emphasis role="bold">Synchronous close:</emphasis> - </para> - - <para> - <computeroutput> - public void close() - throws InterruptedException - </computeroutput> - </para> - - <para> - It takes no input parameters. - </para> - - <para> - <emphasis role="bold">Asynchronous close:</emphasis> - </para> - <para> - <computeroutput> - public void asyncClose(CloseCallback cb, Object ctx) - throws InterruptedException - </computeroutput> - </para> - - <para> - It takes a callback object <computeroutput>cb</computeroutput> and a control object <computeroutput>ctx</computeroutput>. The callback object must implement - the <computeroutput>CloseCallback</computeroutput> interface in <computeroutput>org.apache.bookkeeper.client.AsyncCallback</computeroutput>, and - a class implementing it has to implement a method called <computeroutput>closeComplete</computeroutput> - that has the following signature: - </para> - - <para> - <computeroutput> - void closeComplete(int rc, LedgerHandle lh, Object ctx) - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>rc</computeroutput> is a return code (please refer to <computeroutput>org.apache.bookeeper.client.BKDefs</computeroutput> for a list); - </para> - </listitem> - - <listitem> - <para> - <computeroutput>lh</computeroutput> is a <computeroutput>LedgerHandle</computeroutput> object to manipulate a ledger; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>ctx</computeroutput> is control object used for accountability purposes. - </para> - </listitem> - </itemizedlist> - - </section> - - <section id="bk_openLedger"> - <title> Opening a ledger. </title> - <para> - To read from a ledger, a client must open it first. The following methods belong - to <computeroutput>org.apache.bookkeeper.client.BookKeeper</computeroutput>. - </para> - - <para> - <emphasis role="bold">Synchronous open:</emphasis> - </para> - - <para> - <computeroutput> - public LedgerHandle openLedger(long lId, DigestType type, byte passwd[]) - throws InterruptedException, BKException - </computeroutput> - </para> - - <itemizedlist> - <listitem> - <para> - <computeroutput>ledgerId</computeroutput> is the ledger identifier; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>type</computeroutput> is the type of digest used with entries: either MAC or CRC32. - </para> - </listitem> - - <listitem> - <para> - <computeroutput>passwd</computeroutput> is a password to access the ledger (used only in the case of <computeroutput>VERIFIABLE</computeroutput> ledgers); - </para> - </listitem> - </itemizedlist> - - <para> - <emphasis role="bold">Asynchronous open:</emphasis> - </para> - <para> - <computeroutput> - public void asyncOpenLedger(long lId, DigestType type, byte passwd[], OpenCallback cb, Object ctx) - </computeroutput> - </para> - - <para> - It also takes a a ledger identifier and a password. Additionaly, it takes a callback object - <computeroutput>cb</computeroutput> and a control object <computeroutput>ctx</computeroutput>. The callback object must implement - the <computeroutput>OpenCallback</computeroutput> interface in <computeroutput>org.apache.bookkeeper.client.AsyncCallback</computeroutput>, and - a class implementing it has to implement a method called <computeroutput>openComplete</computeroutput> - that has the following signature: - </para> - - <para> - <computeroutput> - public void openComplete(int rc, LedgerHandle lh, Object ctx) - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>rc</computeroutput> is a return code (please refer to <computeroutput>org.apache.bookeeper.client.BKDefs</computeroutput> for a list); - </para> - </listitem> - - <listitem> - <para> - <computeroutput>lh</computeroutput> is a <computeroutput>LedgerHandle</computeroutput> object to manipulate a ledger; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>ctx</computeroutput> is control object used for accountability purposes. - </para> - </listitem> - </itemizedlist> - </section> - - <section id="bk_readLedger"> - <title> Reading from ledger </title> - <para> - Read calls may request one or more consecutive entries. The following methods belong - to <computeroutput>org.apache.bookkeeper.client.LedgerHandle</computeroutput>. - </para> - - <para> - <emphasis role="bold">Synchronous read:</emphasis> - </para> - - <para> - <computeroutput> - public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry) - throws InterruptedException, BKException - </computeroutput> - </para> - - <itemizedlist> - <listitem> - <para> - <computeroutput>firstEntry</computeroutput> is the identifier of the first entry in the sequence of entries to read; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>lastEntry</computeroutput> is the identifier of the last entry in the sequence of entries to read. - </para> - </listitem> - </itemizedlist> - - <para> - <emphasis role="bold">Asynchronous read:</emphasis> - </para> - <para> - <computeroutput> - public void asyncReadEntries(long firstEntry, - long lastEntry, ReadCallback cb, Object ctx) - throws BKException, InterruptedException - </computeroutput> - </para> - - <para> - It also takes a first and a last entry identifiers. Additionaly, it takes a callback object - <computeroutput>cb</computeroutput> and a control object <computeroutput>ctx</computeroutput>. The callback object must implement - the <computeroutput>ReadCallback</computeroutput> interface in <computeroutput>org.apache.bookkeeper.client.AsyncCallback</computeroutput>, and - a class implementing it has to implement a method called <computeroutput>readComplete</computeroutput> - that has the following signature: - </para> - - <para> - <computeroutput> - void readComplete(int rc, LedgerHandle lh, Enumeration<LedgerEntry> seq, Object ctx) - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>rc</computeroutput> is a return code (please refer to <computeroutput>org.apache.bookeeper.client.BKDefs</computeroutput> for a list); - </para> - </listitem> - - <listitem> - <para> - <computeroutput>lh</computeroutput> is a <computeroutput>LedgerHandle</computeroutput> object to manipulate a ledger; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>seq</computeroutput> is a <computeroutput>Enumeration<LedgerEntry> </computeroutput> object to containing the list of entries requested; - </para> - </listitem> - - <listitem> - <para> - <computeroutput>ctx</computeroutput> is control object used for accountability purposes. - </para> - </listitem> - </itemizedlist> - </section> - - <section id="bk_deleteLedger"> - <title> Deleting a ledger </title> - <para> - Once a client is done with a ledger and is sure that nobody will ever need to read from it again, they can delete the ledger. - The following methods belong to <computeroutput>org.apache.bookkeeper.client.BookKeeper</computeroutput>. - </para> - - <para> - <emphasis role="bold">Synchronous delete:</emphasis> - </para> - - <para> - <computeroutput> - public void deleteLedger(long lId) throws InterruptedException, BKException - </computeroutput> - </para> - - <itemizedlist> - <listitem> - <para> - <computeroutput>lId</computeroutput> is the ledger identifier; - </para> - </listitem> - </itemizedlist> - - <para> - <emphasis role="bold">Asynchronous delete:</emphasis> - </para> - <para> - <computeroutput> - public void asyncDeleteLedger(long lId, DeleteCallback cb, Object ctx) - </computeroutput> - </para> - - <para> - It takes a ledger identifier. Additionally, it takes a callback object - <computeroutput>cb</computeroutput> and a control object <computeroutput>ctx</computeroutput>. The callback object must implement - the <computeroutput>DeleteCallback</computeroutput> interface in <computeroutput>org.apache.bookkeeper.client.AsyncCallback</computeroutput>, and - a class implementing it has to implement a method called <computeroutput>deleteComplete</computeroutput> - that has the following signature: - </para> - - <para> - <computeroutput> - void deleteComplete(int rc, Object ctx) - </computeroutput> - </para> - - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - <computeroutput>rc</computeroutput> is a return code (please refer to <computeroutput>org.apache.bookeeper.client.BKDefs</computeroutput> for a list); - </para> - </listitem> - - <listitem> - <para> - <computeroutput>ctx</computeroutput> is control object used for accountability purposes. - </para> - </listitem> - </itemizedlist> - </section> - </section> -</article> \ No newline at end of file
