Repository: kafka-site Updated Branches: refs/heads/asf-site e2348dc11 -> 8cac98caa
http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/org/apache/kafka/streams/state/StreamsMetadata.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/org/apache/kafka/streams/state/StreamsMetadata.html b/0101/javadoc/org/apache/kafka/streams/state/StreamsMetadata.html index c9b87ac..8a326fc 100644 --- a/0101/javadoc/org/apache/kafka/streams/state/StreamsMetadata.html +++ b/0101/javadoc/org/apache/kafka/streams/state/StreamsMetadata.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>StreamsMetadata (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="../../../../../stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/org/apache/kafka/streams/state/WindowStore.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/org/apache/kafka/streams/state/WindowStore.html b/0101/javadoc/org/apache/kafka/streams/state/WindowStore.html index 33a2509..56d4203 100644 --- a/0101/javadoc/org/apache/kafka/streams/state/WindowStore.html +++ b/0101/javadoc/org/apache/kafka/streams/state/WindowStore.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>WindowStore (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="../../../../../stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/org/apache/kafka/streams/state/WindowStoreIterator.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/org/apache/kafka/streams/state/WindowStoreIterator.html b/0101/javadoc/org/apache/kafka/streams/state/WindowStoreIterator.html index 9d661cf..543cd7d 100644 --- a/0101/javadoc/org/apache/kafka/streams/state/WindowStoreIterator.html +++ b/0101/javadoc/org/apache/kafka/streams/state/WindowStoreIterator.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>WindowStoreIterator (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="../../../../../stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/org/apache/kafka/streams/state/package-frame.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/org/apache/kafka/streams/state/package-frame.html b/0101/javadoc/org/apache/kafka/streams/state/package-frame.html index f9a813c..da70ce0 100644 --- a/0101/javadoc/org/apache/kafka/streams/state/package-frame.html +++ b/0101/javadoc/org/apache/kafka/streams/state/package-frame.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>org.apache.kafka.streams.state (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="../../../../../stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/org/apache/kafka/streams/state/package-summary.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/org/apache/kafka/streams/state/package-summary.html b/0101/javadoc/org/apache/kafka/streams/state/package-summary.html index af98720..46a55ce 100644 --- a/0101/javadoc/org/apache/kafka/streams/state/package-summary.html +++ b/0101/javadoc/org/apache/kafka/streams/state/package-summary.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>org.apache.kafka.streams.state (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="../../../../../stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/org/apache/kafka/streams/state/package-tree.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/org/apache/kafka/streams/state/package-tree.html b/0101/javadoc/org/apache/kafka/streams/state/package-tree.html index a6d9a83..adbf434 100644 --- a/0101/javadoc/org/apache/kafka/streams/state/package-tree.html +++ b/0101/javadoc/org/apache/kafka/streams/state/package-tree.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>org.apache.kafka.streams.state Class Hierarchy (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="../../../../../stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/overview-frame.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/overview-frame.html b/0101/javadoc/overview-frame.html index 2bf11a9..414f751 100644 --- a/0101/javadoc/overview-frame.html +++ b/0101/javadoc/overview-frame.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>Overview List (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/overview-summary.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/overview-summary.html b/0101/javadoc/overview-summary.html index 1cee6f7..170693e 100644 --- a/0101/javadoc/overview-summary.html +++ b/0101/javadoc/overview-summary.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>Overview (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/overview-tree.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/overview-tree.html b/0101/javadoc/overview-tree.html index 5410aed..af317ca 100644 --- a/0101/javadoc/overview-tree.html +++ b/0101/javadoc/overview-tree.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>Class Hierarchy (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/javadoc/serialized-form.html ---------------------------------------------------------------------- diff --git a/0101/javadoc/serialized-form.html b/0101/javadoc/serialized-form.html index c8b1be9..4d973d8 100644 --- a/0101/javadoc/serialized-form.html +++ b/0101/javadoc/serialized-form.html @@ -2,9 +2,9 @@ <!-- NewPage --> <html lang="en"> <head> -<!-- Generated by javadoc (version 1.7.0_79) on Tue Oct 04 12:37:09 PDT 2016 --> +<!-- Generated by javadoc (version 1.7.0_79) on Mon Oct 10 10:43:54 PDT 2016 --> <title>Serialized Form (kafka 0.10.1.0 API)</title> -<meta name="date" content="2016-10-04"> +<meta name="date" content="2016-10-10"> <link rel="stylesheet" type="text/css" href="stylesheet.css" title="Style"> </head> <body> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/ops.html ---------------------------------------------------------------------- diff --git a/0101/ops.html b/0101/ops.html index 7565738..a65269a 100644 --- a/0101/ops.html +++ b/0101/ops.html @@ -695,12 +695,12 @@ We do graphing and alerting on the following metrics: </tr> <tr> <td>Requests waiting in the producer purgatory</td> - <td>kafka.server:type=ProducerRequestPurgatory,name=PurgatorySize</td> + <td>kafka.server:type=DelayedOperationPurgatory,name=PurgatorySize,delayedOperation=Produce</td> <td>non-zero if ack=-1 is used</td> </tr> <tr> <td>Requests waiting in the fetch purgatory</td> - <td>kafka.server:type=FetchRequestPurgatory,name=PurgatorySize</td> + <td>kafka.server:type=DelayedOperationPurgatory,name=PurgatorySize,delayedOperation=Fetch</td> <td>size depends on fetch.wait.max.ms in the consumer</td> </tr> <tr> http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/quickstart.html ---------------------------------------------------------------------- diff --git a/0101/quickstart.html b/0101/quickstart.html index eb310f5..4e03059 100644 --- a/0101/quickstart.html +++ b/0101/quickstart.html @@ -18,6 +18,7 @@ <h3><a id="quickstart" href="#quickstart">1.3 Quick Start</a></h3> This tutorial assumes you are starting fresh and have no existing Kafka or ZooKeeper data. +Since Kafka console scripts are different for Unix-based and Windows platforms, on Windows platforms use <code>bin\windows\</code> instead of <code>bin/</code>, and change the script extension to <code>.bat</code>. <h4><a id="quickstart_download" href="#quickstart_download">Step 1: Download the code</a></h4> @@ -93,7 +94,7 @@ All of the command line tools have additional options; running the command with So far we have been running against a single broker, but that's no fun. For Kafka, a single broker is just a cluster of size one, so nothing much changes other than starting a few more broker instances. But just to get feel for it, let's expand our cluster to three nodes (still all on our local machine). <p> -First we make a config file for each of the brokers: +First we make a config file for each of the brokers (on Windows use the <code>copy</code> command instead): <pre> > <b>cp config/server.properties config/server-1.properties</b> > <b>cp config/server.properties config/server-2.properties</b> @@ -173,6 +174,13 @@ Now let's test out fault-tolerance. Broker 1 was acting as the leader so let's k > <b>kill -9 7564</b> </pre> +On Windows use: +<pre> +> <b>wmic process get processid,caption,commandline | find "java.exe" | find "server-1.properties"</b> +java.exe java -Xmx1G -Xms1G -server -XX:+UseG1GC ... build\libs\kafka_2.10-0.10.1.0.jar" kafka.Kafka config\server-1.properties <i>644</i> +> <b>taskkill /pid 644 /f</b> +</pre> + Leadership has switched to one of the slaves and node 1 is no longer in the in-sync replica set: <pre> > <b>bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic</b> @@ -297,6 +305,12 @@ We will now prepare input data to a Kafka topic, which will subsequently process <pre> > <b>echo -e "all streams lead to kafka\nhello kafka streams\njoin kafka summit" > file-input.txt</b> </pre> +Or on Windows: +<pre> +> <b>echo all streams lead to kafka> file-input.txt</b> +> <b>echo hello kafka streams>> file-input.txt</b> +> <b>echo|set /p=join kafka summit>> file-input.txt</b> +</pre> <p> Next, we send this input data to the input topic named <b>streams-file-input</b> using the console producer (in practice, @@ -313,7 +327,7 @@ stream data will likely be flowing continuously into Kafka where the application <pre> -> <b>cat file-input.txt | bin/kafka-console-producer.sh --broker-list localhost:9092 --topic streams-file-input</b> +> <b>bin/kafka-console-producer.sh --broker-list localhost:9092 --topic streams-file-input < file-input.txt</b> </pre> <p> @@ -349,12 +363,9 @@ with the following output data being printed to the console: <pre> all 1 -streams 1 lead 1 to 1 -kafka 1 hello 1 -kafka 2 streams 2 join 1 kafka 3 http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/upgrade.html ---------------------------------------------------------------------- diff --git a/0101/upgrade.html b/0101/upgrade.html index ca16327..d140ec2 100644 --- a/0101/upgrade.html +++ b/0101/upgrade.html @@ -19,26 +19,29 @@ <h3><a id="upgrade" href="#upgrade">1.5 Upgrading From Previous Versions</a></h3> -<h4><a id="upgrade_10_1" href="#upgrade_10_1">Upgrading from 0.10.0.X to 0.10.1.0</a></h4> +<h4><a id="upgrade_10_1" href="#upgrade_10_1">Upgrading from 0.8.x, 0.9.x or 0.10.0.X to 0.10.1.0</a></h4> 0.10.1.0 has wire protocol changes. By following the recommended rolling upgrade plan below, you guarantee no downtime during the upgrade. However, please notice the <a href="#upgrade_10_1_breaking">Potential breaking changes in 0.10.1.0</a> before upgrade. <br> -Note: Because new protocols are introduced, it is important to upgrade your Kafka clusters before upgrading your clients. +Note: Because new protocols are introduced, it is important to upgrade your Kafka clusters before upgrading your clients (i.e. 0.10.1.x clients +only support 0.10.1.x or later brokers while 0.10.1.x brokers also support older clients). <p><b>For a rolling upgrade:</b></p> <ol> <li> Update server.properties file on all brokers and add the following properties: <ul> - <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g. 0.8.2, 0.9.0.0 or 0.10.0.0).</li> + <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g. 0.8.2.0, 0.9.0.0 or 0.10.0.0).</li> <li>log.message.format.version=CURRENT_KAFKA_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact following the upgrade</a> for the details on what this configuration does.) </ul> </li> - <li> Upgrade the brokers. This can be done a broker at a time by simply bringing it down, updating the code, and restarting it. </li> - <li> Once the entire cluster is upgraded, bump the protocol version by editing inter.broker.protocol.version and setting it to 0.10.1.0. NOTE: If your previous message format version is before 0.10.0, you shouldn't touch log.message.format.version yet - this parameter should only change once all consumers have been upgraded to 0.10.0.0 or later.</li> + <li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. </li> + <li> Once the entire cluster is upgraded, bump the protocol version by editing inter.broker.protocol.version and setting it to 0.10.1.0. </li> + <li> If your previous message format is 0.10.0, change log.message.format.version to 0.10.1 (this is a no-op as the message format is the same for both 0.10.0 and 0.10.1). + If your previous message format version is lower than 0.10.0, do not change log.message.format.version yet - this parameter should only change once all consumers have been upgraded to 0.10.0.0 or later.</li> <li> Restart the brokers one by one for the new protocol version to take effect. </li> - <li> Once all consumers have been upgraded to 0.10.0, change log.message.format.version to 0.10.1 on each broker and restart them one by one. - </li> + <li> If log.message.format.version is still lower than 0.10.0 at this point, wait until all consumers have been upgraded to 0.10.0 or later, + then change log.message.format.version to 0.10.1 on each broker and restart them one by one. </li> </ol> <p><b>Note:</b> If you are willing to accept downtime, you can simply take all the brokers down, update the code and start all of them. They will start with the new protocol by default. http://git-wip-us.apache.org/repos/asf/kafka-site/blob/8cac98ca/0101/uses.html ---------------------------------------------------------------------- diff --git a/0101/uses.html b/0101/uses.html index 5b97272..6214ee6 100644 --- a/0101/uses.html +++ b/0101/uses.html @@ -15,8 +15,6 @@ limitations under the License. --> -<h3><a id="uses" href="#uses">1.2 Use Cases</a></h3> - Here is a description of a few of the popular use cases for Apache Kafka. For an overview of a number of these areas in action, see <a href="http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying";>this blog post</a>. <h4><a id="uses_messaging" href="#uses_messaging">Messaging</a></h4>
