Repository: kafka Updated Branches: refs/heads/0.10.0 a1838755a -> 23b70efa8
KAFKA-3393; Updated the docs to reflect the deprecation of block.on.buffer.full and usage of max.block.ms Author: MayureshGharat <gharatmayures...@gmail.com> Reviewers: Grant Henke <granthe...@gmail.com>, Ismael Juma <ism...@juma.me.uk> Closes #1060 from MayureshGharat/KAFKA-3393 Project: http://git-wip-us.apache.org/repos/asf/kafka/repo Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/23b70efa Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/23b70efa Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/23b70efa Branch: refs/heads/0.10.0 Commit: 23b70efa8d65ef56f7a31e22afcbe653dc736079 Parents: a183875 Author: MayureshGharat <gharatmayures...@gmail.com> Authored: Wed May 18 01:49:54 2016 +0100 Committer: Ismael Juma <ism...@juma.me.uk> Committed: Mon May 23 22:20:22 2016 +0100 ---------------------------------------------------------------------- .../org/apache/kafka/clients/producer/KafkaProducer.java | 6 +++--- .../apache/kafka/clients/producer/ProducerConfig.java | 11 +++++------ docs/upgrade.html | 1 + 3 files changed, 9 insertions(+), 9 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kafka/blob/23b70efa/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java ---------------------------------------------------------------------- diff --git a/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java b/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java index 5b05272..fd3eb09 100644 --- a/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java +++ b/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java @@ -119,8 +119,8 @@ import org.slf4j.LoggerFactory; * <p> * The <code>buffer.memory</code> controls the total amount of memory available to the producer for buffering. If records * are sent faster than they can be transmitted to the server then this buffer space will be exhausted. When the buffer space is - * exhausted additional send calls will block. For uses where you want to avoid any blocking you can set <code>block.on.buffer.full=false</code> which - * will cause the send call to result in an exception. + * exhausted additional send calls will block. The threshold for time to block is determined by <code>max.block.ms</code> after which it throws + * a TimeoutException. * <p> * The <code>key.serializer</code> and <code>value.serializer</code> instruct how to turn the key and value objects the user provides with * their <code>ProducerRecord</code> into bytes. You can use the included {@link org.apache.kafka.common.serialization.ByteArraySerializer} or @@ -420,7 +420,7 @@ public class KafkaProducer<K, V> implements Producer<K, V> { * * @throws InterruptException If the thread is interrupted while blocked * @throws SerializationException If the key or value are not valid objects given the configured serializers - * @throws BufferExhaustedException If <code>block.on.buffer.full=false</code> and the buffer is full. + * @throws TimeoutException if the time taken for fetching metadata or allocating memory for the record has surpassed <code>max.block.ms</code>. * */ @Override http://git-wip-us.apache.org/repos/asf/kafka/blob/23b70efa/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java ---------------------------------------------------------------------- diff --git a/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java b/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java index 5b7a296..4ed083b 100644 --- a/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java +++ b/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java @@ -139,14 +139,14 @@ public class ProducerConfig extends AbstractConfig { /** <code>block.on.buffer.full</code> */ /** - * @deprecated This config will be removed in a future release. Also, the {@link #METADATA_FETCH_TIMEOUT_CONFIG} is no longer honored when this property is set to true. + * @deprecated This config will be removed in a future release. Please use {@link #MAX_BLOCK_MS_CONFIG}. */ @Deprecated public static final String BLOCK_ON_BUFFER_FULL_CONFIG = "block.on.buffer.full"; private static final String BLOCK_ON_BUFFER_FULL_DOC = "When our memory buffer is exhausted we must either stop accepting new records (block) or throw errors. " - + "By default this setting is false and the producer will throw a BufferExhaustedException if a record is sent and the buffer space is full. " - + "However in some scenarios getting an error is not desirable and it is better to block. Setting this to <code>true</code> will accomplish that." - + "<em>If this property is set to true, parameter <code>" + METADATA_FETCH_TIMEOUT_CONFIG + "</code> is not longer honored.</em>" + + "By default this setting is false and the producer will no longer throw a BufferExhaustException but instead will use the {@link #MAX_BLOCK_MS_CONFIG} " + + "value to block, after which it will throw a TimeoutException. Setting this property to true will set the <code>" + MAX_BLOCK_MS_CONFIG + "</code> to Long.MAX_VALUE." + + "<em>Also if this property is set to true, parameter <code>" + METADATA_FETCH_TIMEOUT_CONFIG + "</code> is not longer honored.</em>" + "<p>" + "This parameter is deprecated and will be removed in a future release. " + "Parameter <code>" + MAX_BLOCK_MS_CONFIG + "</code> should be used instead."; @@ -154,8 +154,7 @@ public class ProducerConfig extends AbstractConfig { /** <code>buffer.memory</code> */ public static final String BUFFER_MEMORY_CONFIG = "buffer.memory"; private static final String BUFFER_MEMORY_DOC = "The total bytes of memory the producer can use to buffer records waiting to be sent to the server. If records are " - + "sent faster than they can be delivered to the server the producer will either block or throw an exception based " - + "on the preference specified by <code>" + BLOCK_ON_BUFFER_FULL_CONFIG + "</code>. " + + "sent faster than they can be delivered to the server the producer will block for <code>" + MAX_BLOCK_MS_CONFIG + "</code> after which it will throw an exception." + "<p>" + "This setting should correspond roughly to the total memory the producer will use, but is not a hard bound since " + "not all memory the producer uses is used for buffering. Some additional memory will be used for compression (if " http://git-wip-us.apache.org/repos/asf/kafka/blob/23b70efa/docs/upgrade.html ---------------------------------------------------------------------- diff --git a/docs/upgrade.html b/docs/upgrade.html index d09b9d7..dec0808 100644 --- a/docs/upgrade.html +++ b/docs/upgrade.html @@ -164,6 +164,7 @@ work with 0.10.0.x brokers. Therefore, 0.9.0.0 clients should be upgraded to 0.9 <li> Altering topic configuration from the kafka-topics.sh script (kafka.admin.TopicCommand) has been deprecated. Going forward, please use the kafka-configs.sh script (kafka.admin.ConfigCommand) for this functionality. </li> <li> The kafka-consumer-offset-checker.sh (kafka.tools.ConsumerOffsetChecker) has been deprecated. Going forward, please use kafka-consumer-groups.sh (kafka.admin.ConsumerGroupCommand) for this functionality. </li> <li> The kafka.tools.ProducerPerformance class has been deprecated. Going forward, please use org.apache.kafka.tools.ProducerPerformance for this functionality (kafka-producer-perf-test.sh will also be changed to use the new class). </li> + <li> The producer config block.on.buffer.full has been deprecated and will be removed in future release. Currently its default value has been changed to false. The KafkaProducer will no longer throw BufferExhaustedException but instead will use max.block.ms value to block, after which it will throw a TimeoutException. If block.on.buffer.full property is set to true explicitly, it will set the max.block.ms to Long.MAX_VALUE and metadata.fetch.timeout.ms will not be honoured</li> </ul> <h4><a id="upgrade_82" href="#upgrade_82">Upgrading from 0.8.1 to 0.8.2</a></h4>
