[jira] [Commented] (KAFKA-6792) Wrong pointer in the link for stream dsl
[ https://issues.apache.org/jira/browse/KAFKA-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438990#comment-16438990 ] robin m commented on KAFKA-6792: https://github.com/apache/kafka/pull/4877 > Wrong pointer in the link for stream dsl > > > Key: KAFKA-6792 > URL: https://issues.apache.org/jira/browse/KAFKA-6792 > Project: Kafka > Issue Type: Bug > Components: documentation, streams >Reporter: robin m >Priority: Major > > Wrong pointer in the link for stream dsl. > actual is : > [http://kafka.apache.org/11/documentation/streams/developer-guide#streams|http://kafka.apache.org/11/documentation/streams/developer-guide/#streams]_dsl > correct is : > http://kafka.apache.org/11/documentation/streams/developer-guide/dsl-api.html#streams-dsl -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KAFKA-6790) Broken link in documentation for processor nodes
[ https://issues.apache.org/jira/browse/KAFKA-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-6790: --- Component/s: documentation > Broken link in documentation for processor nodes > > > Key: KAFKA-6790 > URL: https://issues.apache.org/jira/browse/KAFKA-6790 > Project: Kafka > Issue Type: Bug > Components: documentation, streams >Reporter: robin m >Priority: Major > Fix For: 1.1.0 > > > The page here > [https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html] > talks about processor nodes and refers to non existing links. > Broken link (appears twice in the same document): > [https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor] > To find this search for the word "processor node" on the page > memory-management , the ones which are links are broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KAFKA-6792) Wrong pointer in the link for stream dsl
[ https://issues.apache.org/jira/browse/KAFKA-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-6792: --- Component/s: streams documentation > Wrong pointer in the link for stream dsl > > > Key: KAFKA-6792 > URL: https://issues.apache.org/jira/browse/KAFKA-6792 > Project: Kafka > Issue Type: Bug > Components: documentation, streams >Reporter: robin m >Priority: Major > > Wrong pointer in the link for stream dsl. > actual is : > [http://kafka.apache.org/11/documentation/streams/developer-guide#streams|http://kafka.apache.org/11/documentation/streams/developer-guide/#streams]_dsl > correct is : > http://kafka.apache.org/11/documentation/streams/developer-guide/dsl-api.html#streams-dsl -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KAFKA-6792) Wrong pointer in the link for stream dsl
robin m created KAFKA-6792: -- Summary: Wrong pointer in the link for stream dsl Key: KAFKA-6792 URL: https://issues.apache.org/jira/browse/KAFKA-6792 Project: Kafka Issue Type: Bug Reporter: robin m Wrong pointer in the link for stream dsl. actual is : [http://kafka.apache.org/11/documentation/streams/developer-guide#streams|http://kafka.apache.org/11/documentation/streams/developer-guide/#streams]_dsl correct is : http://kafka.apache.org/11/documentation/streams/developer-guide/dsl-api.html#streams-dsl -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-6592) NullPointerException thrown when executing ConsoleCosumer with deserializer set to `WindowedDeserializer`
[
https://issues.apache.org/jira/browse/KAFKA-6592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438962#comment-16438962
]
ASF GitHub Bot commented on KAFKA-6592:
---
guozhangwang closed pull request #4864: KAFKA-6592: Follow-up
URL: https://github.com/apache/kafka/pull/4864
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/clients/src/test/java/org/apache/kafka/test/MockDeserializer.java
b/clients/src/test/java/org/apache/kafka/test/MockDeserializer.java
index 99551f718a9..ac2865e9bb8 100644
--- a/clients/src/test/java/org/apache/kafka/test/MockDeserializer.java
+++ b/clients/src/test/java/org/apache/kafka/test/MockDeserializer.java
@@ -31,6 +31,9 @@
public static ClusterResource noClusterId = new
ClusterResource("no_cluster_id");
public static AtomicReference clusterIdBeforeDeserialize
= new AtomicReference<>(noClusterId);
+public boolean isKey;
+public Map configs;
+
public static void resetStaticVariables() {
initCount = new AtomicInteger(0);
closeCount = new AtomicInteger(0);
@@ -44,6 +47,8 @@ public MockDeserializer() {
@Override
public void configure(Map configs, boolean isKey) {
+this.configs = configs;
+this.isKey = isKey;
}
@Override
diff --git a/core/src/main/scala/kafka/tools/ConsoleConsumer.scala
b/core/src/main/scala/kafka/tools/ConsoleConsumer.scala
index 9df4fb42070..5139324ec17 100755
--- a/core/src/main/scala/kafka/tools/ConsoleConsumer.scala
+++ b/core/src/main/scala/kafka/tools/ConsoleConsumer.scala
@@ -19,8 +19,9 @@ package kafka.tools
import java.io.PrintStream
import java.nio.charset.StandardCharsets
+import java.util
import java.util.concurrent.CountDownLatch
-import java.util.{Locale, Properties, Random}
+import java.util.{Locale, Map, Properties, Random}
import com.typesafe.scalalogging.LazyLogging
import joptsimple._
@@ -46,11 +47,6 @@ import scala.collection.JavaConverters._
object ConsoleConsumer extends Logging {
var messageCount = 0
- // Keep same names with StreamConfig.DEFAULT_WINDOWED_KEY_SERDE_INNER_CLASS
- // and StreamConfig.DEFAULT_WINDOWED_VALUE_SERDE_INNER_CLASS
- // visible for testing
- private[tools] val innerKeySerdeName = "default.windowed.key.serde.inner"
- private[tools] val innerValueSerdeName = "default.windowed.value.serde.inner"
private val shutdownLatch = new CountDownLatch(1)
@@ -306,8 +302,8 @@ object ConsoleConsumer extends Logging {
"\tline.separator=\n" +
"\tkey.deserializer=\n" +
"\tvalue.deserializer=\n" +
-"\tdefault.windowed.key.serde.inner=\n" +
-"\tdefault.windowed.value.serde.inner=")
+"\nUsers can also pass in customized properties for their formatter;
more specifically, users " +
+"can pass in properties keyed with \'key.deserializer.\' and
\'value.deserializer.\' prefixes to configure their deserializers.")
.withRequiredArg
.describedAs("prop")
.ofType(classOf[String])
@@ -344,18 +340,6 @@ object ConsoleConsumer extends Logging {
.withRequiredArg
.describedAs("deserializer for values")
.ofType(classOf[String])
-val innerKeyDeserializerOpt = parser.accepts(innerKeySerdeName,
- "inner serde for key when windowed deserialzier is used; would be
ignored otherwise. " +
-"For example:
org.apache.kafka.common.serialization.Serdes\\$StringSerde")
- .withRequiredArg
- .describedAs("inner serde for key")
- .ofType(classOf[String])
-val innerValueDeserializerOpt = parser.accepts(innerValueSerdeName,
- "inner serde for value when windowed deserialzier is used; would be
ignored otherwise. " +
-"For example:
org.apache.kafka.common.serialization.Serdes\\$StringSerde")
- .withRequiredArg
- .describedAs("inner serde for values")
- .ofType(classOf[String])
val enableSystestEventsLoggingOpt = parser.accepts("enable-systest-events",
"Log lifecycle events
of the consumer in addition to logging consumed " +
"messages. (This is
specific for system tests.)")
@@ -400,8 +384,6 @@ object ConsoleConsumer extends Logging {
val bootstrapServer = options.valueOf(bootstrapServerOpt)
val keyDeserializer = options.valueOf(keyDeserializerOpt)
val valueDeserializer = options.valueOf(valueDeserializerOpt)
-val innerKeyDeserializer = options.valueOf(innerKeyDeserializerOpt)
-val innerValueDeserializer = options.valueOf(innerValueDeserializerOpt)
val isolationLevel = options.valueOf(isolationLevelOpt).toString
val
[jira] [Commented] (KAFKA-6788) Grouping consumer requests per consumer coordinator in admin client
[ https://issues.apache.org/jira/browse/KAFKA-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438957#comment-16438957 ] ASF GitHub Bot commented on KAFKA-6788: --- cyrusv opened a new pull request #4875: KAFKA-6788: Grouping consumer requests per consumer coordinator in admin client URL: https://github.com/apache/kafka/pull/4875 Resolves KAFKA-6788 for the deleteGroup case For the "describeGroup" case, we build a hash-set and `continue` when we have already described the group. The describe API lets us submit all groups so this is the only work we must do. For the "deleteGroup" case, we ask the Coordinator which groups it knows about, and delete those. Also use a hashset to `continue` over groups we have already deleted. Passes tests, no new testing or documentation necessary. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Grouping consumer requests per consumer coordinator in admin client > --- > > Key: KAFKA-6788 > URL: https://issues.apache.org/jira/browse/KAFKA-6788 > Project: Kafka > Issue Type: Improvement > Components: admin >Reporter: Guozhang Wang >Priority: Major > Labels: newbie++ > > In KafkaAdminClient, for some requests like describeGroup and deleteGroup, we > will first try to get the coordinator for each requested group id, and then > send the corresponding request for that group id. However, different group > ids could be hosted on the same coordinator, and these requests do support > multi group ids be sent within the same request. So we can consider optimize > it by grouping the requests per coordinator destination. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KAFKA-6531) SocketServerTest#closingChannelException fails sometimes
[
https://issues.apache.org/jira/browse/KAFKA-6531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved KAFKA-6531.
---
Resolution: Cannot Reproduce
> SocketServerTest#closingChannelException fails sometimes
>
>
> Key: KAFKA-6531
> URL: https://issues.apache.org/jira/browse/KAFKA-6531
> Project: Kafka
> Issue Type: Test
> Components: core
>Reporter: Ted Yu
>Priority: Minor
>
> From
> https://builds.apache.org/job/kafka-trunk-jdk9/361/testReport/junit/kafka.network/SocketServerTest/closingChannelException/
> :
> {code}
> java.lang.AssertionError: Channels not removed
> at kafka.utils.TestUtils$.fail(TestUtils.scala:355)
> at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:865)
> at
> kafka.network.SocketServerTest.assertProcessorHealthy(SocketServerTest.scala:914)
> at
> kafka.network.SocketServerTest.$anonfun$closingChannelException$1(SocketServerTest.scala:763)
> at
> kafka.network.SocketServerTest.$anonfun$closingChannelException$1$adapted(SocketServerTest.scala:747)
> {code}
> Among the test output, I saw:
> {code}
> [2018-02-04 18:51:15,995] ERROR Processor 0 closed connection from
> /127.0.0.1:48261 (kafka.network.SocketServerTest$$anon$5$$anon$1:73)
> java.lang.IllegalStateException: There is already a connection for id
> 127.0.0.1:1-127.0.0.1:2-0
> at
> org.apache.kafka.common.network.Selector.ensureNotRegistered(Selector.java:260)
> at org.apache.kafka.common.network.Selector.register(Selector.java:254)
> at
> kafka.network.SocketServerTest$TestableSelector.super$register(SocketServerTest.scala:1043)
> at
> kafka.network.SocketServerTest$TestableSelector.$anonfun$register$2(SocketServerTest.scala:1043)
> at
> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
> at
> kafka.network.SocketServerTest$TestableSelector.runOp(SocketServerTest.scala:1037)
> at
> kafka.network.SocketServerTest$TestableSelector.register(SocketServerTest.scala:1043)
> at
> kafka.network.Processor.configureNewConnections(SocketServer.scala:723)
> at kafka.network.Processor.run(SocketServer.scala:532)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KAFKA-5846) Use singleton NoOpConsumerRebalanceListener in subscribe() call where listener is not specified
[
https://issues.apache.org/jira/browse/KAFKA-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16191405#comment-16191405
]
Ted Yu edited comment on KAFKA-5846 at 4/16/18 1:46 AM:
Patch looks good to me .
was (Author: yuzhih...@gmail.com):
Patch looks good to me.
> Use singleton NoOpConsumerRebalanceListener in subscribe() call where
> listener is not specified
> ---
>
> Key: KAFKA-5846
> URL: https://issues.apache.org/jira/browse/KAFKA-5846
> Project: Kafka
> Issue Type: Task
> Components: clients
>Reporter: Ted Yu
>Assignee: Kamal Chandraprakash
>Priority: Minor
>
> Currently KafkaConsumer creates instance of NoOpConsumerRebalanceListener for
> each subscribe() call where ConsumerRebalanceListener is not specified:
> {code}
> public void subscribe(Pattern pattern) {
> subscribe(pattern, new NoOpConsumerRebalanceListener());
> {code}
> We can create a singleton NoOpConsumerRebalanceListener to be used in such
> scenarios.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KAFKA-6058) KIP-222: Add "describe consumer groups" and "list consumer groups" to KafkaAdminClient
[
https://issues.apache.org/jira/browse/KAFKA-6058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-6058.
--
Resolution: Fixed
> KIP-222: Add "describe consumer groups" and "list consumer groups" to
> KafkaAdminClient
> --
>
> Key: KAFKA-6058
> URL: https://issues.apache.org/jira/browse/KAFKA-6058
> Project: Kafka
> Issue Type: Improvement
> Components: clients
>Reporter: Matthias J. Sax
>Assignee: Jorge Quilcate
>Priority: Major
> Labels: kip-222
> Fix For: 1.2.0
>
>
> {{KafkaAdminClient}} does not allow to get information about consumer groups.
> This feature is supported by old {{kafka.admin.AdminClient}} though.
> We should add {{KafkaAdminClient#describeConsumerGroups()}} and
> {{KafkaAdminClient#listConsumerGroup()}}.
> Associated KIP: KIP-222
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-6611) Re-write simple benchmark in system tests with JMXTool
[
https://issues.apache.org/jira/browse/KAFKA-6611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438779#comment-16438779
]
ASF GitHub Bot commented on KAFKA-6611:
---
guozhangwang closed pull request #4854: KAFKA-6611, PART II: Improve Streams
SimpleBenchmark
URL: https://github.com/apache/kafka/pull/4854
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/clients/src/main/java/org/apache/kafka/clients/NetworkClient.java
b/clients/src/main/java/org/apache/kafka/clients/NetworkClient.java
index 3a9cbc4d664..fc4745d8c6e 100644
--- a/clients/src/main/java/org/apache/kafka/clients/NetworkClient.java
+++ b/clients/src/main/java/org/apache/kafka/clients/NetworkClient.java
@@ -832,6 +832,7 @@ public long maybeUpdate(long now) {
long waitForMetadataFetch = this.metadataFetchInProgress ?
requestTimeoutMs : 0;
long metadataTimeout = Math.max(timeToNextMetadataUpdate,
waitForMetadataFetch);
+
if (metadataTimeout > 0) {
return metadataTimeout;
}
diff --git
a/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java
b/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java
index 0514c995635..6a108269f9a 100644
---
a/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java
+++
b/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java
@@ -252,6 +252,9 @@ private long sendProducerData(long now) {
// and request metadata update, since there are messages to send
to the topic.
for (String topic : result.unknownLeaderTopics)
this.metadata.add(topic);
+
+log.debug("Requesting metadata update due to unknown leader topics
from the batched records: {}", result.unknownLeaderTopics);
+
this.metadata.requestUpdate();
}
@@ -557,9 +560,13 @@ else if (error == Errors.CLUSTER_AUTHORIZATION_FAILED)
failBatch(batch, response, exception, batch.attempts() <
this.retries);
}
if (error.exception() instanceof InvalidMetadataException) {
-if (error.exception() instanceof
UnknownTopicOrPartitionException)
+if (error.exception() instanceof
UnknownTopicOrPartitionException) {
log.warn("Received unknown topic or partition error in
produce request on partition {}. The " +
"topic/partition may not exist or the user may not
have Describe access to it", batch.topicPartition);
+} else {
+log.warn("Received invalid metadata error in produce
request on partition {} due to {}. Going " +
+"to request metadata update now",
batch.topicPartition, error.exception().toString());
+}
metadata.requestUpdate();
}
diff --git
a/streams/src/test/java/org/apache/kafka/streams/perf/SimpleBenchmark.java
b/streams/src/test/java/org/apache/kafka/streams/perf/SimpleBenchmark.java
index c66d78b7310..faab52e2118 100644
--- a/streams/src/test/java/org/apache/kafka/streams/perf/SimpleBenchmark.java
+++ b/streams/src/test/java/org/apache/kafka/streams/perf/SimpleBenchmark.java
@@ -40,6 +40,7 @@
import org.apache.kafka.streams.kstream.JoinWindows;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.KTable;
+import org.apache.kafka.streams.kstream.TimeWindows;
import org.apache.kafka.streams.kstream.ValueJoiner;
import org.apache.kafka.streams.processor.AbstractProcessor;
import org.apache.kafka.streams.processor.Processor;
@@ -48,9 +49,7 @@
import org.apache.kafka.streams.state.KeyValueStore;
import org.apache.kafka.streams.state.StoreBuilder;
import org.apache.kafka.streams.state.Stores;
-import org.apache.kafka.test.TestUtils;
-import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
@@ -59,7 +58,6 @@
import java.util.Random;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicInteger;
/**
* Class that provides support for a series of benchmarks. It is usually
driven by
@@ -77,114 +75,126 @@
* is still running "consume"
*/
public class SimpleBenchmark {
+private static final String LOADING_PRODUCER_CLIENT_ID =
"simple-benchmark-loading-producer";
-final String kafka;
-final Boolean loadPhase;
-final String testName;
-final int numThreads;
-final Properties props;
-static final String ALL_TESTS = "all";
-private static final String SOURCE_TOPIC = "simpleBenchmarkSourceTopic";
+
[jira] [Commented] (KAFKA-6058) KIP-222: Add "describe consumer groups" and "list consumer groups" to KafkaAdminClient
[
https://issues.apache.org/jira/browse/KAFKA-6058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438777#comment-16438777
]
ASF GitHub Bot commented on KAFKA-6058:
---
guozhangwang closed pull request #4856: KAFKA-6058: Refactor consumer API
result return types
URL: https://github.com/apache/kafka/pull/4856
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/clients/src/main/java/org/apache/kafka/clients/admin/DeleteConsumerGroupsResult.java
b/clients/src/main/java/org/apache/kafka/clients/admin/DeleteConsumerGroupsResult.java
index b4bce264405..dd6835cf10c 100644
---
a/clients/src/main/java/org/apache/kafka/clients/admin/DeleteConsumerGroupsResult.java
+++
b/clients/src/main/java/org/apache/kafka/clients/admin/DeleteConsumerGroupsResult.java
@@ -29,13 +29,24 @@
*/
@InterfaceStability.Evolving
public class DeleteConsumerGroupsResult {
-final KafkaFuture> futures;
+private final Map futures;
-DeleteConsumerGroupsResult(KafkaFuture>
futures) {
+DeleteConsumerGroupsResult(final Map futures) {
this.futures = futures;
}
-public KafkaFuture> deletedGroups() {
+/**
+ * Return a map from group id to futures which can be used to check the
status of
+ * individual deletions.
+ */
+public Map deletedGroups() {
return futures;
}
+
+/**
+ * Return a future which succeeds only if all the consumer group deletions
succeed.
+ */
+public KafkaFuture all() {
+return KafkaFuture.allOf(futures.values().toArray(new KafkaFuture[0]));
+}
}
diff --git
a/clients/src/main/java/org/apache/kafka/clients/admin/DescribeConsumerGroupsResult.java
b/clients/src/main/java/org/apache/kafka/clients/admin/DescribeConsumerGroupsResult.java
index adde031b678..ac2189cc6dc 100644
---
a/clients/src/main/java/org/apache/kafka/clients/admin/DescribeConsumerGroupsResult.java
+++
b/clients/src/main/java/org/apache/kafka/clients/admin/DescribeConsumerGroupsResult.java
@@ -32,16 +32,23 @@
@InterfaceStability.Evolving
public class DescribeConsumerGroupsResult {
-private final KafkaFuture> futures;
+private final Map futures;
-public DescribeConsumerGroupsResult(KafkaFuture> futures) {
+public DescribeConsumerGroupsResult(final Map futures) {
this.futures = futures;
}
/**
- * Return a map from group name to futures which can be used to check the
description of a consumer group.
+ * Return a map from group id to futures which can be used to check the
description of a consumer group.
*/
-public KafkaFuture>
describedGroups() {
+public Map
describedGroups() {
return futures;
}
+
+/**
+ * Return a future which succeeds only if all the consumer group
description succeed.
+ */
+public KafkaFuture all() {
+return KafkaFuture.allOf(futures.values().toArray(new KafkaFuture[0]));
+}
}
diff --git
a/clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java
b/clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java
index 50bcfd38856..fa3f943555b 100644
--- a/clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java
+++ b/clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java
@@ -46,6 +46,7 @@
import org.apache.kafka.common.errors.AuthenticationException;
import org.apache.kafka.common.errors.BrokerNotAvailableException;
import org.apache.kafka.common.errors.DisconnectException;
+import org.apache.kafka.common.errors.InvalidGroupIdException;
import org.apache.kafka.common.errors.InvalidRequestException;
import org.apache.kafka.common.errors.InvalidTopicException;
import org.apache.kafka.common.errors.RetriableException;
@@ -53,6 +54,7 @@
import org.apache.kafka.common.errors.UnknownServerException;
import org.apache.kafka.common.errors.UnsupportedVersionException;
import org.apache.kafka.common.internals.KafkaFutureImpl;
+import org.apache.kafka.common.internals.Topic;
import org.apache.kafka.common.metrics.JmxReporter;
import org.apache.kafka.common.metrics.MetricConfig;
import org.apache.kafka.common.metrics.Metrics;
@@ -916,8 +918,11 @@ private void failCalls(long now, List calls,
AuthenticationException authe
* @param correlationIdToCall A map of correlation IDs to calls.
* @param callsInFlight A map of nodes to the calls they have
in flight.
[jira] [Resolved] (KAFKA-6790) Broken link in documentation for processor nodes
[ https://issues.apache.org/jira/browse/KAFKA-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-6790. -- Resolution: Fixed Fix Version/s: 1.1.0 > Broken link in documentation for processor nodes > > > Key: KAFKA-6790 > URL: https://issues.apache.org/jira/browse/KAFKA-6790 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: robin m >Priority: Major > Fix For: 1.1.0 > > > The page here > [https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html] > talks about processor nodes and refers to non existing links. > Broken link (appears twice in the same document): > [https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor] > To find this search for the word "processor node" on the page > memory-management , the ones which are links are broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-6790) Broken link in documentation for processor nodes
[ https://issues.apache.org/jira/browse/KAFKA-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438768#comment-16438768 ] ASF GitHub Bot commented on KAFKA-6790: --- guozhangwang closed pull request #4874: KAFKA-6790 URL: https://github.com/apache/kafka/pull/4874 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/docs/streams/core-concepts.html b/docs/streams/core-concepts.html index 889fe06c45f..16502879d6b 100644 --- a/docs/streams/core-concepts.html +++ b/docs/streams/core-concepts.html @@ -63,7 +63,7 @@ Stream Processing Topology A stream is the most important abstraction provided by Kafka Streams: it represents an unbounded, continuously updating data set. A stream is an ordered, replayable, and fault-tolerant sequence of immutable data records, where a data record is defined as a key-value pair. A stream processing application is any program that makes use of the Kafka Streams library. It defines its computational logic through one or more processor topologies, where a processor topology is a graph of stream processors (nodes) that are connected by streams (edges). -A stream processor is a node in the processor topology; it represents a processing step to transform data in streams by receiving one input record at a time from its upstream processors in the topology, applying its operation to it, and may subsequently produce one or more output records to its downstream processors. +A stream processor is a node in the processor topology; it represents a processing step to transform data in streams by receiving one input record at a time from its upstream processors in the topology, applying its operation to it, and may subsequently produce one or more output records to its downstream processors. There are two special processors in the topology: diff --git a/docs/streams/developer-guide/memory-mgmt.html b/docs/streams/developer-guide/memory-mgmt.html index 6c6fd2f76cc..a73a814be11 100644 --- a/docs/streams/developer-guide/memory-mgmt.html +++ b/docs/streams/developer-guide/memory-mgmt.html @@ -55,9 +55,9 @@ For such KTable instances, the record cache is used for: Internal caching and compacting of output records before they are written by the underlying stateful - processor node to its internal state stores. + processor node to its internal state stores. Internal caching and compacting of output records before they are forwarded from the underlying stateful - processor node to any of its downstream processor nodes. + processor node to any of its downstream processor nodes. Use the following example to understand the behaviors with and without record caching. In this example, the input is a KStreamString, Integer with the records K,V: A, 1, D, 5, A, 20, A, 300. The focus in this example is This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Broken link in documentation for processor nodes > > > Key: KAFKA-6790 > URL: https://issues.apache.org/jira/browse/KAFKA-6790 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: robin m >Priority: Major > Fix For: 1.1.0 > > > The page here > [https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html] > talks about processor nodes and refers to non existing links. > Broken link (appears twice in the same document): > [https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor] > To find this search for the word "processor node" on the page > memory-management , the ones which are links are broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KAFKA-6790) Broken link in documentation for processor nodes
[ https://issues.apache.org/jira/browse/KAFKA-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-6790: - Component/s: streams > Broken link in documentation for processor nodes > > > Key: KAFKA-6790 > URL: https://issues.apache.org/jira/browse/KAFKA-6790 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: robin m >Priority: Major > > The page here > [https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html] > talks about processor nodes and refers to non existing links. > Broken link (appears twice in the same document): > [https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor] > To find this search for the word "processor node" on the page > memory-management , the ones which are links are broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KAFKA-6791) Add a CoordinatorNodeProvider in KafkaAdminClient
Guozhang Wang created KAFKA-6791: Summary: Add a CoordinatorNodeProvider in KafkaAdminClient Key: KAFKA-6791 URL: https://issues.apache.org/jira/browse/KAFKA-6791 Project: Kafka Issue Type: Improvement Components: admin Reporter: Guozhang Wang As we add more and more coordinator-related requests to the admin client, we can consider adding a CoordinatorNodeProvider to consolidate the common logic pattern of finding the coordinator first, then send the corresponding request. Note that 1) with this provider interface it is almost not possible to batch multiple groupIds per coordinator; there has to be a little more refactoring to make it work. 2) for some requests like list consumers, group ids are not known beforehand and hence we cannot use this provider as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KAFKA-6790) Broken link in documentation for processor nodes
[ https://issues.apache.org/jira/browse/KAFKA-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] robin m updated KAFKA-6790: --- Summary: Broken link in documentation for processor nodes (was: Broken link in documentation for processor nodes on Kafka Streams) > Broken link in documentation for processor nodes > > > Key: KAFKA-6790 > URL: https://issues.apache.org/jira/browse/KAFKA-6790 > Project: Kafka > Issue Type: Bug >Reporter: robin m >Priority: Major > > The page here > [https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html] > talks about processor nodes and refers to non existing links. > Broken link (appears twice in the same document): > [https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor] > To find this search for the word "processor node" on the page > memory-management , the ones which are links are broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KAFKA-6790) Missing link in documentation for processor nodes on Kafka Streams
[ https://issues.apache.org/jira/browse/KAFKA-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] robin m updated KAFKA-6790: --- Summary: Missing link in documentation for processor nodes on Kafka Streams (was: Missing documentation for processor nodes on Kafka Streams) > Missing link in documentation for processor nodes on Kafka Streams > -- > > Key: KAFKA-6790 > URL: https://issues.apache.org/jira/browse/KAFKA-6790 > Project: Kafka > Issue Type: Bug >Reporter: robin m >Priority: Major > > The page here > [https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html] > talks about processor nodes and refers to non existing links. > Broken link (appears twice in the same document): > [https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor] > To find this search for the word "processor node" on the page > memory-management , the ones which are links are broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KAFKA-6790) Broken link in documentation for processor nodes on Kafka Streams
[ https://issues.apache.org/jira/browse/KAFKA-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] robin m updated KAFKA-6790: --- Summary: Broken link in documentation for processor nodes on Kafka Streams (was: Missing link in documentation for processor nodes on Kafka Streams) > Broken link in documentation for processor nodes on Kafka Streams > - > > Key: KAFKA-6790 > URL: https://issues.apache.org/jira/browse/KAFKA-6790 > Project: Kafka > Issue Type: Bug >Reporter: robin m >Priority: Major > > The page here > [https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html] > talks about processor nodes and refers to non existing links. > Broken link (appears twice in the same document): > [https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor] > To find this search for the word "processor node" on the page > memory-management , the ones which are links are broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-6790) Missing documentation for processor nodes on Kafka Streams
[ https://issues.apache.org/jira/browse/KAFKA-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438598#comment-16438598 ] ASF GitHub Bot commented on KAFKA-6790: --- ro7m opened a new pull request #4874: KAFKA-6790 URL: https://github.com/apache/kafka/pull/4874 The page here https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html talks about processor nodes and refers to non existing links. Broken link (appears twice in the same document): https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor To find this search for the word "processor node" on the page memory-management , the ones which are links are broken. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Missing documentation for processor nodes on Kafka Streams > -- > > Key: KAFKA-6790 > URL: https://issues.apache.org/jira/browse/KAFKA-6790 > Project: Kafka > Issue Type: Bug >Reporter: robin m >Priority: Major > > The page here > [https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html] > talks about processor nodes and refers to non existing links. > Broken link (appears twice in the same document): > [https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor] > To find this search for the word "processor node" on the page > memory-management , the ones which are links are broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KAFKA-6790) Missing documentation for processor nodes on Kafka Streams
robin m created KAFKA-6790: -- Summary: Missing documentation for processor nodes on Kafka Streams Key: KAFKA-6790 URL: https://issues.apache.org/jira/browse/KAFKA-6790 Project: Kafka Issue Type: Bug Reporter: robin m The page here [https://kafka.apache.org/11/documentation/streams/developer-guide/memory-mgmt.html] talks about processor nodes and refers to non existing links. Broken link (appears twice in the same document): [https://kafka.apache.org/11/documentation/streams/concepts.html#streams-concepts-processor] To find this search for the word "processor node" on the page memory-management , the ones which are links are broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
