Error from SyncGroup: The request timed out

[Jhon Davis]

Recently we are experiencing "Error from SyncGroup: The request timed out"
frequently with the Java Kafka APIs.

This issue usually happens with few topic or consumer group in
Kafka cluster. Any pointers on this one will be helpful.

As a workaround, if I change the consumer group name I don't see the error.

Exception in thread "main" org.apache.kafka.common.KafkaException:
Unexpected error from SyncGroup: The request timed out.

at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupRequestHandler.handle(
AbstractCoordinator.java:444)

at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupRequestHandler.handle(
AbstractCoordinator.java:411)

at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(
AbstractCoordinator.java:665)

at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(
AbstractCoordinator.java:644)

at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(
RequestFuture.java:167)

at org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(
RequestFuture.java:133)

at org.apache.kafka.clients.consumer.internals.RequestFuture.complete(
RequestFuture.java:107)

at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(
ConsumerNetworkClient.java:380)

at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:274)


Best,

jd

[jira] [Commented] (KAFKA-3782) Transient failure with kafkatest.tests.connect.connect_distributed_test.ConnectDistributedTest.test_bounce.clean=True

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512218#comment-15512218
 ] 

ASF GitHub Bot commented on KAFKA-3782:
---

GitHub user hachikuji opened a pull request:

https://github.com/apache/kafka/pull/1898

KAFKA-3782: Ensure heartbeat thread restarted after rebalance woken up



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hachikuji/kafka KAFKA-3782

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1898.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1898


commit c06ea5de20efb4ab1d3e9a709614a2be795afe73
Author: Jason Gustafson 
Date:   2016-09-22T04:59:35Z

KAFKA-3782: Ensure heartbeat thread restarted after rebalance woken up




> Transient failure with 
> kafkatest.tests.connect.connect_distributed_test.ConnectDistributedTest.test_bounce.clean=True
> -
>
> Key: KAFKA-3782
> URL: https://issues.apache.org/jira/browse/KAFKA-3782
> Project: Kafka
>  Issue Type: Bug
>  Components: KafkaConnect
>Affects Versions: 0.10.0.0
>Reporter: Liquan Pei
>Assignee: Jason Gustafson
>Priority: Minor
> Fix For: 0.10.1.0, 0.10.0.1
>
>
> For commit 946ae60
> max() arg is an empty sequence
> Traceback (most recent call last):
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka-0.10.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
>  line 106, in run_all_tests
> data = self.run_single_test()
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka-0.10.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
>  line 162, in run_single_test
> return self.current_test_context.function(self.current_test)
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka-0.10.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/mark/_mark.py",
>  line 331, in wrapper
> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka-0.10.0/kafka/tests/kafkatest/tests/connect/connect_distributed_test.py",
>  line 321, in test_bounce
> sink_seqno_max = max(sink_seqnos)
> ValueError: max() arg is an empty sequence



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request #1898: KAFKA-3782: Ensure heartbeat thread restarted afte...

[hachikuji]

GitHub user hachikuji opened a pull request:

https://github.com/apache/kafka/pull/1898

KAFKA-3782: Ensure heartbeat thread restarted after rebalance woken up



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hachikuji/kafka KAFKA-3782

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1898.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1898


commit c06ea5de20efb4ab1d3e9a709614a2be795afe73
Author: Jason Gustafson 
Date:   2016-09-22T04:59:35Z

KAFKA-3782: Ensure heartbeat thread restarted after rebalance woken up




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Reopened] (KAFKA-3782) Transient failure with kafkatest.tests.connect.connect_distributed_test.ConnectDistributedTest.test_bounce.clean=True

[Jason Gustafson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gustafson reopened KAFKA-3782:


Reopening this. Not quite the same failure as before: 
http://testing.confluent.io/confluent-kafka-system-test-results/?prefix=2016-09-21--001.1474482458--apache--trunk--4f82183/.
 

After some investigation, it looks like the consumer is not handling wakeups 
during the rebalance correctly. In particular, if the user calls wakeup before 
a rebalance has finished, then the heartbeat thread won't be re-enabled, which 
will eventually cause the consumer's session timeout to expire.

> Transient failure with 
> kafkatest.tests.connect.connect_distributed_test.ConnectDistributedTest.test_bounce.clean=True
> -
>
> Key: KAFKA-3782
> URL: https://issues.apache.org/jira/browse/KAFKA-3782
> Project: Kafka
>  Issue Type: Bug
>  Components: KafkaConnect
>Affects Versions: 0.10.0.0
>Reporter: Liquan Pei
>Assignee: Jason Gustafson
>Priority: Minor
> Fix For: 0.10.1.0, 0.10.0.1
>
>
> For commit 946ae60
> max() arg is an empty sequence
> Traceback (most recent call last):
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka-0.10.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
>  line 106, in run_all_tests
> data = self.run_single_test()
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka-0.10.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
>  line 162, in run_single_test
> return self.current_test_context.function(self.current_test)
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka-0.10.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/mark/_mark.py",
>  line 331, in wrapper
> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka-0.10.0/kafka/tests/kafkatest/tests/connect/connect_distributed_test.py",
>  line 321, in test_bounce
> sink_seqno_max = max(sink_seqnos)
> ValueError: max() arg is an empty sequence



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2657) Kafka clients fail to start if one of broker isn't resolved by DNS

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15511657#comment-15511657
 ] 

ASF GitHub Bot commented on KAFKA-2657:
---

Github user apakulov closed the pull request at:

https://github.com/apache/kafka/pull/336


> Kafka clients fail to start if one of broker isn't resolved by DNS 
> ---
>
> Key: KAFKA-2657
> URL: https://issues.apache.org/jira/browse/KAFKA-2657
> Project: Kafka
>  Issue Type: Bug
>Reporter: Alexander Pakulov
>Priority: Minor
>
> During org.apache.kafka.clients.producer.KafkaProducer and 
> org.apache.kafka.clients.consumer.KafkaConsumer object creation constructors 
> invoke org.apache.kafka.common.utils.ClientUtils#parseAndValidateAddresses
> which potentially could throw an exception if one the nodes hasn't been 
> resolved by DNS. As a result of that - object hasn't been created and you 
> aren't able to use Kafka clients.
> I personally think that Kafka should be able to operate with cluster with 
> quorum number of instances.
> {code:java}
> try {
> InetSocketAddress address = new InetSocketAddress(host, port);
> if (address.isUnresolved())
> throw new ConfigException("DNS resolution failed for url in " + 
> ProducerConfig.BOOTSTRAP_SERVERS_CONFIG + ": " + url);
> addresses.add(address);
> } catch (NumberFormatException e) {
> throw new ConfigException("Invalid port in " + 
> ProducerConfig.BOOTSTRAP_SERVERS_CONFIG + ": " + url);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request #336: KAFKA-2657; Kafka clients fail to start if one of b...

[apakulov]

Github user apakulov closed the pull request at:

https://github.com/apache/kafka/pull/336


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-4178) Replication Throttling: Consolidate Rate Classes

[Joel Koshy (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15511604#comment-15511604
 ] 

Joel Koshy commented on KAFKA-4178:
---

[~junrao] sorry I don't remember, but you might :) It appears the change was a 
follow-up to one of your [review comments on 
KAFKA-2084|https://reviews.apache.org/r/33049/#comment150934] Let me know if 
you need help with revisiting that discussion - it has been a while since I 
have looked at that code.

> Replication Throttling: Consolidate Rate Classes
> 
>
> Key: KAFKA-4178
> URL: https://issues.apache.org/jira/browse/KAFKA-4178
> Project: Kafka
>  Issue Type: Improvement
>  Components: replication
>Affects Versions: 0.10.1.0
>Reporter: Ben Stopford
>
> Replication throttling is using a different implementation of Rate to client 
> throttling (Rate & SimpleRate). These should be consolidated so both use the 
> same approach. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-4204) KafkaService.verify_reassign_partitions is a no-op

[Apurva Mehta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-4204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apurva Mehta updated KAFKA-4204:

Assignee: Apurva Mehta

> KafkaService.verify_reassign_partitions is a no-op 
> ---
>
> Key: KAFKA-4204
> URL: https://issues.apache.org/jira/browse/KAFKA-4204
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Affects Versions: 0.10.0.1
>Reporter: Apurva Mehta
>Assignee: Apurva Mehta
>
> In the 'verify_reassign_partitions' method of the KafkaService class in the 
> system tests, we execute the kafka-reassign-partitions command and then do a 
> regular expression match on the tools output to verify that the reassignment 
> succeeded. 
> In particular, we search for the pattern 'is in progress' in the output 
> string. If the pattern exists, it means that the reassignment is still in 
> progress. 
> As it stands, this mechanism is broken because the tool outputs 'is still in 
> progress' for each reassignment which hasn't completed. Further, the tool 
> outputs a multi-line string, but the regex does not factor this in. 
> In general, depending on a specific pattern on stdout to determine success or 
> failure of an operation like reassignment is very fragile. 
> The right thing to do would be for the tool to output a well defined data 
> structure, which can be accurately interpreted by the test or any other 
> program which needs that information. 
> This JIRA is going to track the discussion and progress for implementing the 
> latter. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-4204) KafkaService.verify_reassign_partitions is a no-op

[Apurva Mehta (JIRA)]

Apurva Mehta created KAFKA-4204:
---

 Summary: KafkaService.verify_reassign_partitions is a no-op 
 Key: KAFKA-4204
 URL: https://issues.apache.org/jira/browse/KAFKA-4204
 Project: Kafka
  Issue Type: Bug
  Components: core
Affects Versions: 0.10.0.1
Reporter: Apurva Mehta


In the 'verify_reassign_partitions' method of the KafkaService class in the 
system tests, we execute the kafka-reassign-partitions command and then do a 
regular expression match on the tools output to verify that the reassignment 
succeeded. 

In particular, we search for the pattern 'is in progress' in the output string. 
If the pattern exists, it means that the reassignment is still in progress. 

As it stands, this mechanism is broken because the tool outputs 'is still in 
progress' for each reassignment which hasn't completed. Further, the tool 
outputs a multi-line string, but the regex does not factor this in. 

In general, depending on a specific pattern on stdout to determine success or 
failure of an operation like reassignment is very fragile. 

The right thing to do would be for the tool to output a well defined data 
structure, which can be accurately interpreted by the test or any other program 
which needs that information. 

This JIRA is going to track the discussion and progress for implementing the 
latter. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: kafka-0.10.1-jdk7 #3

[Apache Jenkins Server]

See 

Changes:

[wangguoz] MINOR: add javadoc comment to PersistenKeyValueFactory.enableCaching

[wangguoz] MINOR: remove unused code from InternalTopicManager

--
[...truncated 12031 lines...]
:271:
 constructor ListOffsetRequest in class ListOffsetRequest is deprecated: see 
corresponding Javadoc for more information.
(new ListOffsetRequest(consumerId, partitions.asJava), 0)
 ^
:281:
 value offsets in class PartitionData is deprecated: see corresponding Javadoc 
for more information.
  partitionData.offsets.get(0)
^
:298:
 method fromReplica in object FetchRequest is deprecated: see corresponding 
Javadoc for more information.
  else JFetchRequest.fromReplica(replicaId, maxWait, minBytes, requestMap)
 ^
:43:
 class OldProducer in package producer is deprecated: This class has been 
deprecated and will be removed in a future release. Please use 
org.apache.kafka.clients.producer.KafkaProducer instead.
new OldProducer(getOldProducerProps(config))
^
:45:
 class NewShinyProducer in package producer is deprecated: This class has been 
deprecated and will be removed in a future release. Please use 
org.apache.kafka.clients.producer.KafkaProducer instead.
new NewShinyProducer(getNewProducerProps(config))
^
24 warnings found
:core:processResources UP-TO-DATE
:core:classes
:core:copyDependantLibs
:core:jar
:examples:compileJava
:examples:processResources UP-TO-DATE
:examples:classes
:examples:checkstyleMain
:examples:compileTestJava UP-TO-DATE
:examples:processTestResources UP-TO-DATE
:examples:testClasses UP-TO-DATE
:examples:checkstyleTest UP-TO-DATE
:examples:test UP-TO-DATE
:log4j-appender:compileJava
:log4j-appender:processResources UP-TO-DATE
:log4j-appender:classes
:log4j-appender:checkstyleMain
:log4j-appender:compileTestJava
:log4j-appender:processTestResources UP-TO-DATE
:log4j-appender:testClasses
:log4j-appender:checkstyleTest
:log4j-appender:test

org.apache.kafka.log4jappender.KafkaLog4jAppenderTest > testLog4jAppends STARTED

org.apache.kafka.log4jappender.KafkaLog4jAppenderTest > testLog4jAppends PASSED

org.apache.kafka.log4jappender.KafkaLog4jAppenderTest > testKafkaLog4jConfigs 
STARTED

org.apache.kafka.log4jappender.KafkaLog4jAppenderTest > testKafkaLog4jConfigs 
PASSED
:core:compileTestJava UP-TO-DATE
:core:compileTestScala
:186:
 constructor ListOffsetRequest in class ListOffsetRequest is deprecated: see 
corresponding Javadoc for more information.
new requests.ListOffsetRequest(Map(tp -> new 
ListOffsetRequest.PartitionData(0, 100)).asJava)
^
:186:
 class PartitionData in object ListOffsetRequest is deprecated: see 
corresponding Javadoc for more information.
new requests.ListOffsetRequest(Map(tp -> new 
ListOffsetRequest.PartitionData(0, 100)).asJava)
   ^
:88:
 method createAndShutdownStep in class MetricsTest is deprecated: This test has 
been deprecated and it will be removed in a future release
createAndShutdownStep("group0", "consumer0", "producer0")
^
:158:
 constructor FetchRequest in class FetchRequest is deprecated: see 
corresponding Javadoc for more information.
val fetchRequest = new FetchRequest(Int.MaxValue, 0, 
createPartitionMap(maxPartitionBytes, Seq(topicPartition)))
   ^
four warnings found
:core:processTestResources UP-TO-DATE
:core:testClasses
:connect:api:compileJava
:connect:api:processResources UP-TO-DATE
:connect:api:classes
:connect:api:copyDependantLibs
:connect:api:jar
:connect:json:compileJava
Download 
https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.3/jackson-databind-2.6.3.pom
Download 
https://repo1.maven.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.1/jackson-parent-2.6.1.pom
Download 

Jenkins build is back to normal : kafka-trunk-jdk8 #904

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-4203) Java producer default max message size does not align with broker default

[Grant Henke (JIRA)]

Grant Henke created KAFKA-4203:
--

 Summary: Java producer default max message size does not align 
with broker default
 Key: KAFKA-4203
 URL: https://issues.apache.org/jira/browse/KAFKA-4203
 Project: Kafka
  Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Grant Henke
Assignee: Grant Henke
Priority: Critical


The Java producer sets max.request.size = 1048576 (the base 2 version of 1 MB 
(MiB))

The broker sets max.message.bytes = 112 (the base 10 value of 1 MB + 12 
bytes for overhead)

This means that by default the producer can try to produce messages larger than 
the broker will accept resulting in RecordTooLargeExceptions.

There were not similar issues in the old producer because it sets 
max.message.size = 100 (the base 10 value of 1 MB)

I propose we increase the broker default for max.message.bytes to 1048588 (the 
base 2 value of 1 MB (MiB) + 12 bytes for overhead) so that any message 
produced with default configs from either producer does not result in a 
RecordTooLargeException.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: kafka-trunk-jdk8 #903

[Apache Jenkins Server]

See 

Changes:

[wangguoz] MINOR: add javadoc comment to PersistenKeyValueFactory.enableCaching

--
[...truncated 13485 lines...]
org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullPredicateOnFilterNot PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldHaveAtLeastOnPredicateWhenBranching STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldHaveAtLeastOnPredicateWhenBranching PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullFilePathOnWriteAsText STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullFilePathOnWriteAsText PASSED

org.apache.kafka.streams.kstream.internals.KeyValuePrinterProcessorTest > 
testPrintKeyValueWithProvidedSerde STARTED

org.apache.kafka.streams.kstream.internals.KeyValuePrinterProcessorTest > 
testPrintKeyValueWithProvidedSerde PASSED

org.apache.kafka.streams.kstream.internals.KeyValuePrinterProcessorTest > 
testPrintKeyValuesWithName STARTED

org.apache.kafka.streams.kstream.internals.KeyValuePrinterProcessorTest > 
testPrintKeyValuesWithName PASSED

org.apache.kafka.streams.kstream.internals.KeyValuePrinterProcessorTest > 
testPrintKeyValueDefaultSerde STARTED

org.apache.kafka.streams.kstream.internals.KeyValuePrinterProcessorTest > 
testPrintKeyValueDefaultSerde PASSED

org.apache.kafka.streams.kstream.internals.KStreamKTableLeftJoinTest > testJoin 
STARTED

org.apache.kafka.streams.kstream.internals.KStreamKTableLeftJoinTest > testJoin 
PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > testJoin 
STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > testJoin 
PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testNotSendingOldValues STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testNotSendingOldValues PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testSendingOldValues STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testSendingOldValues PASSED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > 
testSendingOldValue STARTED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > 
testSendingOldValue PASSED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > 
testNotSendingOldValue STARTED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > 
testNotSendingOldValue PASSED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > 
testSkipNullOnMaterialization STARTED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > 
testSkipNullOnMaterialization PASSED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > testKTable STARTED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > testKTable PASSED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > testValueGetter 
STARTED

org.apache.kafka.streams.kstream.internals.KTableFilterTest > testValueGetter 
PASSED

org.apache.kafka.streams.kstream.internals.KStreamFlatMapTest > testFlatMap 
STARTED

org.apache.kafka.streams.kstream.internals.KStreamFlatMapTest > testFlatMap 
PASSED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > testAggBasic 
STARTED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > testAggBasic 
PASSED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > testCount 
STARTED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > testCount 
PASSED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > 
testAggCoalesced STARTED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > 
testAggCoalesced PASSED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > 
testAggRepartition STARTED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > 
testAggRepartition PASSED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > 
testRemoveOldBeforeAddNew STARTED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > 
testRemoveOldBeforeAddNew PASSED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > 
testCountCoalesced STARTED

org.apache.kafka.streams.kstream.internals.KTableAggregateTest > 
testCountCoalesced PASSED

org.apache.kafka.streams.kstream.internals.KStreamFlatMapValuesTest > 
testFlatMapValues STARTED

org.apache.kafka.streams.kstream.internals.KStreamFlatMapValuesTest > 
testFlatMapValues PASSED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldNotAllowNullSubtractorOnAggregate STARTED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldNotAllowNullSubtractorOnAggregate PASSED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldNotAllowNullInitializerOnAggregate STARTED

[jira] [Updated] (KAFKA-4194) Add more tests for KIP-79

[Jiangjie Qin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jiangjie Qin updated KAFKA-4194:

Status: Patch Available  (was: Open)

> Add more tests for KIP-79
> -
>
> Key: KAFKA-4194
> URL: https://issues.apache.org/jira/browse/KAFKA-4194
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Jiangjie Qin
>Assignee: Jiangjie Qin
>
> This is a follow up ticket to add more tests for KIP-79. Including the 
> integration tests and clients test if necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request #1888: MINOR: remove unused code from InternalTopicManage...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/1888


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] kafka pull request #1891: MINOR: add javadoc comment to PersistenKeyValueFac...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/1891


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-4194) Add more tests for KIP-79

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510654#comment-15510654
 ] 

ASF GitHub Bot commented on KAFKA-4194:
---

GitHub user becketqin reopened a pull request:

https://github.com/apache/kafka/pull/1897

KAFKA-4194 follow up patch for KIP-79



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/becketqin/kafka KAFKA-4194

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1897.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1897


commit 5556276d339486c055ee101397d745e7f86d09e0
Author: Jiangjie Qin 
Date:   2016-09-21T17:40:37Z

KAFKA-4148 follow up patch for KIP-79




> Add more tests for KIP-79
> -
>
> Key: KAFKA-4194
> URL: https://issues.apache.org/jira/browse/KAFKA-4194
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Jiangjie Qin
>Assignee: Jiangjie Qin
>
> This is a follow up ticket to add more tests for KIP-79. Including the 
> integration tests and clients test if necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4194) Add more tests for KIP-79

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510653#comment-15510653
 ] 

ASF GitHub Bot commented on KAFKA-4194:
---

Github user becketqin closed the pull request at:

https://github.com/apache/kafka/pull/1897


> Add more tests for KIP-79
> -
>
> Key: KAFKA-4194
> URL: https://issues.apache.org/jira/browse/KAFKA-4194
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Jiangjie Qin
>Assignee: Jiangjie Qin
>
> This is a follow up ticket to add more tests for KIP-79. Including the 
> integration tests and clients test if necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request #1897: KAFKA-4194 follow up patch for KIP-79

[becketqin]

Github user becketqin closed the pull request at:

https://github.com/apache/kafka/pull/1897


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] kafka pull request #1897: KAFKA-4194 follow up patch for KIP-79

[becketqin]

GitHub user becketqin reopened a pull request:

https://github.com/apache/kafka/pull/1897

KAFKA-4194 follow up patch for KIP-79



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/becketqin/kafka KAFKA-4194

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1897.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1897


commit 5556276d339486c055ee101397d745e7f86d09e0
Author: Jiangjie Qin 
Date:   2016-09-21T17:40:37Z

KAFKA-4148 follow up patch for KIP-79




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-4148) KIP-79 add ListOffsetRequest v1 and search by timestamp interface to consumer.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510642#comment-15510642
 ] 

ASF GitHub Bot commented on KAFKA-4148:
---

GitHub user becketqin opened a pull request:

https://github.com/apache/kafka/pull/1897

KAFKA-4148 follow up patch for KIP-79



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/becketqin/kafka KAFKA-4194

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1897.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1897


commit 5556276d339486c055ee101397d745e7f86d09e0
Author: Jiangjie Qin 
Date:   2016-09-21T17:40:37Z

KAFKA-4148 follow up patch for KIP-79




> KIP-79 add ListOffsetRequest v1 and search by timestamp interface to consumer.
> --
>
> Key: KAFKA-4148
> URL: https://issues.apache.org/jira/browse/KAFKA-4148
> Project: Kafka
>  Issue Type: Task
>Reporter: Jiangjie Qin
>Assignee: Jiangjie Qin
> Fix For: 0.10.1.0
>
>
> This ticket is to implement KIP-79.
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65868090



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request #1897: KAFKA-4148 follow up patch for KIP-79

[becketqin]

GitHub user becketqin opened a pull request:

https://github.com/apache/kafka/pull/1897

KAFKA-4148 follow up patch for KIP-79



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/becketqin/kafka KAFKA-4194

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1897.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1897


commit 5556276d339486c055ee101397d745e7f86d09e0
Author: Jiangjie Qin 
Date:   2016-09-21T17:40:37Z

KAFKA-4148 follow up patch for KIP-79




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-4081) Consumer API consumer new interface commitSyn does not verify the validity of offset

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510415#comment-15510415
 ] 

ASF GitHub Bot commented on KAFKA-4081:
---

GitHub user mimaison reopened a pull request:

https://github.com/apache/kafka/pull/1827

KAFKA-4081: Consumer API consumer new interface commitSync does not v…

…erify the validity of offset

Commit throws InvalidOffsetException if the offset is negative

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mimaison/kafka KAFKA-4081

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1827.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1827


commit 7ec08416868180422dd6df8e921c9f6027218bfa
Author: Mickael Maison 
Date:   2016-09-19T17:22:34Z

KAFKA-4081: Consumer API consumer new interface commitSync does not verify 
the validity of offset

Commit will now fail with an IllegalArgumentException when trying to commit 
an offset that is either negative or larger than the consumer's current position

commit e1c1674b8f661e4063e4d5bc04a86ad069a3f317
Author: Edoardo Comar 
Date:   2016-09-21T16:14:36Z

KAFKA-4081: Consumer API consumer new interface commitSync does not
verify the validity of offset

Commit will now fail with an IllegalArgumentException when trying to
commit an offset that is either negative or larger than the consumer's
current position




> Consumer API consumer new interface commitSyn does not verify the validity of 
> offset
> 
>
> Key: KAFKA-4081
> URL: https://issues.apache.org/jira/browse/KAFKA-4081
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 0.9.0.1
>Reporter: lifeng
>Assignee: Mickael Maison
>
> Consumer API consumer new interface commitSyn synchronization update offset, 
> for the illegal offset successful return, illegal offset<0 or offset>hw



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request #1827: KAFKA-4081: Consumer API consumer new interface co...

[mimaison]

GitHub user mimaison reopened a pull request:

https://github.com/apache/kafka/pull/1827

KAFKA-4081: Consumer API consumer new interface commitSync does not v…

…erify the validity of offset

Commit throws InvalidOffsetException if the offset is negative

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mimaison/kafka KAFKA-4081

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1827.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1827


commit 7ec08416868180422dd6df8e921c9f6027218bfa
Author: Mickael Maison 
Date:   2016-09-19T17:22:34Z

KAFKA-4081: Consumer API consumer new interface commitSync does not verify 
the validity of offset

Commit will now fail with an IllegalArgumentException when trying to commit 
an offset that is either negative or larger than the consumer's current position

commit e1c1674b8f661e4063e4d5bc04a86ad069a3f317
Author: Edoardo Comar 
Date:   2016-09-21T16:14:36Z

KAFKA-4081: Consumer API consumer new interface commitSync does not
verify the validity of offset

Commit will now fail with an IllegalArgumentException when trying to
commit an offset that is either negative or larger than the consumer's
current position




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-3283) Remove beta from new consumer documentation

[Nick Kleinschmidt (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510310#comment-15510310
 ] 

Nick Kleinschmidt commented on KAFKA-3283:
--

I didn't see anything in the documentation about how to move from the old 
consumer to the new consumer. Follow the steps from the FAQ to move offsets to 
Kafka, then I can swap new consumer for old consumer?

> Remove beta from new consumer documentation
> ---
>
> Key: KAFKA-3283
> URL: https://issues.apache.org/jira/browse/KAFKA-3283
> Project: Kafka
>  Issue Type: Improvement
>  Components: consumer
>Reporter: Ismael Juma
>Assignee: Ismael Juma
> Fix For: 0.10.1.0
>
>
> Ideally, we would:
> * Remove the beta label
> * Filling any critical gaps in functionality
> * Update the documentation on the old consumers to recommend the new consumer 
> (without deprecating the old consumer, however)
> Current target is 0.10.1.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-4200) Minor issue with throttle argument in kafka-reassign-partitions.sh

[Ben Stopford (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-4200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ben Stopford updated KAFKA-4200:

Status: Patch Available  (was: In Progress)

> Minor issue with throttle argument in kafka-reassign-partitions.sh
> --
>
> Key: KAFKA-4200
> URL: https://issues.apache.org/jira/browse/KAFKA-4200
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.10.1.0
>Reporter: Ben Stopford
>Assignee: Ben Stopford
>
> 1.
> kafka-reassign-partitions —-verify prints Throttle was removed regardless of 
> whether a throttle was applied. It should only print this if the value was 
> actually changed. 
> 2.
> —verify should exception if the —throttle argument. (check generate too)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (KAFKA-4200) Minor issue with throttle argument in kafka-reassign-partitions.sh

[Ben Stopford (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-4200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on KAFKA-4200 started by Ben Stopford.
---
> Minor issue with throttle argument in kafka-reassign-partitions.sh
> --
>
> Key: KAFKA-4200
> URL: https://issues.apache.org/jira/browse/KAFKA-4200
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.10.1.0
>Reporter: Ben Stopford
>Assignee: Ben Stopford
>
> 1.
> kafka-reassign-partitions —-verify prints Throttle was removed regardless of 
> whether a throttle was applied. It should only print this if the value was 
> actually changed. 
> 2.
> —verify should exception if the —throttle argument. (check generate too)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4200) Minor issue with throttle argument in kafka-reassign-partitions.sh

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509782#comment-15509782
 ] 

ASF GitHub Bot commented on KAFKA-4200:
---

GitHub user benstopford opened a pull request:

https://github.com/apache/kafka/pull/1896

KAFKA-4200: Fix throttle argument in kafka-reassign-partitions.sh

Simple jira which alters two things: 

1. kafka-reassign-partitions --verify prints Throttle was removed 
regardless of whether a throttle was applied. It should only print this if the 
value was actually changed.

2. --verify should exception if the —throttle argument. (check generate too)

To test this I extracted all validation logic into a separate method and 
added a test which covers the majority of combinations. The validation logic 
was retained as is, other than implementing (2) and adding validation to the 
--broker-list option which you can currently apply to any of hte main actions 
(where it is ignored). Requirement 1 was tested manually (as it's just 
println). 

Testing:
- Build passes locally. 
- System test reassign_partitions_test.py also passes.
 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/benstopford/kafka KAFKA-4200

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1896.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1896


commit f85efead9c468d5d6e906a5c11f573a36d620244
Author: Ben Stopford 
Date:   2016-09-21T09:26:50Z

KAFKA-4200: kafka-reassign-partitions —-verify prints "Throttle was 
removed" regardless of whether a throttle was applied. It should only print 
this if the value was actually changed. Tested this maually ensuring the value 
is not printed when we call verify whilst the rebalance is running, but does 
print the value when running verify when the throttle completes. Rerunning 
verify after the throttle completes also does not print the line.

commit 843c00c3cbee354fcfb1f0fdda5f77addd4b6eb4
Author: Ben Stopford 
Date:   2016-09-21T12:15:37Z

KAFKA-4200: kafka-reassign-partitions —-verify prints "Throttle was 
removed" regardless of whether a throttle was applied. It should only print 
this if the value was actually changed. Tested this maually ensuring the value 
is not printed when we call verify whilst the rebalance is running, but does 
print the value when running verify when the throttle completes. Rerunning 
verify after the throttle completes also does not print the line.




> Minor issue with throttle argument in kafka-reassign-partitions.sh
> --
>
> Key: KAFKA-4200
> URL: https://issues.apache.org/jira/browse/KAFKA-4200
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.10.1.0
>Reporter: Ben Stopford
>Assignee: Ben Stopford
>
> 1.
> kafka-reassign-partitions —-verify prints Throttle was removed regardless of 
> whether a throttle was applied. It should only print this if the value was 
> actually changed. 
> 2.
> —verify should exception if the —throttle argument. (check generate too)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request #1896: KAFKA-4200: Fix throttle argument in kafka-reassig...

[benstopford]

GitHub user benstopford opened a pull request:

https://github.com/apache/kafka/pull/1896

KAFKA-4200: Fix throttle argument in kafka-reassign-partitions.sh

Simple jira which alters two things: 

1. kafka-reassign-partitions --verify prints Throttle was removed 
regardless of whether a throttle was applied. It should only print this if the 
value was actually changed.

2. --verify should exception if the —throttle argument. (check generate 
too)

To test this I extracted all validation logic into a separate method and 
added a test which covers the majority of combinations. The validation logic 
was retained as is, other than implementing (2) and adding validation to the 
--broker-list option which you can currently apply to any of hte main actions 
(where it is ignored). Requirement 1 was tested manually (as it's just 
println). 

Testing:
- Build passes locally. 
- System test reassign_partitions_test.py also passes.
 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/benstopford/kafka KAFKA-4200

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1896.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1896


commit f85efead9c468d5d6e906a5c11f573a36d620244
Author: Ben Stopford 
Date:   2016-09-21T09:26:50Z

KAFKA-4200: kafka-reassign-partitions —-verify prints "Throttle was 
removed" regardless of whether a throttle was applied. It should only print 
this if the value was actually changed. Tested this maually ensuring the value 
is not printed when we call verify whilst the rebalance is running, but does 
print the value when running verify when the throttle completes. Rerunning 
verify after the throttle completes also does not print the line.

commit 843c00c3cbee354fcfb1f0fdda5f77addd4b6eb4
Author: Ben Stopford 
Date:   2016-09-21T12:15:37Z

KAFKA-4200: kafka-reassign-partitions —-verify prints "Throttle was 
removed" regardless of whether a throttle was applied. It should only print 
this if the value was actually changed. Tested this maually ensuring the value 
is not printed when we call verify whilst the rebalance is running, but does 
print the value when running verify when the throttle completes. Rerunning 
verify after the throttle completes also does not print the line.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (KAFKA-4202) Facing error while trying to create the Producer.

[Rohan (JIRA)]

Rohan created KAFKA-4202:


 Summary: Facing error while trying to create the Producer.
 Key: KAFKA-4202
 URL: https://issues.apache.org/jira/browse/KAFKA-4202
 Project: Kafka
  Issue Type: Bug
  Components: KafkaConnect
Reporter: Rohan
Assignee: Ewen Cheslack-Postava


While trying to run the command 
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic first-topic
I am facing the below error.

ERROR StatusLogger No log4j2 configuration file found. Using default 
configuration: logging only errors to the console.
Exception in thread "main" java.lang.NoSuchMethodError: 
kafka.utils.CommandLineUtils$.parseKeyValueArgs(Lscala/collection/Iterable;)Ljava/util/Properties;
at 
kafka.tools.ConsoleProducer$ProducerConfig.(ConsoleProducer.scala:279)
at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:38)
at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4189) Consumer poll hangs forever if kafka is disabled

[Umesh Chaudhary (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509565#comment-15509565
 ] 

Umesh Chaudhary commented on KAFKA-4189:


Yep, [Guozhang's 
comment|https://issues.apache.org/jira/browse/KAFKA-1894?focusedCommentId=15036256=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15036256]
 suggests the same 

> Consumer poll hangs forever if kafka is disabled
> 
>
> Key: KAFKA-4189
> URL: https://issues.apache.org/jira/browse/KAFKA-4189
> Project: Kafka
>  Issue Type: Bug
>  Components: clients, consumer
>Affects Versions: 0.9.0.1, 0.10.0.1
>Reporter: Tomas Benc
>Priority: Critical
>
> We develop web application, where client sends REST request and our 
> application downloads messages from Kafka and sends those messages back to 
> client. In our web application we use "New Consumer API" (not High Level nor 
> Simple Consumer API).
> Problem occurs in case of disabling Kafka and web application is running on. 
> Application receives request and tries to poll messages from Kafka. 
> Processing is on that line blocked until Kafka is enabled.
> ConsumerRecords records = consumer.poll(1000);
> Timeout parameter of the poll method has no influence in such case. I expect 
> poll method could throw some Exception describing about connection issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-2392) Kafka Server does not accept 0 as a port

[JIRA]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Buğra Gedik updated KAFKA-2392:
---
Description: 
I want to specify 0 as a port number for Zookeeper as well as Kafka Server. For 
instance server.properties configuration file has a 'port' property, but does 
not accept 0 as a value. Similarly, zookeeper.properties has a 'clientPort' 
property, but does not accept 0 as a value.

I want 0 to specify that the port will be selected automatically (OS assigned). 
In my use case, I want to run Zookeeper with an automatically picked port, and 
use that port to create a Kafka Server configuration file, that specifies the 
Kafka Server port as 0 as well. I parse the output from the servers to figure 
out the actual ports used. All this is needed for a testing environment.

Not supporting automatically selected ports makes it difficult to run Kafka 
server as part of our tests.




  was:
I want to specify 0 as a port number for Zookeeper as well as Kafka Server. For 
instance server.properties configuration file has a 'port' property, but does 
not accept 0 as a value. Similarly, zookeeper.properties has a 'clientPort' 
property, but does not accept 0 as a value.

I want 0 to specify that the port will be selected automatically (OS assigned). 
In my use case, I want to run Zookeeper with a automatically picked port, and 
use that port to create a Kafka Server configuration file, that specifies the 
Kafka Server port as 0 as well. I parse the output from the servers to figure 
out the actual ports used. All this is needed for a testing environment.

Not supporting automatically selected ports makes it difficult to run Kafka 
server as part of our tests.





> Kafka Server does not accept 0 as a port
> 
>
> Key: KAFKA-2392
> URL: https://issues.apache.org/jira/browse/KAFKA-2392
> Project: Kafka
>  Issue Type: Bug
>  Components: config
>Affects Versions: 0.8.2.1
>Reporter: Buğra Gedik
>Priority: Minor
>
> I want to specify 0 as a port number for Zookeeper as well as Kafka Server. 
> For instance server.properties configuration file has a 'port' property, but 
> does not accept 0 as a value. Similarly, zookeeper.properties has a 
> 'clientPort' property, but does not accept 0 as a value.
> I want 0 to specify that the port will be selected automatically (OS 
> assigned). In my use case, I want to run Zookeeper with an automatically 
> picked port, and use that port to create a Kafka Server configuration file, 
> that specifies the Kafka Server port as 0 as well. I parse the output from 
> the servers to figure out the actual ports used. All this is needed for a 
> testing environment.
> Not supporting automatically selected ports makes it difficult to run Kafka 
> server as part of our tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-2392) Kafka Server does not accept 0 as a port

[JIRA]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Buğra Gedik updated KAFKA-2392:
---
Description: 
I want to specify 0 as a port number for Zookeeper as well as Kafka Server. For 
instance server.properties configuration file has a 'port' property, but does 
not accept 0 as a value. Similarly, zookeeper.properties has a 'clientPort' 
property, but does not accept 0 as a value.

I want 0 to specify that the port will be selected automatically (OS assigned). 
In my use case, I want to run Zookeeper with a automatically picked port, and 
use that port to create a Kafka Server configuration file, that specifies the 
Kafka Server port as 0 as well. I parse the output from the servers to figure 
out the actual ports used. All this is needed for a testing environment.

Not supporting automatically selected ports makes it difficult to run Kafka 
server as part of our tests.




  was:
I want to specify 0 as a port number for Zookeeper as well as Kafka Server. For 
instance server.properties configuration file has a 'port' property, but does 
not accept 0 as a value. Similarly, zookeeper.properties has a 'clientPort' 
property, but does not acce[t 0 as a value.

I want 0 to specify that the port will be selected automatically (OS assigned). 
In my use case, I want to run Zookeeper with a automatically picked port, and 
use that port to create a Kafka Server configuration file, that specifies the 
Kafka Server port as 0 as well. I parse the output from the servers to figure 
out the actual ports used. All this is needed for a testing environment.

Not supporting automatically selected ports makes it difficult to run Kafka 
server as part of our tests.





> Kafka Server does not accept 0 as a port
> 
>
> Key: KAFKA-2392
> URL: https://issues.apache.org/jira/browse/KAFKA-2392
> Project: Kafka
>  Issue Type: Bug
>  Components: config
>Affects Versions: 0.8.2.1
>Reporter: Buğra Gedik
>Priority: Minor
>
> I want to specify 0 as a port number for Zookeeper as well as Kafka Server. 
> For instance server.properties configuration file has a 'port' property, but 
> does not accept 0 as a value. Similarly, zookeeper.properties has a 
> 'clientPort' property, but does not accept 0 as a value.
> I want 0 to specify that the port will be selected automatically (OS 
> assigned). In my use case, I want to run Zookeeper with a automatically 
> picked port, and use that port to create a Kafka Server configuration file, 
> that specifies the Kafka Server port as 0 as well. I parse the output from 
> the servers to figure out the actual ports used. All this is needed for a 
> testing environment.
> Not supporting automatically selected ports makes it difficult to run Kafka 
> server as part of our tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4189) Consumer poll hangs forever if kafka is disabled

[Tomas Benc (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509047#comment-15509047
 ] 

Tomas Benc commented on KAFKA-4189:
---

Yes, we're stuck on code, which Edoardo Comar described in comments. But we 
will try to use wakeup() method, hope this helps.

> Consumer poll hangs forever if kafka is disabled
> 
>
> Key: KAFKA-4189
> URL: https://issues.apache.org/jira/browse/KAFKA-4189
> Project: Kafka
>  Issue Type: Bug
>  Components: clients, consumer
>Affects Versions: 0.9.0.1, 0.10.0.1
>Reporter: Tomas Benc
>Priority: Critical
>
> We develop web application, where client sends REST request and our 
> application downloads messages from Kafka and sends those messages back to 
> client. In our web application we use "New Consumer API" (not High Level nor 
> Simple Consumer API).
> Problem occurs in case of disabling Kafka and web application is running on. 
> Application receives request and tries to poll messages from Kafka. 
> Processing is on that line blocked until Kafka is enabled.
> ConsumerRecords records = consumer.poll(1000);
> Timeout parameter of the poll method has no influence in such case. I expect 
> poll method could throw some Exception describing about connection issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4151) Update public docs for KIP-78

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509015#comment-15509015
 ] 

ASF GitHub Bot commented on KAFKA-4151:
---

GitHub user arrawatia opened a pull request:

https://github.com/apache/kafka/pull/1895

KAFKA-4151; Update public docs for KIP-78

- Updated implementation docs with details on Cluster Id generation and ZK 
data structure.
- Mention cluster id in "noteworthy changes for 0.10.1.0" in upgrade docs.
- Add cluster id to metrics table in ops docs.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/arrawatia/kafka kip-78-docs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1895.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1895


commit 17b0962845f08c6a33ca78d7efcc2d5339c3d359
Author: arrawatia 
Date:   2016-09-07T06:52:14Z

Merge remote-tracking branch 'confluentinc/trunk' into trunk

commit ecbceebdaca60a952d02e5f232d9f6507b325eff
Author: Sumit Arrawatia 
Date:   2016-09-19T18:52:38Z

Merge branch 'trunk' of github.com:apache/kafka into trunk

commit d595da09fd511d2f33cedc3f4a36ca79e31f1195
Author: Sumit Arrawatia 
Date:   2016-09-21T07:06:08Z

Add docs for cluster id.

commit a4c2aae082db45bad90bac6e9cf2e67981cf97d3
Author: Sumit Arrawatia 
Date:   2016-09-21T07:08:03Z

Merge changes from trunk.




> Update public docs for KIP-78
> -
>
> Key: KAFKA-4151
> URL: https://issues.apache.org/jira/browse/KAFKA-4151
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Sumit Arrawatia
>Assignee: Sumit Arrawatia
> Fix For: 0.10.1.0
>
>
> Add documentation to include details on Cluster Id in "Implementation" 
> section. The actual implementation is tracked in KAFKA-4093. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request #1895: KAFKA-4151; Update public docs for KIP-78

[arrawatia]

GitHub user arrawatia opened a pull request:

https://github.com/apache/kafka/pull/1895

KAFKA-4151; Update public docs for KIP-78

- Updated implementation docs with details on Cluster Id generation and ZK 
data structure.
- Mention cluster id in "noteworthy changes for 0.10.1.0" in upgrade docs.
- Add cluster id to metrics table in ops docs.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/arrawatia/kafka kip-78-docs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1895.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1895


commit 17b0962845f08c6a33ca78d7efcc2d5339c3d359
Author: arrawatia 
Date:   2016-09-07T06:52:14Z

Merge remote-tracking branch 'confluentinc/trunk' into trunk

commit ecbceebdaca60a952d02e5f232d9f6507b325eff
Author: Sumit Arrawatia 
Date:   2016-09-19T18:52:38Z

Merge branch 'trunk' of github.com:apache/kafka into trunk

commit d595da09fd511d2f33cedc3f4a36ca79e31f1195
Author: Sumit Arrawatia 
Date:   2016-09-21T07:06:08Z

Add docs for cluster id.

commit a4c2aae082db45bad90bac6e9cf2e67981cf97d3
Author: Sumit Arrawatia 
Date:   2016-09-21T07:08:03Z

Merge changes from trunk.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---