[jira] [Updated] (KAFKA-3842) Add Helper Functions Into TestUtils

[ https://issues.apache.org/jira/browse/KAFKA-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck updated KAFKA-3842: --- Description: Per guidance from [~guozhang] from PR #1477 move helper functions from

[jira] [Updated] (KAFKA-3842) Add Helper Functions Into TestUtils

[ https://issues.apache.org/jira/browse/KAFKA-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck updated KAFKA-3842: --- Description: Per guidance from [~guozhang] from PR #1477 move helper functions from

[jira] [Created] (KAFKA-3842) Add Helper Functions Into TestUtils

Bill Bejeck created KAFKA-3842: -- Summary: Add Helper Functions Into TestUtils Key: KAFKA-3842 URL: https://issues.apache.org/jira/browse/KAFKA-3842 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-3839) Handling null keys in KTable.groupBy

[ https://issues.apache.org/jira/browse/KAFKA-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331112#comment-15331112 ] Guozhang Wang commented on KAFKA-3839: -- Great! I have assigned the ticket to you. And I also added

[jira] [Updated] (KAFKA-3839) Handling null keys in KTable.groupBy

[ https://issues.apache.org/jira/browse/KAFKA-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3839: - Assignee: Jeyhun Karimov (was: Guozhang Wang) > Handling null keys in KTable.groupBy >

[jira] [Assigned] (KAFKA-3839) Handling null keys in KTable.groupBy

[ https://issues.apache.org/jira/browse/KAFKA-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-3839: Assignee: Guozhang Wang > Handling null keys in KTable.groupBy >

Re: [DISCUSS] KIP-4 Create Topic Schema

I would second Ismael's points on 2 & 4. Sometimes its good to be liberal with the input you accept, but I don't think that applies here -- there are specific rules and if the user gives inconsistent input that indicates, I don't think we should just hope that prioritizing one arbitrarily (the

[jira] [Comment Edited] (KAFKA-3144) report members with no assigned partitions in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274522#comment-15274522 ] Vahid Hashemian edited comment on KAFKA-3144 at 6/15/16 12:26 AM: --

Memory consumption of Kafka-examples Kafka-streams around 1.5 GB

I am running "mvn test" as per tip from https://github.com/confluentinc/examples/tree/master/kafka-streams README.MD. This uses embedded Kafka components from test alone (no ZK, Kafka, schema-registry running). I monitor on OSX El Capitan (10.11.5) memory usage and it grows on Java processes from

Build failed in Jenkins: kafka-trunk-jdk7 #1361

See Changes: [me] KAFKA-3762 Log.loadSegments() should log the message in exception [me] MINOR: Catch Throwable in commitSourceTask() -- [...truncated 6496 lines...]

[jira] [Commented] (KAFKA-3841) MirrorMaker topic renaming

[ https://issues.apache.org/jira/browse/KAFKA-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330877#comment-15330877 ] Ning Zhang commented on KAFKA-3841: --- Thanks. This is good to know. As MessageHandler was recently added

[jira] [Commented] (KAFKA-3839) Handling null keys in KTable.groupBy

[ https://issues.apache.org/jira/browse/KAFKA-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330865#comment-15330865 ] Jeyhun Karimov commented on KAFKA-3839: --- I can take this task as a starter. > Handling null keys in

[jira] [Commented] (KAFKA-3727) Consumer.poll() stuck in loop on non-existent topic manually assigned

[ https://issues.apache.org/jira/browse/KAFKA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330850#comment-15330850 ] Edoardo Comar commented on KAFKA-3727: -- Another case where poll() hangs forever is when the user

[jira] [Comment Edited] (KAFKA-1894) Avoid long or infinite blocking in the consumer

[ https://issues.apache.org/jira/browse/KAFKA-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330848#comment-15330848 ] Edoardo Comar edited comment on KAFKA-1894 at 6/14/16 11:17 PM: Another

[jira] [Commented] (KAFKA-1894) Avoid long or infinite blocking in the consumer

[ https://issues.apache.org/jira/browse/KAFKA-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330848#comment-15330848 ] Edoardo Comar commented on KAFKA-1894: -- Another case that leads to the loop ``` public void

[jira] [Commented] (KAFKA-3841) MirrorMaker topic renaming

[ https://issues.apache.org/jira/browse/KAFKA-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330827#comment-15330827 ] James Cheng commented on KAFKA-3841: We have a similar use case. Mirrormaker has a config option

[jira] [Commented] (KAFKA-3841) MirrorMaker topic renaming

[ https://issues.apache.org/jira/browse/KAFKA-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330828#comment-15330828 ] James Cheng commented on KAFKA-3841: More comments here:

[jira] [Updated] (KAFKA-3841) MirrorMaker topic renaming

[ https://issues.apache.org/jira/browse/KAFKA-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated KAFKA-3841: -- Description: Our organization (walmart.com) has been a Kafka user since some years back and

[jira] [Created] (KAFKA-3841) MirrorMaker topic renaming

Ning Zhang created KAFKA-3841: - Summary: MirrorMaker topic renaming Key: KAFKA-3841 URL: https://issues.apache.org/jira/browse/KAFKA-3841 Project: Kafka Issue Type: New Feature

[GitHub] kafka pull request #1507: KAFKA-3840; Allow clients default OS buffer sizes

GitHub user slaunay opened a pull request: https://github.com/apache/kafka/pull/1507 KAFKA-3840; Allow clients default OS buffer sizes Follow up on KAFKA-724 (#1469) to allow OS socket buffer sizes auto tuning for both the broker and the clients. You can merge this pull request

[jira] [Commented] (KAFKA-3840) OS auto tuning for socket buffer size in clients not allowed through configuration

[ https://issues.apache.org/jira/browse/KAFKA-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330781#comment-15330781 ] ASF GitHub Bot commented on KAFKA-3840: --- GitHub user slaunay opened a pull request:

[jira] [Commented] (KAFKA-3755) tightening the offset check in ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330773#comment-15330773 ] ASF GitHub Bot commented on KAFKA-3755: --- GitHub user imandhan opened a pull request:

[GitHub] kafka pull request #1506: KAFKA-3755 Tightening the offset check in ReplicaF...

GitHub user imandhan opened a pull request: https://github.com/apache/kafka/pull/1506 KAFKA-3755 Tightening the offset check in ReplicaFetcherThread Including an additional check to make sure that the first offset in the message set to be appended to the log is >= than the log end

[jira] [Created] (KAFKA-3840) OS auto tuning for socket buffer size in clients not allowed through configuration

Sébastien Launay created KAFKA-3840: --- Summary: OS auto tuning for socket buffer size in clients not allowed through configuration Key: KAFKA-3840 URL: https://issues.apache.org/jira/browse/KAFKA-3840

[jira] [Updated] (KAFKA-3837) Report the name of the blocking thread when throwing ConcurrentModificationException

[ https://issues.apache.org/jira/browse/KAFKA-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated KAFKA-3837: -- Status: Patch Available (was: Open) > Report the name of the blocking thread when

[jira] [Comment Edited] (KAFKA-3837) Report the name of the blocking thread when throwing ConcurrentModificationException

[ https://issues.apache.org/jira/browse/KAFKA-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330744#comment-15330744 ] Bharat Viswanadham edited comment on KAFKA-3837 at 6/14/16 10:18 PM: -

[jira] [Commented] (KAFKA-3837) Report the name of the blocking thread when throwing ConcurrentModificationException

[ https://issues.apache.org/jira/browse/KAFKA-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330751#comment-15330751 ] ASF GitHub Bot commented on KAFKA-3837: --- GitHub user bharatviswa504 opened a pull request:

[GitHub] kafka pull request #1505: KAFKA-3837: Report the thread id of the blocking t...

GitHub user bharatviswa504 opened a pull request: https://github.com/apache/kafka/pull/1505 KAFKA-3837: Report the thread id of the blocking thread when throwing ConcurrentModificationException You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (KAFKA-3837) Report the name of the blocking thread when throwing ConcurrentModificationException

[ https://issues.apache.org/jira/browse/KAFKA-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330744#comment-15330744 ] Bharat Viswanadham commented on KAFKA-3837: --- Hi Simon, In the kafka consumer already we have

[jira] [Created] (KAFKA-3839) Handling null keys in KTable.groupBy

Guozhang Wang created KAFKA-3839: Summary: Handling null keys in KTable.groupBy Key: KAFKA-3839 URL: https://issues.apache.org/jira/browse/KAFKA-3839 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-3839) Handling null keys in KTable.groupBy

[ https://issues.apache.org/jira/browse/KAFKA-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3839: - Labels: architecture newbie (was: architecture) > Handling null keys in KTable.groupBy >

Re: [DISCUSS] KIP-4 Create Topic Schema

Thanks for the review Ismael. `partition_count` or `num_partitions` seems clearer to me. Agreed, I updated the wiki and patch to use num_partitions. I wondered if this should be `INT16`. Maybe not worth it as it won't make > much of a difference in terms of the request size though. > Since

[jira] [Commented] (KAFKA-3822) Kafka Consumer close() hangs indefinitely if Kafka Broker shutdown while connected

[ https://issues.apache.org/jira/browse/KAFKA-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330616#comment-15330616 ] Jason Gustafson commented on KAFKA-3822: [~alex-cook4] Thanks for the report. Just to clarify,

[jira] [Comment Edited] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330608#comment-15330608 ] Guozhang Wang edited comment on KAFKA-3775 at 6/14/16 9:13 PM: --- [~kawamuray]

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330608#comment-15330608 ] Guozhang Wang commented on KAFKA-3775: -- @Yuto Kawamura I dumped my thoughts about memory management

[jira] [Assigned] (KAFKA-3837) Report the name of the blocking thread when throwing ConcurrentModificationException

[ https://issues.apache.org/jira/browse/KAFKA-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham reassigned KAFKA-3837: - Assignee: Bharat Viswanadham > Report the name of the blocking thread when

[GitHub] kafka pull request #1402: MINOR: Catch Throwable in commitSourceTask()

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1402 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-4 Create Topic Schema

Hi Grant, Thanks for the proposal. A few comments and questions below. On Fri, Jun 10, 2016 at 6:21 PM, Grant Henke wrote: > > CreateTopic Request (Version: 0) => [create_topic_requests] timeout > > create_topic_requests => topic partitions replication_factor >

Re: [DISCUSS] KIP-4 Create Topic Schema

Hi Guozhang, Thanks for the review. I listed the possible CreateTopic Response error codes in my patch. Below are the relevant error codes and how I think the admin client or producer would handle them. This can be vetted more thoroughly in the code reviews: - INVALID_TOPIC_EXCEPTION(17)

[jira] [Resolved] (KAFKA-3762) Log.loadSegments() should log the message in exception

[ https://issues.apache.org/jira/browse/KAFKA-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3762. -- Resolution: Fixed Fix Version/s: 0.10.1.0 Issue resolved by pull request

[GitHub] kafka pull request #1480: KAFKA-3762 Log.loadSegments() should log the messa...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1480 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3762) Log.loadSegments() should log the message in exception

[ https://issues.apache.org/jira/browse/KAFKA-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330157#comment-15330157 ] ASF GitHub Bot commented on KAFKA-3762: --- Github user asfgit closed the pull request at:

Re: [DISCUSS] KIP-4 Create Topic Schema

Thanks Grant, the design proposal LGTM overall. One minor question about the error codes in CreateTopic Response, what are the possible values? I know this may be out of the scope of this KIP, but would also want to think about how producers should handle each one of them accordingly, especially

[jira] [Commented] (KAFKA-3836) KStreamReduce and KTableReduce should not pass nulls to Deserializers

[ https://issues.apache.org/jira/browse/KAFKA-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330066#comment-15330066 ] Guozhang Wang commented on KAFKA-3836: -- Example places that we should guard against null values: when

[jira] [Commented] (KAFKA-3699) Update protocol page on website to explain how KIP-35 should be used

[ https://issues.apache.org/jira/browse/KAFKA-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329875#comment-15329875 ] ASF GitHub Bot commented on KAFKA-3699: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-3699) Update protocol page on website to explain how KIP-35 should be used

[ https://issues.apache.org/jira/browse/KAFKA-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3699: - Resolution: Fixed Fix Version/s: 0.10.1.0 Status: Resolved

[GitHub] kafka pull request #1395: KAFKA-3699: Update protocol page on website to exp...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1395 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-3838) Bump zkclient and Zookeeper versions

[ https://issues.apache.org/jira/browse/KAFKA-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FIlipe Azevedo updated KAFKA-3838: -- Description: Zookeeper 3.4.8 has some improvements, specifically it handles DNS Re-resolution

[jira] [Commented] (KAFKA-3838) Bump zkclient and Zookeeper versions

[ https://issues.apache.org/jira/browse/KAFKA-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329703#comment-15329703 ] ASF GitHub Bot commented on KAFKA-3838: --- GitHub user mangas opened a pull request:

[GitHub] kafka pull request #1504: KAFKA-3838 zkClient and Zookeeper version bump

GitHub user mangas opened a pull request: https://github.com/apache/kafka/pull/1504 KAFKA-3838 zkClient and Zookeeper version bump You can merge this pull request into a Git repository by running: $ git pull https://github.com/mangas/kafka 0.10.0 Alternatively you can review

[jira] [Created] (KAFKA-3838) Bump zkclient and Zookeeper versions

FIlipe Azevedo created KAFKA-3838: - Summary: Bump zkclient and Zookeeper versions Key: KAFKA-3838 URL: https://issues.apache.org/jira/browse/KAFKA-3838 Project: Kafka Issue Type: Task

[jira] [Commented] (KAFKA-3209) Support single message transforms in Kafka Connect

[ https://issues.apache.org/jira/browse/KAFKA-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329669#comment-15329669 ] Nisarg Shah commented on KAFKA-3209: I'm still working on it. Got a little busy. I thought I'll get a

[jira] [Created] (KAFKA-3837) Report the name of the blocking thread when throwing ConcurrentModificationException

Simon Cooper created KAFKA-3837: --- Summary: Report the name of the blocking thread when throwing ConcurrentModificationException Key: KAFKA-3837 URL: https://issues.apache.org/jira/browse/KAFKA-3837

[jira] [Created] (KAFKA-3836) KStreamReduce and KTableReduce should not pass nulls to Deserializers

Avi Flax created KAFKA-3836: --- Summary: KStreamReduce and KTableReduce should not pass nulls to Deserializers Key: KAFKA-3836 URL: https://issues.apache.org/jira/browse/KAFKA-3836 Project: Kafka

[GitHub] kafka pull request #1503: MINOR: Small enhancement to Deserializer Javadoc

GitHub user aviflax opened a pull request: https://github.com/apache/kafka/pull/1503 MINOR: Small enhancement to Deserializer Javadoc I’ve implemented my own custom Deserializer and been using it with `KStream.reduceByKey`; I observed that `reduceByKey` was passing null to my

[jira] [Updated] (KAFKA-3822) Kafka Consumer close() hangs indefinitely if Kafka Broker shutdown while connected

[ https://issues.apache.org/jira/browse/KAFKA-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Cook updated KAFKA-3822: -- Affects Version/s: 0.10.0.0 > Kafka Consumer close() hangs indefinitely if Kafka Broker

[jira] [Updated] (KAFKA-3835) Streams is creating two ProducerRecords for each send via RecordCollector

[ https://issues.apache.org/jira/browse/KAFKA-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-3835: -- Description: The RecordCollector.send(..) method below, currently receives a ProducerRecord from its

[jira] [Commented] (KAFKA-3737) Closing connection during produce request should be log with WARN level.

[ https://issues.apache.org/jira/browse/KAFKA-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329482#comment-15329482 ] Florian Hussonnois commented on KAFKA-3737: --- Hi Gwen, My concern is more about the

[jira] [Updated] (KAFKA-3835) Streams is creating two ProducerRecords for each send via RecordCollector

[ https://issues.apache.org/jira/browse/KAFKA-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-3835: -- Description: The RecordCollector.send(..) method below, currently receives a ProducerRecord from its

[jira] [Updated] (KAFKA-3835) Streams is creating two ProducerRecords for each send via RecordCollector

[ https://issues.apache.org/jira/browse/KAFKA-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-3835: -- Description: The RecordCollector.send(..) method below, currently receives a ProducerRecord from its

[jira] [Created] (KAFKA-3835) Streams is creating two ProducerRecords for each send via RecordCollector

Damian Guy created KAFKA-3835: - Summary: Streams is creating two ProducerRecords for each send via RecordCollector Key: KAFKA-3835 URL: https://issues.apache.org/jira/browse/KAFKA-3835 Project: Kafka