[jira] [Created] (KAFKA-4218) Enable access to key in {{ValueTransformer}}

2016-09-24 Thread Elias Levy (JIRA)
Elias Levy created KAFKA-4218: - Summary: Enable access to key in {{ValueTransformer}} Key: KAFKA-4218 URL: https://issues.apache.org/jira/browse/KAFKA-4218 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-4217) KStream.transform equivalent of flatMap

2016-09-24 Thread Elias Levy (JIRA)
Elias Levy created KAFKA-4217: - Summary: KStream.transform equivalent of flatMap Key: KAFKA-4217 URL: https://issues.apache.org/jira/browse/KAFKA-4217 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-4058) Failure in org.apache.kafka.streams.integration.ResetIntegrationTest.testReprocessingFromScratchAfterReset

2016-09-24 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15520134#comment-15520134 ] Matthias J. Sax commented on KAFKA-4058: [~guozhang] I did run this test locally like 30 times. It

[jira] [Commented] (KAFKA-4215) Consumers miss messages during partition reassignment

2016-09-24 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15519566#comment-15519566 ] Apurva Mehta commented on KAFKA-4215: - Another observation: the consumer only misses messages when

[jira] [Commented] (KAFKA-4215) Consumers miss messages during partition reassignment

2016-09-24 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15519348#comment-15519348 ] Apurva Mehta commented on KAFKA-4215: - Here is [~junrao]'s theory for what is going on: {quote} For

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

2016-09-24 Thread Jun Rao
Hi, Radi, For 10, yes, we don't want the buffer pool to wake up the selector every time some memory is freed up. We only want to do that when there is pending requests to the buffer pool not honored due to not enough memory. For 11, we probably want to be a bit careful with Weak References. In

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

2016-09-24 Thread radai
Hi Jun, 10 - I'll add this functionality to the mute/unmute branch. as every mute/unmute operation is O(#connections / #selectorThreads) maybe a watermark approach is better than waking when _any_ mem is available? 11 - "gc notifications" are done by using a ReferenceQueue (

[jira] [Updated] (KAFKA-4216) Replication Throttling: Leader may not be throttled if it is not "moving"

2016-09-24 Thread Ben Stopford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Stopford updated KAFKA-4216: Description: The current algorithm in kafka-reassign-partitions applies a throttle to all moving

[jira] [Created] (KAFKA-4216) Replication Throttling: Leader may not be throttled if it is not "moving"

2016-09-24 Thread Ben Stopford (JIRA)
Ben Stopford created KAFKA-4216: --- Summary: Replication Throttling: Leader may not be throttled if it is not "moving" Key: KAFKA-4216 URL: https://issues.apache.org/jira/browse/KAFKA-4216 Project: Kafka

Jenkins build is back to normal : kafka-trunk-jdk8 #910

2016-09-24 Thread Apache Jenkins Server
See

Build failed in Jenkins: kafka-trunk-jdk7 #1567

2016-09-24 Thread Apache Jenkins Server
See Changes: [ismael] KAFKA-4151; Update public docs for Cluster Id (KIP-78) -- [...truncated 3609 lines...] kafka.network.SocketServerTest > tooBigRequestIsRejected STARTED

[jira] [Commented] (KAFKA-4151) Update public docs for KIP-78

2016-09-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15518747#comment-15518747 ] ASF GitHub Bot commented on KAFKA-4151: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #1895: [DOCS] KAFKA-4151; Update public docs for KIP-78

2016-09-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1895 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-4151) Update public docs for KIP-78

2016-09-24 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4151. Resolution: Fixed Issue resolved by pull request 1895 [https://github.com/apache/kafka/pull/1895]

it this a bug? - message disorder in async send mode -- 0.9.0 java client sdk InFlightRequests

2016-09-24 Thread ????????
We know that in the async send mode, kafka do not guarantee the message order even for the same partition. That is, if we send 3 request ( the same topic, the same partition) to a kafka server in the async mode, the send order is 1, 2, 3 (correlation id is 1, 2, 3), while the kafka server