[jira] [Commented] (KAFKA-4281) Should be able to forward aggregation values immediately

[ https://issues.apache.org/jira/browse/KAFKA-4281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15560765#comment-15560765 ] Greg Fodor commented on KAFKA-4281: --- PR: https://github.com/apache/kafka/pull/1998 If this approach

Re: Store flushing on commit.interval.ms from KIP-63 introduces aggregation latency

JIRA opened here: https://issues.apache.org/jira/browse/KAFKA-4281 On Sun, Oct 9, 2016 at 2:02 AM, Greg Fodor wrote: > I went ahead and did some more testing, and it feels to me one option > for resolving this issue is having a method on KGroupedStream which > can be used to

[jira] [Commented] (KAFKA-4281) Should be able to forward aggregation values immediately

[ https://issues.apache.org/jira/browse/KAFKA-4281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15560764#comment-15560764 ] ASF GitHub Bot commented on KAFKA-4281: --- GitHub user gfodor opened a pull request:

[GitHub] kafka pull request #1998: KAFKA-4281: Should be able to forward aggregation ...

GitHub user gfodor opened a pull request: https://github.com/apache/kafka/pull/1998 KAFKA-4281: Should be able to forward aggregation values immediately https://issues.apache.org/jira/browse/KAFKA-4281 You can merge this pull request into a Git repository by running: $ git

[jira] [Created] (KAFKA-4281) Should be able to forward aggregation values immediately

Greg Fodor created KAFKA-4281: - Summary: Should be able to forward aggregation values immediately Key: KAFKA-4281 URL: https://issues.apache.org/jira/browse/KAFKA-4281 Project: Kafka Issue Type:

Build failed in Jenkins: kafka-trunk-jdk8 #959

See Changes: [jason] MINOR: Update Quickstart in documentation to account for Windows -- [...truncated 5509 lines...] kafka.api.AuthorizerIntegrationTest >

Build failed in Jenkins: kafka-0.10.1-jdk7 #58

See Changes: [jason] MINOR: Update Quickstart in documentation to account for Windows -- [...truncated 6445 lines...] kafka.api.test.ProducerCompressionTest > testCompression[0] STARTED

Build failed in Jenkins: kafka-trunk-jdk7 #1614

See Changes: [jason] MINOR: Update Quickstart in documentation to account for Windows -- [...truncated 14045 lines...] org.apache.kafka.streams.processor.internals.assignment.TaskAssignorTest >

Re: [DISCUSS] KIP-82 - Add Record Headers

I think here we have a situation like SQS in Amazon had, originally not supporting headers but as time and requirements increased the need becomes evident and finally headers or by which ever name it goes by are introduced got added back in 2014. A blog from way back when SQS first added

Re: [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hey David, Thanks for updating the wiki. 1. I was actually thinking of letting every broker just consume the __consumer_offsets topic. But it seems less efficient if there are only a few topics configured for committed offsets based retention. So querying the committed offsets seems reasonable.

[jira] [Commented] (KAFKA-4217) KStream.transform equivalent of flatMap

[ https://issues.apache.org/jira/browse/KAFKA-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15560348#comment-15560348 ] Guozhang Wang commented on KAFKA-4217: -- We designed {{process()}} to be differentiated from

[GitHub] kafka pull request #1990: MINOR: Update Quickstart in documentation to accou...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1990 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Temporary node of broke in zookeeper lost

hi, I meet a issue that the temporary node of broke in zookeeper was lost while the process of the broker still exist. At this time, the controller would consider it to be offline. According to zkClient log, I find the session is timeout, but handleStateChanged and handleNewSession(in

RE?? [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hi Becket, This is david, thanks for the comments. I have update some info in the wiki. All the changes is nearly described in the workflow. Answer for the commnets: 1. Every brokers only have some of the groups' commit offset which are storaged in the __comsumer_offsets topics, it still

Re: Store flushing on commit.interval.ms from KIP-63 introduces aggregation latency

I went ahead and did some more testing, and it feels to me one option for resolving this issue is having a method on KGroupedStream which can be used to configure if the operations on it (reduce/aggregate) will forward immediately or not. I did a quick patch and was able to determine that if the

Store flushing on commit.interval.ms from KIP-63 introduces aggregation latency

I'm taking 0.10.1 for a spin on our existing Kafka Streams jobs and I'm hitting what seems to be a serious issue (at least, for us) with the changes brought about in KIP-63. In our job, we have a number of steps in the topology where we perform a repartition and aggregation on topics that require

Re: [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hey David, Thanks for the KIP. Can you help with the following two questions: 1) If someone start a consumer (e.g. kafka-console-consumer) to consume a topic for debug/validation purpose, a randome consumer group may be created and offset may be committed for this consumer group. If no offset