[GitHub] kafka pull request #3348: MINOR: Fix unsafe access to incomplete collection ...

2017-06-14 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3348 MINOR: Fix unsafe access to incomplete collection in RecordAccumulator You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka

Build failed in Jenkins: kafka-trunk-jdk7 #2403

2017-06-14 Thread Apache Jenkins Server
See Changes: [ismael] KAFKA-5275; AdminClient API consistency -- [...truncated 962.88 KB...] kafka.tools.ConsoleConsumerTest > shouldLimitReadsToMaxMessageLimit PASSED

[GitHub] kafka pull request #3347: MINOR: Upgrade to Gradle 4.0

2017-06-14 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3347 MINOR: Upgrade to Gradle 4.0 Among other things, it includes performance and CLI improvements. You can merge this pull request into a Git repository by running: $ git pull

[VOTE] KIP-160: Augment KStream.print() to allow users pass in extra parameters in the printed string

2017-06-14 Thread James Chain
I had start a voting process for KIP-160 https://cwiki.apache.org/confluence/display/KAFKA/KIP-160:+Augment+KStream.print()+to+allow+users+pass+in+extra+parameters+in+the+printed+string

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-06-14 Thread Bill Bejeck
+1 Thanks, Bill On Wed, Jun 14, 2017 at 8:10 PM, Xavier Léauté wrote: > +1 from me > > any stream should be able to initialize the cogroup > > On Wed, Jun 14, 2017 at 3:44 PM Kyle Winkelman > wrote: > > > I will update the kip to have only the

[jira] [Commented] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049874#comment-16049874 ] ASF GitHub Bot commented on KAFKA-5275: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #3339: KAFKA-5275: AdminClient API consistency

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3339 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-06-14 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5275: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[jira] [Commented] (KAFKA-5449) Flaky test TransactionsTest.testReadCommittedConsumerShouldNotSeeUndecidedData

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049859#comment-16049859 ] ASF GitHub Bot commented on KAFKA-5449: --- GitHub user apurvam opened a pull request:

[GitHub] kafka pull request #3346: MINOR: Cleanups for TransactionsTest

2017-06-14 Thread apurvam
GitHub user apurvam opened a pull request: https://github.com/apache/kafka/pull/3346 MINOR: Cleanups for TransactionsTest The motivation is that KAFKA-5449 seems to indicate that producer instances can be shared across tests, and that producers from one test seem to be hitting

[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98

2017-06-14 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/kafka-site/pull/60#discussion_r122096399 --- Diff: 0110/design.html --- @@ -264,21 +264,22 @@ messages have a primary key and so the updates are idempotent (receiving the same message

[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98

2017-06-14 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/kafka-site/pull/60#discussion_r122096525 --- Diff: 0110/design.html --- @@ -264,21 +264,22 @@ messages have a primary key and so the updates are idempotent (receiving the same message

[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98

2017-06-14 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/kafka-site/pull/60#discussion_r122096221 --- Diff: 0110/design.html --- @@ -264,21 +264,22 @@ messages have a primary key and so the updates are idempotent (receiving the same message

[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98

2017-06-14 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/kafka-site/pull/60#discussion_r122096607 --- Diff: 0110/design.html --- @@ -264,21 +264,22 @@ messages have a primary key and so the updates are idempotent (receiving the same message

[GitHub] kafka pull request #3345: MINOR: Add Processing Guarantees in Streams docs

2017-06-14 Thread guozhangwang
GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/3345 MINOR: Add Processing Guarantees in Streams docs You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

2017-06-14 Thread Matthias J. Sax
Including KAFKA-3907 sounds reasonable to me. -Matthias On 6/14/17 2:29 PM, Guozhang Wang wrote: > LGTM. Thanks! > > > Guozhang > > On Tue, Jun 13, 2017 at 2:20 PM, Jeyhun Karimov > wrote: > >> Thanks for the comment Matthias. After all the discussion (thanks to all >>

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-06-14 Thread Xavier Léauté
+1 from me any stream should be able to initialize the cogroup On Wed, Jun 14, 2017 at 3:44 PM Kyle Winkelman wrote: > I will update the kip to have only the aggregator in the first cogroup call > and the initializer and serde in the aggregate calls. > > This seems to

[jira] [Commented] (KAFKA-4829) Improve logging of StreamTask commits

2017-06-14 Thread Jeyhun Karimov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049834#comment-16049834 ] Jeyhun Karimov commented on KAFKA-4829: --- [~guozhang] I would suggest similar model to SQL Server's

[jira] [Commented] (KAFKA-5450) Scripts to startup Connect in system tests have too short a timeout

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049803#comment-16049803 ] ASF GitHub Bot commented on KAFKA-5450: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #3344: KAFKA-5450 Increased timeout of Connect system tes...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3344 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-5450) Scripts to startup Connect in system tests have too short a timeout

2017-06-14 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-5450: - Resolution: Fixed Fix Version/s: 0.11.0.0 Status: Resolved

[jira] [Commented] (KAFKA-5449) Flaky test TransactionsTest.testReadCommittedConsumerShouldNotSeeUndecidedData

2017-06-14 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049800#comment-16049800 ] Apurva Mehta commented on KAFKA-5449: - With the latest patch, it appears that the producer which

[GitHub] kafka pull request #3340: MINOR: Add random aborts to system test transactio...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3340 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-5444) Producer.send() will hang 8+ hours

2017-06-14 Thread huxihx (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049787#comment-16049787 ] huxihx commented on KAFKA-5444: --- Is it possible to try using the new producer? And why did you create

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-06-14 Thread Kyle Winkelman
I will update the kip to have only the aggregator in the first cogroup call and the initializer and serde in the aggregate calls. This seems to be the consensus on the email chain. Thanks, Kyle On Jun 14, 2017 5:41 PM, wrote: That is not the case. No matter which stream the record comes in on

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-06-14 Thread Kyle Winkelman
That is not the case. No matter which stream the record comes in on the initializer is called if there is not currently an object in the store. On Jun 14, 2017 5:12 PM, "Guozhang Wang" wrote: > While regarding where we should ask users to set serdes: I think I'm > convinced

[GitHub] kafka pull request #3342: KAFKA-5448: Change TimestampConverter configuratio...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3342 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-5448) TimestampConverter's "type" config conflicts with the basic Transformation "type" config

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049750#comment-16049750 ] ASF GitHub Bot commented on KAFKA-5448: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-5448) TimestampConverter's "type" config conflicts with the basic Transformation "type" config

2017-06-14 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-5448. - Resolution: Fixed Fix Version/s: 0.11.1.0 Issue resolved by pull request 3342

[jira] [Assigned] (KAFKA-3907) Better support for user-specific committing in the Streams DSL

2017-06-14 Thread Jeyhun Karimov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeyhun Karimov reassigned KAFKA-3907: - Assignee: Jeyhun Karimov > Better support for user-specific committing in the Streams

[jira] [Updated] (KAFKA-5452) Aggressive log compaction ratio appears to have no negative effect on log-compacted topics

2017-06-14 Thread Jeff Chao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Chao updated KAFKA-5452: - Description: Some of our users are seeing unintuitive/unexpected behavior with log-compacted topics

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-06-14 Thread Guozhang Wang
While regarding where we should ask users to set serdes: I think I'm convinced by Xavier that they should be in the `aggregate` call (but only those does not pass in a state store supplier) instead of the `KStream#cogroup` call to be consistent with other aggregate functions. BTW another

[jira] [Created] (KAFKA-5452) Aggressive log compaction ratio appears to have no negative effect on log-compacted topics

2017-06-14 Thread Jeff Chao (JIRA)
Jeff Chao created KAFKA-5452: Summary: Aggressive log compaction ratio appears to have no negative effect on log-compacted topics Key: KAFKA-5452 URL: https://issues.apache.org/jira/browse/KAFKA-5452

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-06-14 Thread Kyle Winkelman
To clarify it isn't required to have the initializer in the first cogroup because the first aggregator will have the value type. I like how the initializer makes it abundantly clear that the final type will be that. Right now I'm split because the case may be made that you want to supply a

[GitHub] kafka pull request #3341: MINOR: Update `TransactionMarkerChannelManager` me...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3341 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-06-14 Thread Matthias J. Sax
I have no strong opinion, but it seems that at least InitializerWithKey with be helpful if you want to have different start values for different keys (even if I cannot come up with a use case example why one wanted to do this...). Otherwise, there is just the "completeness" argument, that is not

[jira] [Commented] (KAFKA-5451) Kafka Connect should scan classpath asynchronously

2017-06-14 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049712#comment-16049712 ] Randall Hauch commented on KAFKA-5451: -- KAFKA-5450 demonstrates a case where shorter Kafka Connect

[jira] [Created] (KAFKA-5451) Kafka Connect should scan classpath asynchronously

2017-06-14 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-5451: Summary: Kafka Connect should scan classpath asynchronously Key: KAFKA-5451 URL: https://issues.apache.org/jira/browse/KAFKA-5451 Project: Kafka Issue Type:

[jira] [Comment Edited] (KAFKA-5450) Scripts to startup Connect in system tests have too short a timeout

2017-06-14 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049699#comment-16049699 ] Randall Hauch edited comment on KAFKA-5450 at 6/14/17 9:46 PM: --- Added the

[jira] [Updated] (KAFKA-5450) Scripts to startup Connect in system tests have too short a timeout

2017-06-14 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-5450: - Status: Patch Available (was: Open) Added the patch that changes the timeout in the Connect

[jira] [Commented] (KAFKA-5450) Scripts to startup Connect in system tests have too short a timeout

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049697#comment-16049697 ] ASF GitHub Bot commented on KAFKA-5450: --- GitHub user rhauch opened a pull request:

[GitHub] kafka pull request #3344: KAFKA-5450 Increased timeout of Connect system tes...

2017-06-14 Thread rhauch
GitHub user rhauch opened a pull request: https://github.com/apache/kafka/pull/3344 KAFKA-5450 Increased timeout of Connect system test utilities Increased the timeout from 30sec to 60sec. When running the system tests with packaged Kafka, Connect workers can take about 30seconds

Re: [DISCUSS] KIP-167: Add a restoreAll method to StateRestoreCallback

2017-06-14 Thread Bill Bejeck
Guozhang, Thanks for the comments. 1. As for the granularity, I agree that having one global StateRestoreListener could be restrictive. But I think it's important to have a "setStateRestoreListener" on KafkaStreams as this allows users to define an anonymous instance that has access to local

[jira] [Created] (KAFKA-5450) Scripts to startup Connect in system tests have too short a timeout

2017-06-14 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-5450: Summary: Scripts to startup Connect in system tests have too short a timeout Key: KAFKA-5450 URL: https://issues.apache.org/jira/browse/KAFKA-5450 Project: Kafka

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

2017-06-14 Thread Guozhang Wang
LGTM. Thanks! Guozhang On Tue, Jun 13, 2017 at 2:20 PM, Jeyhun Karimov wrote: > Thanks for the comment Matthias. After all the discussion (thanks to all > participants), I think this (single method that passes in a RecordContext > object) is the best alternative. > Just

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-06-14 Thread Guozhang Wang
I'd suggest we do not block this KIP until the serde work has been sorted out: we cannot estimate yet how long it will take yet. Instead let's say make an agreement on where we want to specify the serdes: whether on the first co-group call or on the aggregate call. Also about the initializer

[GitHub] kafka pull request #3343: WIP: KAFKA-5449: fix bad state transition in trans...

2017-06-14 Thread apurvam
GitHub user apurvam opened a pull request: https://github.com/apache/kafka/pull/3343 WIP: KAFKA-5449: fix bad state transition in transaction manager The `kafka.api.TransactionsTest.testReadCommittedConsumerShouldNotSeeUndecidedData` very rarely sees the following. I

[jira] [Commented] (KAFKA-5449) Flaky test TransactionsTest.testReadCommittedConsumerShouldNotSeeUndecidedData

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049670#comment-16049670 ] ASF GitHub Bot commented on KAFKA-5449: --- GitHub user apurvam opened a pull request:

[jira] [Commented] (KAFKA-5448) TimestampConverter's "type" config conflicts with the basic Transformation "type" config

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049658#comment-16049658 ] ASF GitHub Bot commented on KAFKA-5448: --- GitHub user ewencp opened a pull request:

[GitHub] kafka pull request #3342: KAFKA-5448: Change TimestampConverter configuratio...

2017-06-14 Thread ewencp
GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3342 KAFKA-5448: Change TimestampConverter configuration name to avoid conflicting with reserved 'type' configuration used by all Transformations You can merge this pull request into a Git repository

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-06-14 Thread Guozhang Wang
I'm not particularly concerning that we should NEVER break compatibility; in fact if we think that is worthwhile (i.e. small impact with large benefit) I think we can break compatibility as long as we have not removed the compatibility annotations from Streams. All I was saying is that if we

[GitHub] kafka-site issue #60: Update delivery semantics section for KIP-98

2017-06-14 Thread guozhangwang
Github user guozhangwang commented on the issue: https://github.com/apache/kafka-site/pull/60 lgtm. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] kafka pull request #3341: MINOR: Update `TransactionMarkerChannelManager` me...

2017-06-14 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3341 MINOR: Update `TransactionMarkerChannelManager` metric name Also remove broker-id tags as we generally use them to provide additional context. Finally, do a few clean-ups (could not

[jira] [Updated] (KAFKA-5446) Annoying braces showed on log.error using streams

2017-06-14 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-5446: - Affects Version/s: 0.11.0.0 > Annoying braces showed on log.error using streams >

[jira] [Assigned] (KAFKA-5446) Annoying braces showed on log.error using streams

2017-06-14 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-5446: Assignee: Paolo Patierno > Annoying braces showed on log.error using streams >

[GitHub] kafka-site issue #60: Update delivery semantics section for KIP-98

2017-06-14 Thread hachikuji
Github user hachikuji commented on the issue: https://github.com/apache/kafka-site/pull/60 @guozhangwang @ijuma @apurvam Pushed a couple changes based on the review. Please take another look. --- If your project is set up for it, you can reply to this email and have your reply

Re: [DISCUSS] KIP-167: Add a restoreAll method to StateRestoreCallback

2017-06-14 Thread Guozhang Wang
Thanks Bill for the updated wiki. I have a couple of more comments: 1. Setting StateRestoreListener on the KafkaStreams granularity may not be sufficient, as in the listener callback we do not which store it is restoring right now: if the topic is a changelog topic then from the `TopicPartition`

[jira] [Commented] (KAFKA-5430) new consumers getting data for revoked partitions

2017-06-14 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049582#comment-16049582 ] Vahid Hashemian commented on KAFKA-5430: I could simulate a network failure and get the warning

[jira] [Commented] (KAFKA-5440) Kafka Streams report state RUNNING even if all threads are dead

2017-06-14 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049574#comment-16049574 ] Guozhang Wang commented on KAFKA-5440: -- This is a good find, and maybe it is related to some

Re: [DISCUSS] KIP-167: Add a restoreAll method to StateRestoreCallback

2017-06-14 Thread Bill Bejeck
Eno, Thanks for the comments. 1. As for having both restore and restoreAll, I kept the restore method for backward compatibility as that is what is used by current implementing classes. However as I think about it makes things cleaner to have a single restore method taking a collection. I'll

Build failed in Jenkins: kafka-0.10.2-jdk7 #172

2017-06-14 Thread Apache Jenkins Server
See Changes: [wangguoz] HOFIX: Introduce max wait time for retry-and-backoff while creating -- [...truncated 1.31 MB...]

[jira] [Assigned] (KAFKA-5449) Flaky test TransactionsTest.testReadCommittedConsumerShouldNotSeeUndecidedData

2017-06-14 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apurva Mehta reassigned KAFKA-5449: --- Assignee: Apurva Mehta > Flaky test

[jira] [Created] (KAFKA-5449) Flaky test TransactionsTest.testReadCommittedConsumerShouldNotSeeUndecidedData

2017-06-14 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created KAFKA-5449: -- Summary: Flaky test TransactionsTest.testReadCommittedConsumerShouldNotSeeUndecidedData Key: KAFKA-5449 URL: https://issues.apache.org/jira/browse/KAFKA-5449

[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes

2017-06-14 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049528#comment-16049528 ] Matthias J. Sax commented on KAFKA-5245: I just did this :) > KStream builder should capture

[jira] [Updated] (KAFKA-5245) KStream builder should capture serdes

2017-06-14 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5245: --- Labels: needs-kip (was: beginner newbie) > KStream builder should capture serdes >

[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes

2017-06-14 Thread Evgeny Veretennikov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049523#comment-16049523 ] Evgeny Veretennikov commented on KAFKA-5245: [~guozhang], thanks for explanation! I guess, we

[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes

2017-06-14 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049515#comment-16049515 ] Guozhang Wang commented on KAFKA-5245: -- Anyways [~evis] I have added you to the contributor list,

[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes

2017-06-14 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049511#comment-16049511 ] Guozhang Wang commented on KAFKA-5245: -- Here is my take on the serde issues: We currently follow

[jira] [Commented] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049506#comment-16049506 ] ASF GitHub Bot commented on KAFKA-3123: --- Github user soumyajit-sahu closed the pull request at:

[GitHub] kafka pull request #1716: KAFKA-3123: Make abortAndPauseCleaning() a no-op w...

2017-06-14 Thread soumyajit-sahu
Github user soumyajit-sahu closed the pull request at: https://github.com/apache/kafka/pull/1716 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (KAFKA-5448) TimestampConverter's "type" config conflicts with the basic Transformation "type" config

2017-06-14 Thread David Tucker (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049471#comment-16049471 ] David Tucker commented on KAFKA-5448: - I'd suggest something like "target.type" or "output.type", to

[jira] [Commented] (KAFKA-5413) Log cleaner fails due to large offset in segment file

2017-06-14 Thread Kelvin Rutt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049440#comment-16049440 ] Kelvin Rutt commented on KAFKA-5413: [~junrao], no problem. It might be a couple of days before I can

[jira] [Created] (KAFKA-5448) TimestampConverter's "type" config conflicts with the basic Transformation "type" config

2017-06-14 Thread Ewen Cheslack-Postava (JIRA)
Ewen Cheslack-Postava created KAFKA-5448: Summary: TimestampConverter's "type" config conflicts with the basic Transformation "type" config Key: KAFKA-5448 URL:

[jira] [Commented] (KAFKA-5445) Document exceptions thrown by AdminClient methods

2017-06-14 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049406#comment-16049406 ] Ismael Juma commented on KAFKA-5445: Sounds good. > Document exceptions thrown by AdminClient methods

[GitHub] kafka pull request #3340: MINOR: Add random aborts to system test transactio...

2017-06-14 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3340 MINOR: Add random aborts to system test transactional copier service You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka

[jira] [Commented] (KAFKA-5443) Consumer should use last offset from batch to set next fetch offset

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049371#comment-16049371 ] ASF GitHub Bot commented on KAFKA-5443: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #3331: KAFKA-5443: Consumer should use last offset from b...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3331 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-5443) Consumer should use last offset from batch to set next fetch offset

2017-06-14 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5443: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[GitHub] kafka pull request #3312: HOFIX: Introduce max wait time for retry-and-backo...

2017-06-14 Thread mjsax
Github user mjsax closed the pull request at: https://github.com/apache/kafka/pull/3312 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-5398) Joins on GlobalKTable don't work properly when combined with Avro and the Confluent Schema Registry

2017-06-14 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049358#comment-16049358 ] Matthias J. Sax commented on KAFKA-5398: I guess, that is expected behavior. I would claim, that's

[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes

2017-06-14 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049353#comment-16049353 ] Matthias J. Sax commented on KAFKA-5245: That's a valid point you raise... Not sure about a KIP --

[jira] [Commented] (KAFKA-5445) Document exceptions thrown by AdminClient methods

2017-06-14 Thread Colin P. McCabe (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049351#comment-16049351 ] Colin P. McCabe commented on KAFKA-5445: We should also add tests that these exceptions are

[jira] [Commented] (KAFKA-5441) Fix transaction marker grouping by producerId in TC

2017-06-14 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049349#comment-16049349 ] Guozhang Wang commented on KAFKA-5441: -- I see it is not a blocker for 0.11.0.0, but I can still take

[GitHub] kafka pull request #3327: HOTFIX: Introduce max wait time for retry-and-back...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3327 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-5361) Add EOS integration tests for Streams API

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049338#comment-16049338 ] ASF GitHub Bot commented on KAFKA-5361: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #3276: KAFKA-5361: Add more integration tests for Streams...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3276 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-5274) Review and improve AdminClient Javadoc for the first release (KIP-117)

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049323#comment-16049323 ] ASF GitHub Bot commented on KAFKA-5274: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #3316: KAFKA-5274: AdminClient Javadoc improvements

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3316 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-5274) Review and improve AdminClient Javadoc for the first release (KIP-117)

2017-06-14 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-5274: Resolution: Fixed Fix Version/s: 0.11.1.0 Status: Resolved (was: Patch Available)

Re: [DISCUSS] KIP-167: Add a restoreAll method to StateRestoreCallback

2017-06-14 Thread Eno Thereska
Thanks Bill, A couple of questions: 1. why do we need both restore and restoreAll, why can't we just have one, that takes a collection (i.e., restore all)? Are there cases when people want to restore one at a time? In that case, they could still use restoreAll with just 1 record in the

[jira] [Updated] (KAFKA-5443) Consumer should use last offset from batch to set next fetch offset

2017-06-14 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5443: --- Status: Patch Available (was: Open) > Consumer should use last offset from batch to set next fetch

[jira] [Updated] (KAFKA-5272) Improve validation for Describe/Alter Configs (KIP-133)

2017-06-14 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5272: --- Fix Version/s: (was: 0.11.0.0) 0.11.0.1 > Improve validation for

[jira] [Updated] (KAFKA-5276) Support derived and prefixed configs in DescribeConfigs (KIP-133)

2017-06-14 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5276: --- Fix Version/s: (was: 0.11.0.0) 0.11.0.1 > Support derived and prefixed configs

[jira] [Updated] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-06-14 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5275: --- Status: Patch Available (was: Open) > Review and potentially tweak AdminClient API for the initial

[jira] [Commented] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049301#comment-16049301 ] ASF GitHub Bot commented on KAFKA-5275: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request #3339: KAFKA-5275: AdminClient API consistency

2017-06-14 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3339 KAFKA-5275: AdminClient API consistency You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka kafka-5275-admin-client-api-consistency

[jira] [Commented] (KAFKA-5354) MirrorMaker not preserving headers

2017-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049277#comment-16049277 ] ASF GitHub Bot commented on KAFKA-5354: --- Github user michaelandrepearce closed the pull request at:

[GitHub] kafka pull request #3323: KAFKA-5354: MirrorMaker not preserving headers

2017-06-14 Thread michaelandrepearce
Github user michaelandrepearce closed the pull request at: https://github.com/apache/kafka/pull/3323 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (KAFKA-2111) Command Line Standardization - Add Help Arguments & List Required Fields

2017-06-14 Thread Tom Bentley (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049265#comment-16049265 ] Tom Bentley commented on KAFKA-2111: I'm happy work work on this if [~johnma] is no longer looking at

[GitHub] kafka-site pull request #61: Removed hyphens:auto.

2017-06-14 Thread JNKHunter
GitHub user JNKHunter opened a pull request: https://github.com/apache/kafka-site/pull/61 Removed hyphens:auto. This rule creates misleading setting names on the configuration tables. For example with hyphens: auto we get ``` advertised.lis-teners ``` when the

  1   2   >