[GitHub] kafka pull request #3533: KAFKA-5596: TopicCommand methods fail to handle to...

Github user sh-z closed the pull request at: https://github.com/apache/kafka/pull/3533 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #3536: KAFKA-3856: Refactoring for KIP-120

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/3536 KAFKA-3856: Refactoring for KIP-120 - extract InternalTopologyBuilder from TopologyBuilder - deprecate all "leaking" methods from public TopologyBuilder API You can merge this pull request into

[jira] [Created] (KAFKA-5599) ConsumerConsole : --new-consumer option as deprecated

Paolo Patierno created KAFKA-5599: - Summary: ConsumerConsole : --new-consumer option as deprecated Key: KAFKA-5599 URL: https://issues.apache.org/jira/browse/KAFKA-5599 Project: Kafka Issue

[DISCUSS] KIP-176: Remove deprecated new-consumer option in ConsoleConsumer tool

Hi dev, I'd like to discuss the possibility to remove the new-consumer option in the ConsoleConsumer tool. https://cwiki.apache.org/confluence/display/KAFKA/KIP-176%3A+Remove+deprecated+new-consumer+option+in+ConsoleConsumer+tool I have already opened a JIRA with related PR for marking it as

[jira] [Created] (KAFKA-5600) GroupMetadataManager doesn't read Offsets of segmented log correctly

Jan Burkhardt created KAFKA-5600: Summary: GroupMetadataManager doesn't read Offsets of segmented log correctly Key: KAFKA-5600 URL: https://issues.apache.org/jira/browse/KAFKA-5600 Project: Kafka

[VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

Hi, I'd like to kick off the vote for KIP-173: https://cwiki.apache.org/confluence/display/KAFKA/KIP-173%3A+Add+prefix+to+StreamsConfig+to+enable+setting+default+internal+topic+configs A PR for this can be found here: https://github.com/apache/kafka/pull/3459 Thanks, Damian

[GitHub] kafka pull request #3538: KAFKA-5600: fix GroupMetadataManager doesn't read ...

GitHub user bjrke opened a pull request: https://github.com/apache/kafka/pull/3538 KAFKA-5600: fix GroupMetadataManager doesn't read offsets of segmente… …d log correctly the while loop was too big and need to be closed earlier to see the fix, ignore whitespace

[GitHub] kafka pull request #3537: KAFKA-5599: ConsoleConsumer : --new-consumer optio...

GitHub user ppatierno opened a pull request: https://github.com/apache/kafka/pull/3537 KAFKA-5599: ConsoleConsumer : --new-consumer option as deprecated Marking --new-consumer as deprecated in the help of ConsoleConsumer Printing a warning on using the new consumer but adding

Re: [VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

+1 (non-binding) Thanks Eno > On Jul 17, 2017, at 12:43 PM, Damian Guy wrote: > > Hi, > > I'd like to kick off the vote for KIP-173: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-173%3A+Add+prefix+to+StreamsConfig+to+enable+setting+default+internal+topic+configs

Re: [DISCUSS] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

I'm going to leave it as is for now. We can do another KIP later if it is deemed necessary - which i'm not yet convinced it is. On Sun, 16 Jul 2017 at 04:39 Matthias J. Sax wrote: > We can use the `admin.` prefix for the current `StreamsKafkaClient`, > too. As switching

[jira] [Resolved] (KAFKA-5585) Failover in a replicated Cluster does not work

[ https://issues.apache.org/jira/browse/KAFKA-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Bayer resolved KAFKA-5585. - Resolution: Cannot Reproduce After setting up everything from scratch I was not able to reproduce

Jenkins build is back to normal : kafka-trunk-jdk8 #1811

See

[jira] [Resolved] (KAFKA-5596) TopicCommand methods fail to handle topic name with "."

[ https://issues.apache.org/jira/browse/KAFKA-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuai Zhang resolved KAFKA-5596. Resolution: Not A Problem > TopicCommand methods fail to handle topic name with "." >

[GitHub] kafka pull request #3529: MINOR: Use correct connectionId in error message

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3529 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-5588) ConsumerConsole : uselss --new-consumer option

[ https://issues.apache.org/jira/browse/KAFKA-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paolo Patierno resolved KAFKA-5588. --- Resolution: Later Before removing the --new-consumer option, the first step is to mark it as

Re: Behavior when kafka producer is network partitioned

On Sun, Jul 16, 2017 at 8:32 PM, Abhishek Agarwal wrote: > What is the behaviour in 0.10.x if kafka producer is not able to reach any > of the broker for a brief duration? > > From what I understand from the code, all the nodes will be in > disconnected state. Since there

Re: [VOTE] KIP-167: Add interface for the state store restoration process

This has now passed. The votes where: Binding +1: Sriram, Guozhang, Damian Non-binding +1: Matthias, Eno. The discussion will move to the PR now https://github.com/apache/kafka/pull/3325. Thanks, Bill On Sat, Jul 15, 2017 at 11:17 PM, Matthias J. Sax wrote: > +1 > >

Kafka issues during Developement

Hi, I am using kafka as the messaging queue for my application which will run on the windows platform. I am facing issues during the log file deletion with following erros. Also i found some disclaimer is available in the kafka documentation to use kafka in windows platform Caused by:

[GitHub] kafka pull request #3539: KAFKA-5602: ducker-ak: support --custom-ducktape

GitHub user cmccabe opened a pull request: https://github.com/apache/kafka/pull/3539 KAFKA-5602: ducker-ak: support --custom-ducktape Support a --custom-ducktape flag which allows developers to install their own versions of ducktape into Docker images. This is helpful for

[jira] [Created] (KAFKA-5601) Refactor ReassignPartitionsCommand to use AdminClient

Tom Bentley created KAFKA-5601: -- Summary: Refactor ReassignPartitionsCommand to use AdminClient Key: KAFKA-5601 URL: https://issues.apache.org/jira/browse/KAFKA-5601 Project: Kafka Issue Type:

Re: [VOTE] KIP-168: Add TotalTopicCount metric per cluster

Dong, Onur, Mickael, Vahid, and Joel: Thank you all for voting! Best, Abhishek On Thu, Jul 13, 2017 at 8:43 PM, Joel Koshy wrote: > +1 > > On Thu, Jul 13, 2017 at 12:24 PM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > > > +1 (non-binding) > > > > > > > > > >

Re: [VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

+1 On Mon, Jul 17, 2017 at 5:42 AM, Eno Thereska wrote: > +1 (non-binding) > > Thanks > Eno > > On Jul 17, 2017, at 12:43 PM, Damian Guy wrote: > > > > Hi, > > > > I'd like to kick off the vote for KIP-173: > >

Re: [VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

+1 On 7/17/17 9:18 AM, Guozhang Wang wrote: > +1 > > On Mon, Jul 17, 2017 at 5:42 AM, Eno Thereska > wrote: > >> +1 (non-binding) >> >> Thanks >> Eno >>> On Jul 17, 2017, at 12:43 PM, Damian Guy wrote: >>> >>> Hi, >>> >>> I'd like to kick off the

Re: Kafka issues during Developement

This is a know issue. You cannot delete shared (or opened files) on Windows, but UNIX/Linux is okay. Please read Kafka online documentation on WIndows usage. A relevant jira case is KAFKA-1194. On 17 July 2017 at 11:31, Dharma Raj wrote: > Hi, > > I am using kafka as

Re: Dealing with noisy timestamps in kafka streams + 10.1

Hello Greg, Sorry for the long delay in responses, but very glad to see you have gone this far to resolve the problem. All of your solutions make sense to me. I think for the first problem your raised, it is a general problem not only for Streams' sending changelog records but for any clients

[jira] [Created] (KAFKA-5602) Add --custom-ducktape flag to ducker-ak

Colin P. McCabe created KAFKA-5602: -- Summary: Add --custom-ducktape flag to ducker-ak Key: KAFKA-5602 URL: https://issues.apache.org/jira/browse/KAFKA-5602 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-176: Remove deprecated new-consumer option in ConsoleConsumer tool

Hi, I agree that the option can be removed/deprecated. However, I don't understand this: > there will be no check that the new-consumer option is an invalid argument > with the old consumer. I thinks, for backward compatibility, this check should be there. The only change should be, that if

Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand

Jason, Onur, thank you for reviewing the KIP. Regarding the default `--describe` option, so far there have been a few suggestions that conflict a bit. Here are the suggestions: - Keep the current behavior exactly as is (Edo, Jeff) - Remove members with no assignments from the current result set

[jira] [Resolved] (KAFKA-5579) SchemaBuilder.type(Schema.Type) should not allow null.

[ https://issues.apache.org/jira/browse/KAFKA-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-5579. -- Resolution: Fixed Fix Version/s: 0.11.1.0 Issue resolved by pull request

[GitHub] kafka pull request #3517: KAFKA-5579 check for null.

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3517 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Jenkins build is back to normal : kafka-0.11.0-jdk7 #225

See

[jira] [Created] (KAFKA-5604) All producer methods should raise `ProducerFencedException` after the first time.

Apurva Mehta created KAFKA-5604: --- Summary: All producer methods should raise `ProducerFencedException` after the first time. Key: KAFKA-5604 URL: https://issues.apache.org/jira/browse/KAFKA-5604

[GitHub] kafka pull request #3538: KAFKA-5600: fix GroupMetadataManager doesn't read ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3538 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-5603) Streams should not abort transaction when closing zombie task

Matthias J. Sax created KAFKA-5603: -- Summary: Streams should not abort transaction when closing zombie task Key: KAFKA-5603 URL: https://issues.apache.org/jira/browse/KAFKA-5603 Project: Kafka

Jenkins build is back to normal : kafka-trunk-jdk7 #2522

See

Re: [DISCUSS] KIP-168: Add TotalTopicCount metric per cluster

Hi Becket, Thank you for the comments. Vahid also requested a TotalPartitionCount metric and I guess the argument of "aggregating this metric across all brokers in a large cluster can be cumbersome and inefficient" is pretty convincing. I will add this to the proposal and call this metric

Re: [DISCUSS] KIP-168: Add TotalTopicCount metric per cluster

Hi Abhishek, Sorry for the late comments. I feel that the TotalPartitionCount seems a convenient metric to have. It is true that one can add the partition count on each broker and get the total partition count, but it is more work and in a large cluster with hundreds of brokers it seems not that

Re: [VOTE] KIP-134: Delay initial consumer group rebalance

Becket: I think the problem is that when we have a single member joining an unknown group for the first time ever, do we want to complete the rebalance immediately or not; it does not matter if we want to "start" the rebalance, since even for now if the group coordinator is in the SyncGroup phase

[GitHub] kafka pull request #3540: Streams landing page

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka/pull/3540 Streams landing page Content and assets for the updated Streams API landing page You can merge this pull request into a Git repository by running: $ git pull

[jira] [Resolved] (KAFKA-5605) Multiple KStreams being created on application startup are not processing

[ https://issues.apache.org/jira/browse/KAFKA-5605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Connie Chen resolved KAFKA-5605. Resolution: Not A Problem > Multiple KStreams being created on application startup are not

[GitHub] kafka-site issue #67: Updated docs nav and docs toc

Github user derrickdoo commented on the issue: https://github.com/apache/kafka-site/pull/67 @guozhangwang cc @miguno --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

Re: [ANNOUNCE] New Kafka PMC member Jason Gustafson

Thanks everyone! On Thu, Jul 13, 2017 at 6:47 AM, Becket Qin wrote: > Congratulations, Jason! > > On Wed, Jul 12, 2017 at 7:05 PM, Kamal C > wrote: > > > Congrats Jason! > > > > On 13-Jul-2017 00:32, "Jun Rao" wrote: > >

[GitHub] kafka pull request #3447: MINOR: Do not wait for first line of console consu...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3447 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #3542: KAFKA-5362: Follow up to Streams EOS system test

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/3542 KAFKA-5362: Follow up to Streams EOS system test - improve tests to get rid of calls to `sleep` in Python - fixed some flaky test conditions - improve debugging You can merge this pull

Build failed in Jenkins: kafka-0.10.2-jdk7 #182

See Changes: [me] MINOR: Do not wait for first line of console consumer output since we -- [...truncated 326.28 KB...] kafka.server.KafkaConfigTest >

Build failed in Jenkins: kafka-trunk-jdk7 #2525

See Changes: [me] MINOR: Do not wait for first line of console consumer output since we -- [...truncated 2.08 MB...] org.apache.kafka.clients.producer.KafkaProducerTest

Build failed in Jenkins: kafka-trunk-jdk8 #1815

See Changes: [me] MINOR: Do not wait for first line of console consumer output since we -- [...truncated 2.08 MB...] org.apache.kafka.common.security.ssl.SslFactoryTest >

Build failed in Jenkins: kafka-0.11.0-jdk7 #227

See Changes: [me] MINOR: Do not wait for first line of console consumer output since we -- [...truncated 2.08 MB...] org.apache.kafka.clients.producer.MockProducerTest >

Re: [VOTE] KIP-134: Delay initial consumer group rebalance

Hi Guozhang, Sorry for the confusion. I actually meant always "complete" the rebalance immediately when the first consumer joining the group. i.e. the configurable delta only kicks in after the first rebalance. The concern I have was actually not the frequent rebalance for the users, but the

[jira] [Resolved] (KAFKA-5593) Kafka streams not re-balancing when 3 consumer streams are there

[ https://issues.apache.org/jira/browse/KAFKA-5593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh BG resolved KAFKA-5593. -- Resolution: Invalid > Kafka streams not re-balancing when 3 consumer streams are there >

Re: [DISCUSS] KIP-177 Consumer perf tool should count rebalance time

+Users Thanks for the KIP. I think tracking the rebalance time separately will help resolve some confusion about the performance results given the rebalance delay in KIP-134. And it seems generally useful to know how much overhead is coming from the rebalance in any case. -Jason On Thu, Jul 13,

Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand

Hi Jason, Thanks for your quick feedback. Your suggestions seem reasonable. I'll start updating the KIP accordingly and will send out another note when it's ready. Regards. --Vahid From: Jason Gustafson To: dev@kafka.apache.org Cc: Kafka Users

[GitHub] kafka pull request #3526: KAFKA-5587: Remove channel only after staged recei...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3526 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka-site pull request #67: Updated docs nav and docs toc

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/67 Updated docs nav and docs toc - Updated sub nav under docs navigation to display Streams and Connect below APIs - Removed old links to Streams docs that are no longer relevant from table

Build failed in Jenkins: kafka-trunk-jdk7 #2524

See Changes: [ismael] KAFKA-5587; Remove channel only after staged receives are delivered -- [...truncated 2.08 MB...]

[jira] [Resolved] (KAFKA-5587) Processor got uncaught exception: NullPointerException

[ https://issues.apache.org/jira/browse/KAFKA-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-5587. Resolution: Fixed Fix Version/s: 0.11.1.0 0.11.0.1 > Processor got

Re: [DISCUSS] KIP-168: Add TotalTopicCount metric per cluster

Hi All, I updated the proposal to include the GlobalParitionCount metric. The proposal for KIP-168 can be found here ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-168%3A+Add+GlobalTopicCount+and+GlobalPartitionCountmetric+per+cluster ) Best, Abhishek On Mon, Jul 17, 2017 at 2:06 PM,

[GitHub] kafka pull request #3541: MINOR: Add another common error case for CorruptRe...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3541 MINOR: Add another common error case for CorruptRecordException's error message You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka

Build failed in Jenkins: kafka-trunk-jdk8 #1814

See Changes: [ismael] KAFKA-5587; Remove channel only after staged receives are delivered -- [...truncated 2.08 MB...]

Build failed in Jenkins: kafka-0.11.0-jdk7 #226

See Changes: [ismael] KAFKA-5587; Remove channel only after staged receives are delivered -- [...truncated 2.08 MB...]

Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand

Hey Vahid, Hmm... If possible, it would be nice to avoid cluttering the default option too much, especially if it is information which is going to be the same for all members (such as the generation). My preference would be to use the --state option that you've suggested for that info so that we

[jira] [Created] (KAFKA-5605) KStream being created on application startup is not processing

Connie Chen created KAFKA-5605: -- Summary: KStream being created on application startup is not processing Key: KAFKA-5605 URL: https://issues.apache.org/jira/browse/KAFKA-5605 Project: Kafka