Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Thanks Guozhang for the new proposal here! So I'd like to propose a slightly modified version of LeaveGroupRequest: instead of letting the static member consumer client themselves to send the request (which means we still need to have some hidden configs to turn it off like we did today), how

Jenkins build is back to normal : kafka-2.1-jdk8 #64

See

[jira] [Created] (KAFKA-7686) Remove PowerMock from Connect Tests

Magesh kumar Nandakumar created KAFKA-7686: -- Summary: Remove PowerMock from Connect Tests Key: KAFKA-7686 URL: https://issues.apache.org/jira/browse/KAFKA-7686 Project: Kafka Issue

Build failed in Jenkins: kafka-trunk-jdk8 #3226

See Changes: [wangguoz] KAFKA-7671: Stream-Global Table join should not reset repartition flag [junrao] Forward topic from console consumer to deserializer (#5704) --

Re: [DISCUSS] KIP-394: Require member.id for initial join group request

Thanks Matthias for the question, and Stanislav for the explanation! For the scenario described, we will never let a member join the GroupMetadata map if it uses UNKNOWN_MEMBER_ID. So the workflow will be like this: 1. Group is empty. Consumer c1 started. Join with UNKNOWN_MEMBER_ID; 2.

[jira] [Resolved] (KAFKA-7449) Kafka console consumer is not sending topic to deserializer

[ https://issues.apache.org/jira/browse/KAFKA-7449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-7449. Resolution: Fixed Assignee: Mathieu Chataigner Fix Version/s: 2.2.0 Merged to trunk. >

Re: [VOTE] KIP-351: Add --under-min-isr option to describe TopicCommand

@Manikumar I have updated the Compatibility section to note that this option is not supported with the deprecated "--zookeeper" option. We still need 1 more binding vote for this to be accepted. Thanks~ Regards, Kevin On Wed, Nov 28, 2018 at 9:23 AM Gwen Shapira wrote: > +1 (binding) > On

Re: [DISCUSS] KIP-379: Multiple Consumer Group Management

Hi Alex, Sorry for the late reply. Your message didn't get attached to the main thread and I missed it. #1. Yeah, I think we should continue to accept the old csv format. Compatibility is important for all public APIs. #2. I think this is less important from a compatibility perspective. On the

Re: [DISCUSS] KIP-392: Allow consumers to fetch from the closest replica

Hi, Jason, Thanks for the KIP. Looks good overall. A few minor comments below. 1. The section on handling FETCH_OFFSET_TOO_LARGE error says "Use the OffsetForLeaderEpoch API to verify the current position with the leader". The OffsetForLeaderEpoch request returns log end offset if the request

Build failed in Jenkins: kafka-trunk-jdk8 #3225

See Changes: [colin] MINOR: Support long maxMessages in Trogdor consume/produce bench workers -- [...truncated 2.64 MB...]

Problem in CI for pull request

Hi, I made a pull request and it passed CI on JDK 11 but failed on JDK 8 I think the JDK 8 error may not related to my commit but an environment problem on the CI How can I rerun the CI for my pull request ? The pull request is at https://github.com/apache/kafka/pull/5960 error states

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Hi Boyang, I was thinking that with the optional static members in the admin ConsumerRebalanceRequest it should be sufficient to kick out the static member before their session timeout (arguably long in practice) have not reached. But now I see your concern is that in some situations the admin

Re: [VOTE] KIP-351: Add --under-min-isr option to describe TopicCommand

+1 (binding) On Thu, Nov 8, 2018 at 1:37 PM Kevin Lu wrote: > > Hi All, > > I'm starting the vote thread for KIP-351: Add --under-min-isr option to > describe topics command. > > KIP: >

RE: [DISCUSS] KIP-383 Pluggable interface for SSL Factory

If no one objects, I'll rewrite the KIP like this and resubmit for discussion: Ideal interface, ignore the atomicity issue, only keystore and truststore is reconfigurable. I was not planning to log a Jira for the CertificateEntries comparison bug because my fix will correct it anyway.

Re: [DISCUSS] KIP-390: Add producer option to adjust compression level

Hi Dongjin, To clarify, I mean a broker topic config with regards to point 1. As you know, compression can be done by the producer and/or by the broker. The default is for the broker to just use whatever compression was used by the producer, but this can be changed by the user on a per topic

Re: [VOTE] KIP-351: Add --under-min-isr option to describe TopicCommand

Thanks for the KIP +1 (non-binding) -Bill On Wed, Nov 28, 2018 at 7:06 AM Manikumar wrote: > +1 (binding) > Thanks for the KIP. > > Can you update the KIP stating that this option is not supported with > "--zookeeper" option. > > On Mon, Nov 26, 2018 at 11:15 PM Mickael Maison > wrote: > > >

Re: problems in Kafka unit testing trunk

ulimit shows $ ulimit -n 1024 is it too small for Kafka ? On Wed, Nov 28, 2018 at 6:18 AM Dhruvil Shah wrote: > The unit test itself does not seem to use too many files. What is the > output for `ulimit -n` on your system? Running `lsof` might also be helpful > to determine how many open

Re: [DISCUSS] KIP-390: Add producer option to adjust compression level

Thank you Ismael, here are the answers: *1. About topic config* After some consideration, I concluded that topic config doesn't need to support compression.level. Here is why: since the compression is conducted by the client, the one who can select the best compression level is the client

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

On Tue, Nov 27, 2018 at 6:31 PM Randall Hauch wrote: > Thanks for the feedback. Some thoughts inline. > > On Tue, Nov 27, 2018 at 5:47 PM Ewen Cheslack-Postava > wrote: > >> re: AdminClient vs this proposal, one consideration is that AdminClient >> exposes a lot more surface area and probably a

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Thanks Guozhang and Mayuresh for the follow up! Answers are listed below. > 5. Regarding "So in summary, *the member will only be removed due to > session timeout*. We shall remove it from both in-memory static member name > mapping and member list." If the rebalance is invoked manually using

[jira] [Created] (KAFKA-7685) Support loading trust stores from classpath

Noa Resare created KAFKA-7685: - Summary: Support loading trust stores from classpath Key: KAFKA-7685 URL: https://issues.apache.org/jira/browse/KAFKA-7685 Project: Kafka Issue Type: Improvement

Re: [VOTE] KIP-351: Add --under-min-isr option to describe TopicCommand

+1 (binding) Thanks for the KIP. Can you update the KIP stating that this option is not supported with "--zookeeper" option. On Mon, Nov 26, 2018 at 11:15 PM Mickael Maison wrote: > +1 (non-binding) > Thanks for the KIP! > On Mon, Nov 26, 2018 at 4:32 PM Kevin Lu wrote: > > > > Hi All, > > >

Re: [DISCUSS] KIP-394: Require member.id for initial join group request

Hey Matthias, I think the notion is to have the `session.timeout.ms` to start ticking when the broker responds with the member.id. Then, the broker would properly expire consumers and not hold too many stale ones. This isn't mentioned in the KIP though so it is worth to wait for Boyang to confirm

Jenkins build is back to normal : kafka-trunk-jdk11 #120

See

Build failed in Jenkins: kafka-2.1-jdk8 #63

See Changes: [me] KAFKA-7620: Fix restart logic for TTLs in WorkerConfigTransformer -- [...truncated 2.63 MB...] org.apache.kafka.streams.TopologyTest >

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

Hi Jason, You raise some very valid points. > The benefit of this KIP is probably limited to preventing "runaway" consumer groups due to leaks or some other application bug What do you think about the use case I mentioned in my previous reply about a more resilient self-service Kafka? I believe

[jira] [Resolved] (KAFKA-6149) LogCleanerManager should include topic partition name when warning of invalid cleaner offset

[ https://issues.apache.org/jira/browse/KAFKA-6149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-6149. -- Resolution: Fixed Fix Version/s: 0.11.0.1 1.0.0 This was fixed in 1.0 and

Jenkins build is back to normal : kafka-trunk-jdk8 #3222

See

Build failed in Jenkins: kafka-trunk-jdk11 #119

See Changes: [me] KAFKA-7620: Fix restart logic for TTLs in WorkerConfigTransformer -- [...truncated 2.26 MB...] org.apache.kafka.trogdor.common.WorkerUtilsTest >