[jira] [Created] (KAFKA-7300) Add Fetcher fetch-error-rate and fetch-error-total metrics

Kevin Lu created KAFKA-7300: --- Summary: Add Fetcher fetch-error-rate and fetch-error-total metrics Key: KAFKA-7300 URL: https://issues.apache.org/jira/browse/KAFKA-7300 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-7301) KTable to KTable join invocation does not resolve in Scala DSL

Michal created KAFKA-7301: - Summary: KTable to KTable join invocation does not resolve in Scala DSL Key: KAFKA-7301 URL: https://issues.apache.org/jira/browse/KAFKA-7301 Project: Kafka Issue Type:

Build failed in Jenkins: kafka-trunk-jdk8 #2900

See Changes: [github] KAFKA-7285: Create new producer on each rebalance if EOS enabled (#5501) -- [...truncated 2.48 MB...]

[DISCUSS] KIP-356: Add KafkaConsumer fetch-error-rate and fetch-error-total metrics

Hi friends! :) I believe we currently have a gap in KafkaConsumer metrics for errors since the KafkaConsumer is complex and are many places where things can go wrong. Currently, these failures are logged and certain ones can be inferred from the existing metrics (ex. heartbeat-rate). This KIP

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

Hi Vahid, Thanks for the KIP! +1 (binding) Regards, Rajini On Tue, Aug 14, 2018 at 12:18 AM, Satish Duggana wrote: > +1 (non binding) > Thanks Vahid, > > On Tue, Aug 14, 2018 at 2:22 AM, Gwen Shapira wrote: > > > +1 (binding) > > > > On Tue, Aug 7, 2018 at 11:14 AM, Vahid S Hashemian < > >

[jira] [Resolved] (KAFKA-7119) Intermittent test failure with GSSAPI authentication failure

[ https://issues.apache.org/jira/browse/KAFKA-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-7119. --- Resolution: Fixed > Intermittent test failure with GSSAPI authentication failure >

Add to JIRA

Hi, Could anyone please add me to JIRA? My username is : stana Thanks a lot. -- Best Regards

Java 11 support

Hi All, I have just subscribed to this group. I don't know what is format of questions to be submitted yet but in hurry thought of getting more details on kafka java support. We are planning to use kafka in our project but since our whole env/infra is moving to java 11. I wanted to know the

Re: Java 11 support

check this: https://issues.apache.org/jira/browse/KAFKA-7264 On Thu, Aug 16, 2018 at 2:40 PM Manoj Jain wrote: > Hi All, > > I have just subscribed to this group. I don't know what is format of > questions to be submitted yet but in hurry thought of getting more details > on kafka java

Re: Add to JIRA

Hi, Could anyone please add me to JIRA and confluence My JIRA username is : stana, and Confluence full name is: Yung-An He Thanks a lot.

Re: Java 11 support

Thanks Mani kumar. On Thu, 16 Aug 2018 at 14:42, Manikumar wrote: > check this: https://issues.apache.org/jira/browse/KAFKA-7264 > > On Thu, Aug 16, 2018 at 2:40 PM Manoj Jain wrote: > > > Hi All, > > > > I have just subscribed to this group. I don't know what is format of > > questions to

Jenkins build is back to normal : kafka-1.0-jdk7 #233

See

Build failed in Jenkins: kafka-trunk-jdk10 #409

See Changes: [github] KAFKA-7119: Handle transient Kerberos errors on server side (#5509) -- [...truncated 1.99 MB...]

Build failed in Jenkins: kafka-1.1-jdk7 #182

See Changes: [rajinisivaram] KAFKA-7119: Handle transient Kerberos errors on server side (#5509) -- [...truncated 1.93 MB...] org.apache.kafka.streams.KafkaStreamsTest >

Jenkins build is back to normal : kafka-trunk-jdk8 #2901

See

[DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

Hello, Kafka developers. I would like to start a discussion of KIP-358 [1]. It based on a ticket KAFKA-7277 [2]. I crawled through Stream API and made my suggestions for API changes. I have several questions about changes. Please, share your comments: 1. I propose do not remove existing API

Re: [DISCUSS] KIP-354 Time-based log compaction policy

Hi Xiongqi Thanks for thinking about implementing this as well. :) I was thinking about using `segment.ms` to trigger the segment roll. Also, its value can be the largest time bias for the record deletion. For example, if the `segment.ms` is 1 day and `max.compaction.ms` is 30 days, the

[DISCUSS] KIP-357: Add support to list ACLs per principal

Hi all, I have created a minor KIP to add support to list ACLs per principal using AclCommand (kafka-acls.sh) https://cwiki.apache.org/confluence/display/KAFKA/KIP-357%3A++Add+support+to+list+ACLs+per+principal Please take a look. Thanks,

Re: Add to JIRA

Done. On 8/16/18 3:22 AM, Yung-An He wrote: > Hi, > Could anyone please add me to JIRA and confluence > > My JIRA username is : stana, > and Confluence full name is: Yung-An He > > > Thanks a lot. > signature.asc Description: OpenPGP digital signature

RE: [VOTE] KIP-280: Enhanced log compaction

I want to add my support for the Header strategy. That's the one we care about as well and would very much want to have KIP-280 move forward. Being able to provide an external sequence is very important. Time is not precise enough in a distributed system but there are know good ways to have

Re: [DISCUSS] KIP-354 Time-based log compaction policy

Hi Brett, In my opinion, we don't need very accurate estimation of first record timestamp because the compaction processing itself might take some time. There is no strictly guaranteed that after the time interval "T", the compaction is finished. Rather than, after time "T" + some compaction job

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

+1 (non-binding), thanks for this Vahid! On Thu, Aug 16, 2018 at 10:39 AM Rajini Sivaram wrote: > Hi Vahid, Thanks for the KIP! > > +1 (binding) > > Regards, > > Rajini > > On Tue, Aug 14, 2018 at 12:18 AM, Satish Duggana > > wrote: > > > +1 (non binding) > > Thanks Vahid, > > > > On Tue, Aug

[jira] [Resolved] (KAFKA-7296) Producer should handle COORDINATOR_LOADING error in TxnOffsetCommit

[ https://issues.apache.org/jira/browse/KAFKA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7296. Resolution: Fixed > Producer should handle COORDINATOR_LOADING error in TxnOffsetCommit >

Re: [DISCUSS] KIP-354 Time-based log compaction policy

Hi Xiaohe, Quick note: 1) Use minimum of segment.ms and max.compaction.lag.ms 2) I am not sure if I get your second question. first, we have jitter when we roll the active segment. second, on each compaction, we compact upto the offsetmap could allow. Those will not

Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests

Hi Lucas, Thanks for updating the wiki. The updated motivation description looks good to me. One additional benefit of having a separate port for controller messages is that we can protect the control plane with something like IP table. Reading the proposed change a bit more, I found it is a

RE: [VOTE] KIP-280: Enhanced log compaction

Hi, @Guozhang & @Jun: There were some left over comments from when the topic was still fresh (you may have to read the email chain to refresh your memory). Are these now clarified for you? @Jason: Was the reason opaque to you? I think we can avoid adding something so simple to the

Re: [DISCUSS] KIP-280: Enhanced log compaction

I'm interested to know the status of this KIP. I see that the status is "Voting". How long does this normally take? We want to use Kafka and this KIP provides exactly the log compaction logic that we want for many of our projects. One piece of feedback that I have is that

Upgrading Kaka to 1.0 and fetch response ignores fetch.message.max.bytes

Good afternoon everyone First of all just let me say that this is my first post and I am new to this mailing list and I am new to Apache Kafka, so please bear with me as I figure things out. My problem is this: I am trying to upgrade a proprietary application that consumes records from Kafka.

Re: [DISCUSS] KIP-280: Enhanced log compaction

Regarding "broker-agnostic of headers": there are some KIPs from Streams to use headers for internal purposes as well, e.g. KIP-258 and KIP-213 (I admit there may be a conflict with user space, but practically I think it is very rare). So I think we are very likely going to make Kafka internals to

Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests

Thanks for the review, Becket. (1) After comparing the two approaches, I still feel the current writeup is a little better. a. The current writeup asks for an explicit endpoint while reusing the existing "inter.broker.listener.name" with the exactly same semantic, and your proposed change asks

Build failed in Jenkins: kafka-trunk-jdk10 #410

See Changes: [github] KAFKA-7296; Handle coordinator loading error in TxnOffsetCommit (#5514) -- [...truncated 2.11 MB...] at

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

This KIP has passed with 3 binding and 8 non-binding +1 votes. Binding votes: * Jason Gustafson * Gwen Shapira * Rajini Sivaram Non-binding votes: * Stanislav Kozlovski * Kamal Chandraprakash * Mickael Maison * Ted Yu * Manikumar * Colin McCabe * Satish Duggana * Viktor Somogyi Thanks to all

Re: [VOTE] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

+1 On Thu, Aug 16, 2018 at 12:05 PM Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > I would like to start a vote on KIP-325 which aims at adding a beginning > offset column to consumer group command describe output. > > The KIP: > >

Build failed in Jenkins: kafka-1.1-jdk7 #183

See Changes: [rajinisivaram] MINOR: Clean up to avoid errors in dynamic broker config tests (#5486) -- [...truncated 1.93 MB...]

Re: [VOTE] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

+1 Thanks, Bill On Thu, Aug 16, 2018 at 3:13 PM Ted Yu wrote: > +1 > > On Thu, Aug 16, 2018 at 12:05 PM Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > > > I would like to start a vote on KIP-325 which aims at adding a beginning > > offset column to consumer group command describe

[jira] [Created] (KAFKA-7303) Kafka client is stuck when specifying wrong brokkers

Chun Zhang created KAFKA-7303: - Summary: Kafka client is stuck when specifying wrong brokkers Key: KAFKA-7303 URL: https://issues.apache.org/jira/browse/KAFKA-7303 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-7302) Remove Java7 examples from Streams Docs

Matthias J. Sax created KAFKA-7302: -- Summary: Remove Java7 examples from Streams Docs Key: KAFKA-7302 URL: https://issues.apache.org/jira/browse/KAFKA-7302 Project: Kafka Issue Type:

[VOTE] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

I would like to start a vote on KIP-325 which aims at adding a beginning offset column to consumer group command describe output. The KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-325%3A+Extend+Consumer+Group+Command+to+Show+Beginning+Offsets Discussion thread:

[VOTE] KIP-347: Enable batching in FindCoordinatorRequest

Hi all, I want to start a vote on this KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-347%3A++Enable+batching+in+FindCoordinatorRequest Here is the discussion thread:

Re: [DISCUSS] KIP-354 Time-based log compaction policy

Hey Xiongqi, Thanks for the update. I have two questions for the latest KIP. 1) The motivation section says that one use case is to delete PII (Personal Identifiable information) data within 7 days while keeping non-PII indefinitely in compacted format. I suppose the use-case depends on the

Re: [VOTE] KIP-347: Enable batching in FindCoordinatorRequest

+1 On Thu, Aug 16, 2018 at 5:15 PM Yishun Guan wrote: > Hi all, > > I want to start a vote on this KIP: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-347%3A++Enable+batching+in+FindCoordinatorRequest > > Here is the discussion thread: > >

[jira] [Resolved] (KAFKA-7299) batch LeaderAndIsr requests during auto preferred leader election

[ https://issues.apache.org/jira/browse/KAFKA-7299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-7299. Resolution: Fixed Fix Version/s: 2.1.0 Merged the PR to trunk. > batch LeaderAndIsr requests

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

It is not implemented, but should not be hard to do so (and again you do NOT have to do that in this KIP, I'm bringing this up so that you can help thinking about the process). Quoting from Colin's comment: " The pattern is that you would try to send a request for more than one group, and then

答复: [VOTE] KIP-347: Enable batching in FindCoordinatorRequest

+1 (non-binding) 发件人: Yishun Guan 发送时间: 2018年8月17日 8:14 收件人: dev@kafka.apache.org 主题: [VOTE] KIP-347: Enable batching in FindCoordinatorRequest Hi all, I want to start a vote on this KIP:

Jenkins build is back to normal : kafka-trunk-jdk10 #411

See

Re: [DISCUSS] KIP-354 Time-based log compaction policy

1, Owner of data (in this sense, kafka is the not the owner of data) should keep track of lifecycle of the data in some external storage/DB. The owner determines when to delete the data and send the delete request to kafka. Kafka doesn't know about the content of data but to provide a mean for

Re: [DISCUSS] KIP-354 Time-based log compaction policy

To just clarify a bit on 1. whether there's an external storage/DB isn't relevant here. Compacted topics allow a tombstone record to be sent (a null value for a key) which currently will result in old values for that key being deleted if some conditions are met. There are existing controls to

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

I see! Thanks! On Thu, Aug 16, 2018, 4:35 PM Guozhang Wang wrote: > It is not implemented, but should not be hard to do so (and again you do > NOT have to do that in this KIP, I'm bringing this up so that you can help > thinking about the process). > > Quoting from Colin's comment: > > " > The

Build failed in Jenkins: kafka-trunk-jdk8 #2903

See Changes: [junrao] KAFKA-7299: Batch LeaderAndIsr requests for AutoLeaderRebalance (#5515) -- [...truncated 425.21 KB...] kafka.zk.KafkaZkClientTest >

Re: [DISCUSS] KIP-354 Time-based log compaction policy

on 2) The offsetmap is built starting from dirty segment. The compaction starts from the beginning of the log partition. That's how it ensure the deletion of tomb keys. I will double check tomorrow. Xiongqi (Wesley) Wu On Thu, Aug 16, 2018 at 6:46 PM Brett Rann wrote: > To just clarify a bit

Re: [VOTE] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

+1 Thanks, Satish. On Fri, Aug 17, 2018 at 1:45 AM, Bill Bejeck wrote: > +1 > > Thanks, > Bill > > On Thu, Aug 16, 2018 at 3:13 PM Ted Yu wrote: > > > +1 > > > > On Thu, Aug 16, 2018 at 12:05 PM Vahid S Hashemian < > > vahidhashem...@us.ibm.com> wrote: > > > > > I would like to start a vote