[jira] [Assigned] (KAFKA-3201) Add system test for KIP-31 and KIP-32 - Upgrade Test

[ https://issues.apache.org/jira/browse/KAFKA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner reassigned KAFKA-3201: --- Assignee: Anna Povzner > Add system test for KIP-31 and KIP-32 - Upgrade T

[jira] [Commented] (KAFKA-3201) Add system test for KIP-31 and KIP-32 - Upgrade Test

[ https://issues.apache.org/jira/browse/KAFKA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153249#comment-15153249 ] Anna Povzner commented on KAFKA-3201: - Actually, it makes more sense to do this one before

[jira] [Commented] (KAFKA-3188) Add system test for KIP-31 and KIP-32 - Compatibility Test

[ https://issues.apache.org/jira/browse/KAFKA-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133419#comment-15133419 ] Anna Povzner commented on KAFKA-3188: - Cool! I took compatibility test. I can take one more later

[jira] [Assigned] (KAFKA-3188) Add system test for KIP-31 and KIP-32 - Compatibility Test

[ https://issues.apache.org/jira/browse/KAFKA-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner reassigned KAFKA-3188: --- Assignee: Anna Povzner > Add system test for KIP-31 and KIP-32 - Compatibility T

[jira] [Commented] (KAFKA-3188) Add system test for KIP-31 and KIP-32 - Compatibility Test

[ https://issues.apache.org/jira/browse/KAFKA-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1519#comment-1519 ] Anna Povzner commented on KAFKA-3188: - [~becket_qin] Thanks! Do you mind if I take one of these three

Re: [VOTE] KIP-42 -- Add Producer and Consumer Interceptors.

2016 at 7:46 AM, Joel Koshy <jjkosh...@gmail.com> wrote: > > > +1 > > > > On Monday, February 1, 2016, Anna Povzner <a...@confluent.io> wrote: > > > > > Hi All, > > > > > > I am opening the voting thread for KIP-42: Add Produce

[jira] [Updated] (KAFKA-3162) KIP-42: Add Producer and Consumer Interceptors

[ https://issues.apache.org/jira/browse/KAFKA-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-3162: Description: This JIRA is for KIP-42 implementation, which includes: 1. Add ProducerInterceptor

[jira] [Commented] (KAFKA-3188) Add integration test for KIP-31 and KIP-32

[ https://issues.apache.org/jira/browse/KAFKA-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15129105#comment-15129105 ] Anna Povzner commented on KAFKA-3188: - [~becket_qin] Do you mind making separate JIRAs for each

[jira] [Updated] (KAFKA-3162) KIP-42: Add Producer and Consumer Interceptors

[ https://issues.apache.org/jira/browse/KAFKA-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-3162: Status: Patch Available (was: In Progress) > KIP-42: Add Producer and Consumer Intercept

[jira] [Created] (KAFKA-3196) KIP-42 (part 2): add record size and CRC to RecordMetadata and ConsumerRecords

Anna Povzner created KAFKA-3196: --- Summary: KIP-42 (part 2): add record size and CRC to RecordMetadata and ConsumerRecords Key: KAFKA-3196 URL: https://issues.apache.org/jira/browse/KAFKA-3196 Project

[jira] [Updated] (KAFKA-3162) KIP-42: Add Producer and Consumer Interceptors

[ https://issues.apache.org/jira/browse/KAFKA-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-3162: Description: This JIRA is for main part of KIP-42 implementation, which includes: 1. Add

[jira] [Updated] (KAFKA-3162) KIP-42: Add Producer and Consumer Interceptors

[ https://issues.apache.org/jira/browse/KAFKA-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-3162: Issue Type: Improvement (was: Bug) > KIP-42: Add Producer and Consumer Intercept

[VOTE] KIP-42: Add Producer and Consumer Interceptors

Hi All, I am opening the voting thread for KIP-42: Add Producer and Consumer Interceptors. For reference, here's the KIP wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-42%3A+Add+Producer+and+Consumer+Interceptors And the mailing list threads: January:

[VOTE] KIP-42 -- Add Producer and Consumer Interceptors.

Hi All, I am opening the voting thread for KIP-42: Add Producer and Consumer Interceptors. For reference, here's the KIP wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-42%3A+Add+Producer+and+Consumer+Interceptors And the mailing list threads: January:

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

Joel, thanks for your feedback. I updated the wiki based on your comments about the wiki writeup. On Fri, Jan 29, 2016 at 11:50 AM, Anna Povzner <a...@confluent.io> wrote: > Becket, > > In your scenario with one message from producer A and one message from > producer B, those

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

gt; > > > > > 1. Isn't the TopicPartition + Offset already uniquely identified a > > > message? > > > > It seems better than CRC no matter from summary point of view or > > auditing > > > > point of view. > > > > > > > > 2. Curr

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

t; > > Anna, > > > > Thanks for being diligent. > > > > +1 on #1.2 and sounds good on #3. I recommend adding checksum and size > > fields to RecordMetadata and ConsumerRecord instead of exposing metadata > > piecemeal in the interceptor APIs. > > >

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

eptor at the batch level that > can provide this? The other option is to estimate it (such as making an > assumption that the messages in a batch are equal in size, which is not > necessarily true), which is probably not the right answer. > > -Todd > > > On Thu, Jan 28, 2016 a

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

e record size after serialization, i.e. application bytes. (The > uncompressed record size can be easily derived as well) > 2. The actual bytes sent over the wire. > We can get (1) easily, but (2) is difficult to get at Record level when we > use compression. > > Thanks, > &g

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

to discuss. Thanks, Anna On Thu, Jan 28, 2016 at 2:25 PM, Anna Povzner <a...@confluent.io> wrote: > Regarding record size as bytes sent over the wire, my concern is that it > is almost impossible to calculate per-record. We could do as: 1) compressed > bytes / number of records

[jira] [Created] (KAFKA-3162) KIP-42: Add Producer and Consumer Interceptors

Anna Povzner created KAFKA-3162: --- Summary: KIP-42: Add Producer and Consumer Interceptors Key: KAFKA-3162 URL: https://issues.apache.org/jira/browse/KAFKA-3162 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

me know if you have any suggestions or objections to the above. Thanks, Anna On Wed, Jan 27, 2016 at 2:56 PM, Anna Povzner <a...@confluent.io> wrote: > Hi Mayuresh, > > I see why you would want to check for messages left in the > RecordAccumulator. However, I don't thin

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

ing one mutable interceptor on each > side > > > > allows for the use cases we can envision right now, and I think > that’s > > > > going to provide a great deal of opportunity for implementing things > > like > > > > audit, especially within a multi-tenant environ

[jira] [Work started] (KAFKA-3162) KIP-42: Add Producer and Consumer Interceptors

[ https://issues.apache.org/jira/browse/KAFKA-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3162 started by Anna Povzner. --- > KIP-42: Add Producer and Consumer Intercept

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

t; > > > > > > > > > Yeah this is really similar to the callback. The difference > is > > > > really > > > > > > in > > > > > > > > who sets the behavior. The idea of the interceptor is that it > > >

Re: Kafka KIP meeting Jan 26 at 11:00am PST

I will send more detailed meeting notes regarding KIP-42 in the KIP-42 discussion thread. Thanks, Anna On Tue, Jan 26, 2016 at 1:21 PM, Jun Rao wrote: > The following are the notes from today's KIP discussion. > > >- KIP-42: We agreed to leave the broker side interceptor

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

t; > > If we know that there are no more messages left in the RecordAccumulator to > be produced to cluster B, we can safely delete the topic in cluster B > without disturbing the pipeline. > > Thanks, > > Mayuresh > > On Tue, Jan 26, 2016 at 10:31 AM, Anna Povzner &l

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

ntioned. > > > > > > > > > > > > > > I am wondering if there is functional overlapping between > > > > > > > ProducerInterceptor.onAcknowledgement() and the producer > > callback? > > > I > > > > > can > &

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

a Narkhede <n...@confluent.io> > > > > wrote: > > > > > > > > > > > James, > > > > > > > > > > > > That is one of the many monitoring use cases for the interceptor > > > > > interface. > > > >

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

ight be orthogonal, but is there a plan to expose an API to > check if we have messages in RecordAccumulator for a particular topic? I > have a usecase that I can discuss if we plan for this API. > > Thanks, > > Mayuresh > > > On Mon, Jan 25, 2016 at 4:49 PM, Anna Pov

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

> > > interface. > > > > > > > > > > > > Thanks, > > > > > > Neha > > > > > > > > > > > > On Fri, Jan 22, 2016 at 6:18 PM, James Cheng <jch...@tivo.com> > > > wrote: > > > &

[DISCUSS] KIP-42: Add Producer and Consumer Interceptors

Hi, I just created a KIP-42 for adding producer and consumer interceptors for intercepting messages at different points on producer and consumer. https://cwiki.apache.org/confluence/display/KAFKA/KIP-42%3A+Add+Producer+and+Consumer+Interceptors Comments and suggestions are welcome! Thanks,

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

;>> 4. Start upgrading clients > > > >>>>>> 5. When "enough" clients are upgraded, bump format.version to 1 > > > >>> (rolling). > > > >>>>>> > > > >>>>>> Becket, can you confirm? &g

[jira] [Updated] (KAFKA-3025) KIP-32 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord

[ https://issues.apache.org/jira/browse/KAFKA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-3025: Description: The discussion about this JIRA assignment is still under discussion with [~becket_qin

[jira] [Updated] (KAFKA-3026) KIP-32 (part 2): Changes in broker to over-write timestamp or reject message

[ https://issues.apache.org/jira/browse/KAFKA-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-3026: Description: The discussion about this JIRA assignment is still under discussion with [~becket_qin

[jira] [Updated] (KAFKA-3026) KIP-32 (part 2): Changes in broker to over-write timestamp or reject message

[ https://issues.apache.org/jira/browse/KAFKA-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-3026: Description: The decision about this JIRA assignment is still under discussion with [~becket_qin

[VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

Hi, I am opening the voting thread for KIP-32: Add CreateTime and LogAppendTime to Kafka message. For reference, here's the KIP wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+CreateTime+and+LogAppendTime+to+Kafka+message And the mailing list threads: September:

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

iscussion, > I'd > > like to read up rather than repeating it :) > > > > Gwen > > > > On Tue, Dec 22, 2015 at 12:34 PM, Anna Povzner <a...@confluent.io> > wrote: > > > > > Hi, > > > > > > I am opening the voting thread for

[jira] [Commented] (KAFKA-3025) KIP-32 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord

[ https://issues.apache.org/jira/browse/KAFKA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068580#comment-15068580 ] Anna Povzner commented on KAFKA-3025: - [~lindong] : I am in contact with [~becket_qin] about KIP-32

[jira] [Created] (KAFKA-3025) KIP-31 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord

Anna Povzner created KAFKA-3025: --- Summary: KIP-31 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord Key: KAFKA-3025 URL: https://issues.apache.org/jira/browse/KAFKA-3025

[jira] [Created] (KAFKA-3026) KIP-32 (part 2): Changes in broker to over-write timestamp or reject message

Anna Povzner created KAFKA-3026: --- Summary: KIP-32 (part 2): Changes in broker to over-write timestamp or reject message Key: KAFKA-3026 URL: https://issues.apache.org/jira/browse/KAFKA-3026 Project

[jira] [Updated] (KAFKA-3025) KIP-32 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord

[ https://issues.apache.org/jira/browse/KAFKA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-3025: Summary: KIP-32 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord

[jira] [Updated] (KAFKA-2896) System test for partition re-assignment

[ https://issues.apache.org/jira/browse/KAFKA-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-2896: Status: Patch Available (was: In Progress) > System test for partition re-assignm

[jira] [Updated] (KAFKA-2825) Add controller failover to existing replication tests

[ https://issues.apache.org/jira/browse/KAFKA-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-2825: Reviewer: Guozhang Wang (was: Geoff Anderson) > Add controller failover to existing replicat

[jira] [Comment Edited] (KAFKA-2851) system tests: error copying keytab file

[ https://issues.apache.org/jira/browse/KAFKA-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010231#comment-15010231 ] Anna Povzner edited comment on KAFKA-2851 at 12/2/15 6:16 PM: -- Pull request

[jira] [Updated] (KAFKA-2851) system tests: error copying keytab file

[ https://issues.apache.org/jira/browse/KAFKA-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-2851: Reviewer: Guozhang Wang (was: Gwen Shapira) > system tests: error copying keytab f

[jira] [Updated] (KAFKA-2851) system tests: error copying keytab file

[ https://issues.apache.org/jira/browse/KAFKA-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-2851: Resolution: Fixed Status: Resolved (was: Patch Available) > system tests: error copy

[jira] [Commented] (KAFKA-2896) System test for partition re-assignment

[ https://issues.apache.org/jira/browse/KAFKA-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034606#comment-15034606 ] Anna Povzner commented on KAFKA-2896: - [~jinxing6...@126.com] I am already working on it, sorry I did

[jira] [Work started] (KAFKA-2896) System test for partition re-assignment

[ https://issues.apache.org/jira/browse/KAFKA-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2896 started by Anna Povzner. --- > System test for partition re-assignm

[jira] [Assigned] (KAFKA-2896) System test for partition re-assignment

[ https://issues.apache.org/jira/browse/KAFKA-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner reassigned KAFKA-2896: --- Assignee: Anna Povzner > System test for partition re-assignm

[jira] [Issue Comment Deleted] (KAFKA-2851) system tests: error copying keytab file

[ https://issues.apache.org/jira/browse/KAFKA-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-2851: Comment: was deleted (was: In the same PR as KAFKA-2825) > system tests: error copying keytab f

[jira] [Comment Edited] (KAFKA-2851) system tests: error copying keytab file

[ https://issues.apache.org/jira/browse/KAFKA-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010231#comment-15010231 ] Anna Povzner edited comment on KAFKA-2851 at 12/2/15 12:10 AM: --- Pull request

[jira] [Updated] (KAFKA-2851) system tests: error copying keytab file

[ https://issues.apache.org/jira/browse/KAFKA-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-2851: Reviewer: Gwen Shapira Status: Patch Available (was: In Progress) In the same PR as KAFKA

[jira] [Updated] (KAFKA-2825) Add controller failover to existing replication tests

[ https://issues.apache.org/jira/browse/KAFKA-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-2825: Reviewer: Gwen Shapira Status: Patch Available (was: In Progress) The PR also contains

[jira] [Assigned] (KAFKA-2851) system tests: error copying keytab file

[ https://issues.apache.org/jira/browse/KAFKA-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner reassigned KAFKA-2851: --- Assignee: Anna Povzner > system tests: error copying keytab f

[jira] [Work started] (KAFKA-2851) system tests: error copying keytab file

[ https://issues.apache.org/jira/browse/KAFKA-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2851 started by Anna Povzner. --- > system tests: error copying keytab f

[jira] [Commented] (KAFKA-2851) system tests: error copying keytab file

[ https://issues.apache.org/jira/browse/KAFKA-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010231#comment-15010231 ] Anna Povzner commented on KAFKA-2851: - Pull request: https://github.com/apache/kafka/pull/518

[jira] [Work started] (KAFKA-2825) Add controller failover to existing replication tests

[ https://issues.apache.org/jira/browse/KAFKA-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2825 started by Anna Povzner. --- > Add controller failover to existing replication te

[jira] [Created] (KAFKA-2825) Add controller failover to existing replication tests

Anna Povzner created KAFKA-2825: --- Summary: Add controller failover to existing replication tests Key: KAFKA-2825 URL: https://issues.apache.org/jira/browse/KAFKA-2825 Project: Kafka Issue Type

[jira] [Work started] (KAFKA-2769) Integration tests for multi-consumer assignment including session timeouts

[ https://issues.apache.org/jira/browse/KAFKA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2769 started by Anna Povzner. --- > Integration tests for multi-consumer assignment including session timeo

[jira] [Created] (KAFKA-2769) Integration tests for multi-consumer assignment including session timeouts

Anna Povzner created KAFKA-2769: --- Summary: Integration tests for multi-consumer assignment including session timeouts Key: KAFKA-2769 URL: https://issues.apache.org/jira/browse/KAFKA-2769 Project

[jira] [Created] (KAFKA-2737) Integration tests for round-robin assignment

Anna Povzner created KAFKA-2737: --- Summary: Integration tests for round-robin assignment Key: KAFKA-2737 URL: https://issues.apache.org/jira/browse/KAFKA-2737 Project: Kafka Issue Type: Test

[jira] [Work started] (KAFKA-2714) Add integration tests for exceptional cases in Fetching for new consumer

[ https://issues.apache.org/jira/browse/KAFKA-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2714 started by Anna Povzner. --- > Add integration tests for exceptional cases in Fetching for new consu

[jira] [Created] (KAFKA-2714) Add integration tests for exceptional cases in Fetching for new consumer

Anna Povzner created KAFKA-2714: --- Summary: Add integration tests for exceptional cases in Fetching for new consumer Key: KAFKA-2714 URL: https://issues.apache.org/jira/browse/KAFKA-2714 Project: Kafka

<    1   2