Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

alances. > > Hope this answers your question. > > Thanks, > Apurva > > On Tue, Dec 6, 2016 at 3:22 PM, Ben Kirwin <b...@kirw.in> wrote: > > > Thanks for this! I'm looking forward to going through the full proposal > in > > detail soon; a few early

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Thanks for this! I'm looking forward to going through the full proposal in detail soon; a few early questions: First: what happens when a consumer rebalances in the middle of a transaction? The full documentation suggests that such a transaction ought to be rejected: > [...] if a rebalance has

[jira] [Commented] (KAFKA-3335) Kafka Connect hangs in shutdown hook

[ https://issues.apache.org/jira/browse/KAFKA-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313267#comment-15313267 ] Ben Kirwin commented on KAFKA-3335: --- Excellent -- thanks for following up on this! > Kafka Conn

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313258#comment-15313258 ] Ben Kirwin commented on KAFKA-2260: --- Hi! I haven't had time to push on this at all in the last few

[jira] [Created] (KAFKA-3335) Kafka Connect hangs in shutdown hook

Ben Kirwin created KAFKA-3335: - Summary: Kafka Connect hangs in shutdown hook Key: KAFKA-3335 URL: https://issues.apache.org/jira/browse/KAFKA-3335 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-27 - Conditional Publish

to think about what should do if a request got rejected and the answer varies for different use cases. Thanks, Jiangjie (Becket) Qin On Sun, Jul 26, 2015 at 11:54 AM, Ben Kirwin b...@kirw.in wrote: So I had another look at the 'Idempotent Producer' proposal this afternoon, and made

Re: [DISCUSS] KIP-27 - Conditional Publish

at 11:54 AM, Ben Kirwin b...@kirw.in wrote: So I had another look at the 'Idempotent Producer' proposal this afternoon, and made a few notes on how I think they compare; if I've made any mistakes, I'd be delighted if someone with more context on the idempotent producer design would

Re: [DISCUSS] KIP-27 - Conditional Publish

a transient multiple producer scenario will cause issue because user need to think about what should do if a request got rejected and the answer varies for different use cases. Thanks, Jiangjie (Becket) Qin On Sun, Jul 26, 2015 at 11:54 AM, Ben Kirwin b...@kirw.in wrote: So I had another look

Re: [DISCUSS] KIP-27 - Conditional Publish

or not. Why a batch would be partially committed? Thanks, Jiangjie (Becket) Qin On Tue, Jul 21, 2015 at 10:42 AM, Ben Kirwin b...@kirw.in wrote: That's a fair point. I've added some imagined job logic to the KIP, so we can make sure the proposal stays in sync with the usages we're

Re: [DISCUSS] KIP-27 - Conditional Publish

interface does sound convenient. We plan to have a KIP discussion meeting tomorrow at 11am PST. Perhaps you can describe this KIP a bit then? Sure, happy to join. Thanks, Jun On Sat, Jul 18, 2015 at 10:37 AM, Ben Kirwin b...@kirw.in wrote: Just wanted to flag a little discussion

Re: [DISCUSS] KIP-27 - Conditional Publish

. -Flavio On 17 Jul 2015, at 07:30, Ben Kirwin b...@kirw.in wrote: Hi there, I just added a KIP for a 'conditional publish' operation: a simple CAS-like mechanism for the Kafka producer. The wiki page is here: https://cwiki.apache.org/confluence/display/KAFKA

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632537#comment-14632537 ] Ben Kirwin commented on KAFKA-2260: --- Ah, clever! Thanks for sharing this -- I've linked

Re: [DISCUSS] KIP-27 - Conditional Publish

of the drawbacks of the per-key approach mentioned at the bottom of the KIP. On Fri, Jul 17, 2015 at 6:47 PM, Ben Kirwin b...@kirw.in wrote: Hi all, So, perhaps it's worth adding a couple specific examples of where this feature is useful, to make this a bit more concrete: - Suppose I'm using Kafka

[DISCUSS] KIP-27 - Conditional Publish

Hi there, I just added a KIP for a 'conditional publish' operation: a simple CAS-like mechanism for the Kafka producer. The wiki page is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-27+-+Conditional+Publish And there's some previous discussion on the ticket and the users list:

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630870#comment-14630870 ] Ben Kirwin commented on KAFKA-2260: --- Opened a KIP for this here: https

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632120#comment-14632120 ] Ben Kirwin commented on KAFKA-2260: --- Interesting -- the idea is that we track the max

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632113#comment-14632113 ] Ben Kirwin commented on KAFKA-2260: --- Ah, sorry! Let me try again. Suppose you try

Re: [DISCUSS] KIP-27 - Conditional Publish

2015, at 07:30, Ben Kirwin b...@kirw.in wrote: Hi there, I just added a KIP for a 'conditional publish' operation: a simple CAS-like mechanism for the Kafka producer. The wiki page is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-27+-+Conditional+Publish

Confluence Wiki Permissions

Hi there, I'm interested in posting a KIP (for this ticket: https://issues.apache.org/jira/browse/KAFKA-2260) but I don't seem to have the right permissions to create the wiki page. Can anyone here set that up? (Username is `bkirwi`.)

Re: Confluence Wiki Permissions

Great -- thank you! On Wed, Jul 15, 2015 at 9:13 PM, Jun Rao j...@confluent.io wrote: Done. Thanks, Jun On Wed, Jul 15, 2015 at 5:22 PM, Ben Kirwin b...@kirw.in wrote: Hi there, I'm interested in posting a KIP (for this ticket: https://issues.apache.org/jira/browse/KAFKA-2260) but I

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627460#comment-14627460 ] Ben Kirwin commented on KAFKA-2260: --- Hi [~ewencp] -- thanks for the interest! I'd

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627484#comment-14627484 ] Ben Kirwin commented on KAFKA-2260: --- Will do; thanks! Allow specifying expected offset

[jira] [Updated] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Kirwin updated KAFKA-2260: -- Status: Patch Available (was: Open) Worked up a draft of this over the weekend, implementing the cas

[jira] [Updated] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Kirwin updated KAFKA-2260: -- Attachment: expected-offsets.patch Allow specifying expected offset on produce

[jira] [Created] (KAFKA-2260) Allow specifying expected offset on produce

Ben Kirwin created KAFKA-2260: - Summary: Allow specifying expected offset on produce Key: KAFKA-2260 URL: https://issues.apache.org/jira/browse/KAFKA-2260 Project: Kafka Issue Type: Improvement