Re: [VOTE] 2.2.1 RC0

2019-05-10 Thread Jason Gustafson
Hi Vahid, I'd like to make the case for https://issues.apache.org/jira/browse/KAFKA-8335. This issue can cause unbounded growth in the __consumer_offsets topic when using transactions. I will have a patch ready today. Can we do another RC? Thanks, Jason On Wed, May 8, 2019 at 1:26 PM Vahid Hashe

[jira] [Created] (KAFKA-8351) Log cleaner must handle transactions spanning multiple segments

2019-05-10 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8351: -- Summary: Log cleaner must handle transactions spanning multiple segments Key: KAFKA-8351 URL: https://issues.apache.org/jira/browse/KAFKA-8351 Project: Kafka

[jira] [Resolved] (KAFKA-6789) Add retry logic in AdminClient requests

2019-05-09 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6789. Resolution: Fixed Fix Version/s: 2.2.2 2.1.2

Re: [VOTE] KIP-461 Improve Replica Fetcher behavior at handling partition failure

2019-05-08 Thread Jason Gustafson
+1. Thanks! On Wed, May 8, 2019 at 4:30 PM Aishwarya Gune wrote: > Hi All! > > I would like to call for a vote on KIP-461 that would improve the behavior > of replica fetcher in case of partition failure. The fetcher thread would > just stop monitoring the crashed partition instead of terminatin

[jira] [Created] (KAFKA-8341) AdminClient should retry coordinator lookup after NOT_COORDINATOR error

2019-05-08 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8341: -- Summary: AdminClient should retry coordinator lookup after NOT_COORDINATOR error Key: KAFKA-8341 URL: https://issues.apache.org/jira/browse/KAFKA-8341 Project

[jira] [Resolved] (KAFKA-8049) remove KafkaMbean when network close

2019-05-08 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8049. Resolution: Duplicate > remove KafkaMbean when network cl

[jira] [Resolved] (KAFKA-8050) remove KafkaMbean when network close

2019-05-08 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8050. Resolution: Duplicate > remove KafkaMbean when network cl

[jira] [Resolved] (KAFKA-8048) remove KafkaMbean when network close

2019-05-08 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8048. Resolution: Duplicate > remove KafkaMbean when network cl

[jira] [Resolved] (KAFKA-8051) remove KafkaMbean when network close

2019-05-08 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8051. Resolution: Duplicate > remove KafkaMbean when network cl

Re: [DISCUSS] KIP-461 - Improving replica fetcher behavior in case of partition failures

2019-05-08 Thread Jason Gustafson
Hey Aishwarya, Thanks for the KIP. I'd suggest we move to a vote since this is a straightforward improvement with a large impact. -Jason On Tue, May 7, 2019 at 3:02 PM Aishwarya Gune wrote: > Hi Colin! > > Whenever the thread has all of its partitions marked as failed (i.e. thread > is idle),

[jira] [Resolved] (KAFKA-7320) Provide ability to disable auto topic creation in KafkaConsumer

2019-05-08 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7320. Resolution: Fixed Fix Version/s: 2.3.0 > Provide ability to disable auto to

[jira] [Created] (KAFKA-8333) Load high watermark checkpoint only once when handling LeaderAndIsr requests

2019-05-07 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8333: -- Summary: Load high watermark checkpoint only once when handling LeaderAndIsr requests Key: KAFKA-8333 URL: https://issues.apache.org/jira/browse/KAFKA-8333

[jira] [Resolved] (KAFKA-8275) NetworkClient leastLoadedNode selection should consider throttled nodes

2019-05-07 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8275. Resolution: Fixed Fix Version/s: 2.3.0 > NetworkClient leastLoadedNode select

[jira] [Resolved] (KAFKA-8255) Replica fetcher thread exits with OffsetOutOfRangeException

2019-05-06 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8255. Resolution: Resolved I'm going to resolve this. We believe the root cause was KAFKA

[jira] [Resolved] (KAFKA-8056) Replace FindCoordinator request/response with automated protocol

2019-05-06 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8056. Resolution: Fixed > Replace FindCoordinator request/response with automated proto

[jira] [Resolved] (KAFKA-8306) Ensure consistency of checkpointed log start offset and current log end offset

2019-05-04 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8306. Resolution: Fixed Fix Version/s: 2.2.1 2.1.2

[jira] [Resolved] (KAFKA-7601) Handle message format downgrades during upgrade of message format version

2019-05-02 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7601. Resolution: Fixed Fix Version/s: 2.2.1 2.1.2 The patch fixes the

Re: [VOTE] KIP-460: Admin Leader Election RPC

2019-05-01 Thread Jason Gustafson
+1 Thanks for the KIP. -Jason On Wed, May 1, 2019 at 2:44 PM Jose Armando Garcia Sancio < jsan...@confluent.io> wrote: > Hi all, > > I would like to start the voting for KIP-460: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-460%3A+Admin+Leader+Election+RPC > > The thread discussion

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-01 Thread Jason Gustafson
Looks helpful. +1 -Jason On Tue, Apr 30, 2019 at 5:54 PM Randall Hauch wrote: > Nice simple improvement. Thanks, Paul! > > +1 (binding) > > Randall > > On Mon, Apr 29, 2019 at 5:06 PM Magesh Nandakumar > wrote: > > > Looks good to me and a very useful feature. > > > > +1 ( non-binding) > > > >

[jira] [Resolved] (KAFKA-8066) ReplicaFetcherThread fails to startup because of failing to register the metric.

2019-05-01 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8066. Resolution: Fixed Fix Version/s: 2.2.1 2.1.2

Re: [DISCUSS] KIP-460: Admin Leader Election RPC

2019-04-30 Thread Jason Gustafson
Thanks for the updates, Jose. The proposal looks good to me. Just one minor question I had is whether we should even have a default --election-type in kafka-leader-election.sh. I am wondering if it is reasonable to make the user be explicit about what they are trying to do? -Jason On Fri, Apr 26,

[jira] [Created] (KAFKA-8306) Ensure consistency of checkpointed log start offset and current log end offset

2019-04-29 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8306: -- Summary: Ensure consistency of checkpointed log start offset and current log end offset Key: KAFKA-8306 URL: https://issues.apache.org/jira/browse/KAFKA-8306

[jira] [Resolved] (KAFKA-7779) Avoid unnecessary loop iteration in leastLoadedNode

2019-04-29 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7779. Resolution: Fixed > Avoid unnecessary loop iteration in leastLoadedN

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-04-26 Thread Jason Gustafson
t; I'll update the KIP to reflect this. > > > > Viktor > > > > On Thu, Apr 25, 2019 at 8:07 PM Jason Gustafson > > wrote: > > > >> Hi Viktor, > >> > >> This looks good. Just one question I had is whether we may as well cover >

[jira] [Created] (KAFKA-8294) Batch StopReplica requests with partition deletion and add test cases

2019-04-26 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8294: -- Summary: Batch StopReplica requests with partition deletion and add test cases Key: KAFKA-8294 URL: https://issues.apache.org/jira/browse/KAFKA-8294 Project

Re: [DISCUSS] KIP-460: Admin Leader Election RPC

2019-04-25 Thread Jason Gustafson
Hi Jose, This looks useful. One comment I had is whether we can improve the leader election tool. Needing to provide a json file is a bit annoying. Could we have a way to specify partitions directly through the command line? Often when we need to enable unclean leader election, it is just one or a

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-04-25 Thread Jason Gustafson
Hi Viktor, This looks good. Just one question I had is whether we may as well cover the log dir fetchers as well. Thanks, Jason On Thu, Apr 25, 2019 at 7:46 AM Viktor Somogyi-Vass wrote: > Hi Folks, > > This thread sunk a bit but I'd like to bump it hoping to get some feedback > and/or votes.

[jira] [Resolved] (KAFKA-8237) Untangle TopicDeletionManager and add test cases

2019-04-24 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8237. Resolution: Fixed > Untangle TopicDeletionManager and add test ca

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2019-04-24 Thread Jason Gustafson
Hi George, I think focusing on the admin API for cancellation in this KIP is reasonable. Colin wrote up KIP-455 which adds APIs to submit a reassignment and to list reassignments in progress. Probably as long as we make these APIs all consistent with each other, it is fine to do them separately.

[jira] [Created] (KAFKA-8283) Add additional test cases for the topic deletion state machine

2019-04-23 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8283: -- Summary: Add additional test cases for the topic deletion state machine Key: KAFKA-8283 URL: https://issues.apache.org/jira/browse/KAFKA-8283 Project: Kafka

[jira] [Created] (KAFKA-8275) NetworkClient leastLoadedNode selection should consider throttled nodes

2019-04-22 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8275: -- Summary: NetworkClient leastLoadedNode selection should consider throttled nodes Key: KAFKA-8275 URL: https://issues.apache.org/jira/browse/KAFKA-8275 Project

[jira] [Resolved] (KAFKA-7747) Consumer should check for truncation after leader changes

2019-04-21 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7747. Resolution: Fixed Fix Version/s: 2.3.0 > Consumer should check for truncation af

[jira] [Created] (KAFKA-8266) Improve `testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup`

2019-04-19 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8266: -- Summary: Improve `testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup` Key: KAFKA-8266 URL: https://issues.apache.org/jira/browse/KAFKA-8266

[jira] [Reopened] (KAFKA-7965) Flaky Test ConsumerBounceTest#testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup

2019-04-18 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson reopened KAFKA-7965: Assignee: Jason Gustafson (was: huxihx) I'm still seeing this:  [

[jira] [Resolved] (KAFKA-7866) Duplicate offsets after transaction index append failure

2019-04-18 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7866. Resolution: Fixed Assignee: Jason Gustafson Fix Version/s: 2.2.1

[jira] [Resolved] (KAFKA-7026) Sticky assignor could assign a partition to multiple consumers (KIP-341)

2019-04-18 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7026. Resolution: Fixed Fix Version/s: 2.3.0 > Sticky assignor could assign a partit

[jira] [Resolved] (KAFKA-7965) Flaky Test ConsumerBounceTest#testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup

2019-04-17 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7965. Resolution: Fixed > Flaky Test > ConsumerBoun

[jira] [Created] (KAFKA-8237) Untangle TopicDeletionManager and add test cases

2019-04-15 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8237: -- Summary: Untangle TopicDeletionManager and add test cases Key: KAFKA-8237 URL: https://issues.apache.org/jira/browse/KAFKA-8237 Project: Kafka Issue

Re: [DISCUSS] KIP-429 : Smooth Auto-Scaling for Kafka Streams

2019-04-12 Thread Jason Gustafson
Hi Guozhang, Responses below: 2. The interface's default implementation will just be > `onPartitionRevoked`, so for user's instantiation if they do not make any > code changes they should be able to recompile the code and continue. Ack, makes sense. 4. Hmm.. not sure if it will work. The main

Re: [DISCUSS] KIP-429 : Smooth Auto-Scaling for Kafka Streams

2019-04-10 Thread Jason Gustafson
Hi Guozhang and Boyang, Thanks for the KIP. A few comments/questions below: 1. More of a nitpick, but `onPartitionsEmigrated` is not a very clear name. How about `onPartitionsEvicted`? Or even perhaps `onMembershipLost`? 2. For `onPartitionsEmigrated`, how will we maintain compatibility with the

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

2019-04-09 Thread Jason Gustafson
icas in > /brokers/topics/[topic]/partitions/[partitionId]/state, rather than in > the reassignment znode? > I don't think this requires a major change to the proposal-- when the > controller becomes > aware that it should do a reassignment, the controller could make the > ch

Re: [VOTE] KIP-360: Improve handling of unknown producer when using EOS

2019-04-09 Thread Jason Gustafson
Bump (for Guozhang) On Mon, Apr 8, 2019 at 8:55 AM Jason Gustafson wrote: > Hi All, > > I'd like to start a vote on KIP-360: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-360%3A+Improve+handling+of+unknown+producer > . > > +1 from me (duh) > > Thanks, > Jason >

[jira] [Resolved] (KAFKA-8013) Avoid buffer underflow when reading a Struct from a partially correct buffer

2019-04-08 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8013. Resolution: Fixed > Avoid buffer underflow when reading a Struct from a partially corr

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

2019-04-08 Thread Jason Gustafson
gt; It would be also nice to separate the metrics MaxLag/TotalLag for > > > Reassignments. I think that will also require "original_replicas" (the > > > topic/partition's replicas just before reassignment when the AR > > > (Assigned Replicas) is set to Set(or

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-04-08 Thread Jason Gustafson
IP. +1 from me. > > Jun > > On Thu, Apr 4, 2019 at 2:26 PM Jason Gustafson wrote: > > > Hi Jun, > > > > I have updated the KIP to remove `replica.selection.policy` from the > > consumer configuration. Thanks for the suggestion. > > > > Best, > > J

[VOTE] KIP-360: Improve handling of unknown producer when using EOS

2019-04-08 Thread Jason Gustafson
Hi All, I'd like to start a vote on KIP-360: https://cwiki.apache.org/confluence/display/KAFKA/KIP-360%3A+Improve+handling+of+unknown+producer . +1 from me (duh) Thanks, Jason

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

2019-04-05 Thread Jason Gustafson
Hi Viktor, Thanks for writing this up. As far as questions about overlap with KIP-236, I agree it seems mostly orthogonal. I think KIP-236 may have had a larger initial scope, but now it focuses on cancellation and batching is left for future work. With that said, I think we may not actually need

[jira] [Resolved] (KAFKA-7893) Refactor ConsumerBounceTest.scala

2019-04-05 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7893. Resolution: Fixed > Refactor ConsumerBounceTest.sc

Re: [DISCUSS] KIP-360: Improve handling of unknown producer

2019-04-04 Thread Jason Gustafson
UCER_ID" error on > > 2.1.1 for a multitude of our internal topics, and I suspect that a proper > > fix is needed. > > > > Adam > > > > On Mon, Jan 7, 2019 at 7:42 PM Guozhang Wang wrote: > > > >> Thanks Jason. The proposed solution sound

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-04-04 Thread Jason Gustafson
Hi Jun, I have updated the KIP to remove `replica.selection.policy` from the consumer configuration. Thanks for the suggestion. Best, Jason On Wed, Mar 27, 2019 at 9:46 AM Jason Gustafson wrote: > @Jun > > Re; 200: It's a fair point that it is useful to minimize the client >

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-27 Thread Jason Gustafson
n can potentially be turned on with just a broker side change > > (assuming affinity can be determined w/o client rack.id). > > > > 201. I am wondering if PreferredReplica in the protocol should be named > > PreferredReadReplica since it's intended for reads? > >

[VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-25 Thread Jason Gustafson
Hi All, discussion on the KIP seems to have died down, so I'd like to go ahead and start a vote. Here is a link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Allow+consumers+to+fetch+from+closest+replica . +1 from me (duh) -Jason

[jira] [Created] (KAFKA-8152) Offline partition state not propagated by controller

2019-03-23 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-8152: -- Summary: Offline partition state not propagated by controller Key: KAFKA-8152 URL: https://issues.apache.org/jira/browse/KAFKA-8152 Project: Kafka Issue

Re: [VOTE] 2.2.0 RC2

2019-03-22 Thread Jason Gustafson
+1 - Ran the basic quickstart - Verified the release notes Just one thing I noticed is that we don't have the usual upgrade instructions. We can address this separately. Thanks, Jason On Thu, Mar 21, 2019 at 3:41 PM Ewen Cheslack-Postava wrote: > +1 > > -Ewen > > On Thu, Mar 21, 2019 at 10:

Re: [DISCUSS] KIP-392: Allow consumers to fetch from the closest replica

2019-03-20 Thread Jason Gustafson
ailable, since, as the name implies, the whole > point of stretching a cluster across availability zones is for increased > availability. But it sounds like nothing is changing in that regard, so > carry on :) > > Ryanne > > On Wed, Mar 20, 2019 at 6:41 PM Jason Gustafson >

Re: [DISCUSS] KIP-392: Allow consumers to fetch from the closest replica

2019-03-20 Thread Jason Gustafson
n would cause the client to only connect to the rack > specified by rack.id. If an application wants to do something fancy like > automatically failover between racks, the app can just change its rack.id, > perhaps after a short timeout. > > Ryanne > > On Tue, Mar 19, 2019 at 5:55 P

Re: [DISCUSS] KIP-392: Allow consumers to fetch from the closest replica

2019-03-19 Thread Jason Gustafson
t availability of the cluster when > a rack is unreachable. Is there a scenario where availability is improved > or impaired due to the proposed changes? > > Ryanne > > On Tue, Mar 19, 2019 at 4:32 PM Jason Gustafson > wrote: > > > Hi Jun, > > > > Yes, tha

Re: [DISCUSS] KIP-392: Allow consumers to fetch from the closest replica

2019-03-19 Thread Jason Gustafson
of the client. > > Node select(String rackId, PartitionInfo partitionInfo) > > Jun > > > On Mon, Mar 11, 2019 at 4:24 PM Jason Gustafson > wrote: > > > Hey Everyone, > > > > Apologies for the long delay. I am picking this work back up. > > > >

Re: [VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-03-11 Thread Jason Gustafson
+1 Thanks for all the work on this. My only minor comment is that `connect.protocol` probably should be `compatible` by default. The cost is low and it will save upgrade confusion. Best, Jason On Fri, Mar 8, 2019 at 10:37 AM Robert Yokota wrote: > Thanks for the great KIP Konstantine! > > +1 (n

Re: [DISCUSS] KIP-392: Allow consumers to fetch from the closest replica

2019-03-11 Thread Jason Gustafson
that change in KAFKA-2722. Jiangjie, could you > comment on whether it's reasonable to reduce the propagation delay now? > > Thanks, > > Jun > > On Wed, Jan 2, 2019 at 11:06 AM Jason Gustafson > wrote: > > > Hey Jun, > > > > Sorry for the lat

[jira] [Resolved] (KAFKA-8069) Committed offsets get cleaned up right after the coordinator loading them back from __consumer_offsets in broker with old inter-broker protocol version (< 2.2)

2019-03-08 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8069. Resolution: Fixed Fix Version/s: 2.1.2 2.0.2 > Committed offs

[jira] [Resolved] (KAFKA-7764) Authentication exceptions during consumer metadata updates may not get propagated

2019-03-07 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7764. Resolution: Fixed This was fixed by the patch for KAFKA-7831 which removes the metadata

[jira] [Resolved] (KAFKA-7831) Consumer SubscriptionState missing synchronization

2019-03-07 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7831. Resolution: Fixed Fix Version/s: 2.3.0 > Consumer SubscriptionState miss

[jira] [Resolved] (KAFKA-8061) Handle concurrent ProducerId reset and call to Sender thread shutdown

2019-03-07 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8061. Resolution: Fixed Fix Version/s: 2.2.1 2.1.2

[jira] [Resolved] (KAFKA-8002) Replica reassignment to new log dir may not complete if future and current replicas segment files have different base offsets

2019-03-04 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8002. Resolution: Fixed Fix Version/s: 2.1.2 2.0.2

[jira] [Resolved] (KAFKA-7959) Clear/delete epoch cache if old message format is in use

2019-02-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7959. Resolution: Fixed Fix Version/s: 2.0.2 > Clear/delete epoch cache if old mess

[jira] [Resolved] (KAFKA-7763) KafkaProducer with transactionId endless waits when network is disconnection for 10-20s

2019-02-21 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7763. Resolution: Fixed Fix Version/s: 2.3.0 > KafkaProducer with transactionId endl

[jira] [Created] (KAFKA-7961) Handle subscription changes with a rebalance in progress

2019-02-20 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-7961: -- Summary: Handle subscription changes with a rebalance in progress Key: KAFKA-7961 URL: https://issues.apache.org/jira/browse/KAFKA-7961 Project: Kafka

[jira] [Created] (KAFKA-7959) Clear/delete epoch cache if old message format is in use

2019-02-20 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-7959: -- Summary: Clear/delete epoch cache if old message format is in use Key: KAFKA-7959 URL: https://issues.apache.org/jira/browse/KAFKA-7959 Project: Kafka

[jira] [Resolved] (KAFKA-7884) Docs for message.format.version and log.message.format.version show invalid (corrupt?) "valid values"

2019-02-15 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7884. Resolution: Fixed > Docs for message.format.version and log.message.format.version s

[jira] [Resolved] (KAFKA-7886) Some partitions are fully truncated during recovery when log.message.format = 0.10.2 & inter.broker.protocol >= 0.11

2019-02-15 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7886. Resolution: Duplicate Resolving this as a duplicate of KAFKA-7897. This will be included

Re: [kafka-clients] [VOTE] 2.1.1 RC2

2019-02-14 Thread Jason Gustafson
Ran the quickstart against the 2.11 artifact and checked the release notes. For some reason, KAFKA-7897 is not included in the notes, though I definitely see it in the tagged version. The RC was probably created before the JIRA was resolved. I think we can regenerate without another RC, so +1 from

[jira] [Resolved] (KAFKA-7670) Fix flaky test - KafkaAdminClientTest.testUnreachableBootstrapServer

2019-02-14 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7670. Resolution: Fixed > Fix flaky test - KafkaAdminClientTest.testUnreachableBootstrapSer

[jira] [Resolved] (KAFKA-7401) Broker fails to start when recovering a segment from before the log start offset

2019-02-12 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7401. Resolution: Fixed Fix Version/s: 1.1.2 > Broker fails to start when recoverin

[jira] [Created] (KAFKA-7920) Do not permit zstd use until inter.broker.protocol.version is updated to 2.1

2019-02-11 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-7920: -- Summary: Do not permit zstd use until inter.broker.protocol.version is updated to 2.1 Key: KAFKA-7920 URL: https://issues.apache.org/jira/browse/KAFKA-7920

[jira] [Created] (KAFKA-7906) Improve failed leader election logging

2019-02-07 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-7906: -- Summary: Improve failed leader election logging Key: KAFKA-7906 URL: https://issues.apache.org/jira/browse/KAFKA-7906 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-7540) Transient failure: kafka.api.ConsumerBounceTest.testClose

2019-02-07 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7540. Resolution: Fixed Going to go on a limb and resolve this. Please reopen if the problem

[jira] [Created] (KAFKA-7897) Invalid use of epoch cache following message format downgrade

2019-02-04 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-7897: -- Summary: Invalid use of epoch cache following message format downgrade Key: KAFKA-7897 URL: https://issues.apache.org/jira/browse/KAFKA-7897 Project: Kafka

[jira] [Resolved] (KAFKA-7641) Add `consumer.group.max.size` to cap consumer metadata size on broker

2019-02-01 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7641. Resolution: Fixed Fix Version/s: 2.2.0 > Add `consumer.group.max.size` to

[jira] [Resolved] (KAFKA-7873) KafkaBasedLog's consumer should always seek to beginning when starting

2019-02-01 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7873. Resolution: Fixed Fix Version/s: 2.0.2 2.1.1

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-25 Thread Jason Gustafson
ut just defaulting to including all > versions would be better. That said, it took 3+ years to get to v1 and I'm > not sure we're aware of any blatant deficiencies in this version, so maybe > we won't ever get to v2 anyway... > > -Ewen > > On Fri, Jan

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-25 Thread Jason Gustafson
Hey Konstantine, Thanks for the reply. Just one response below: In 'compatible' mode, the worker sends both protocols to the broker > coordinator during the Join request. The field is already a list of > ProtocolMetadata. The broker, after gathering all the requests follows a > process of selecti

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-24 Thread Jason Gustafson
Hi Konstantine, Thanks for the clear, well-written proposal. I think my only doubt is about the `connect.protocol` configuration. A couple questions: 1. Do we need the `eager` option? It sounds like we're planning to deprecate it anyway. 2. Can you explain how `compatible` works? The basic idea

[jira] [Created] (KAFKA-7866) Duplicate offsets after transaction index append failure

2019-01-23 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-7866: -- Summary: Duplicate offsets after transaction index append failure Key: KAFKA-7866 URL: https://issues.apache.org/jira/browse/KAFKA-7866 Project: Kafka

[jira] [Resolved] (KAFKA-7738) Track partition leader epochs in client metadata

2019-01-17 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7738. Resolution: Fixed Fix Version/s: 2.2.0 > Track partition leader epochs in cli

[jira] [Created] (KAFKA-7831) Consumer SubscriptionState missing synchronization

2019-01-16 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-7831: -- Summary: Consumer SubscriptionState missing synchronization Key: KAFKA-7831 URL: https://issues.apache.org/jira/browse/KAFKA-7831 Project: Kafka Issue

[ANNOUNCE] New Committer: Vahid Hashemian

2019-01-15 Thread Jason Gustafson
Hi All, The PMC for Apache Kafka has invited Vahid Hashemian as a project committer and we are pleased to announce that he has accepted! Vahid has made numerous contributions to the Kafka community over the past few years. He has authored 13 KIPs with core improvements to the consumer and the too

[jira] [Resolved] (KAFKA-7824) Require member.id for initial join group request

2019-01-15 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7824. Resolution: Fixed Fix Version/s: 2.2.0 > Require member.id for initial join gr

[jira] [Resolved] (KAFKA-7391) Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)

2019-01-15 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7391. Resolution: Fixed Fix Version/s: 2.2.0 > Introduce close(Duration) to Producer

Re: [VOTE] KIP-389: Introduce a configurable consumer group size limit

2019-01-14 Thread Jason Gustafson
+1 Thanks for the KIP! -Jason On Mon, Jan 14, 2019 at 12:54 AM Andrew Schofield wrote: > +1 (non-binding) > > Looks like a good improvement. > > Andrew Schofield > IBM Event Streams > > On 11/01/2019, 17:33, "Boyang Chen" wrote: > > +1 (non-binding) > > Thanks for the great work! > >

Re: [VOTE] KIP-379: Multiple Consumer Group Management

2019-01-14 Thread Jason Gustafson
+1 Thanks for the KIP! -Jason On Mon, Jan 14, 2019 at 5:15 PM Gwen Shapira wrote: > I was also wondering about that. I think its for compatibility with > the existing output. > > We can have a separate KIP to add JSON output. > > On Mon, Jan 14, 2019 at 7:55 AM M. Manna wrote: > > > > Hi, > >

[jira] [Resolved] (KAFKA-7786) Fast update of leader epoch may stall partition fetching due to FENCED_LEADER_EPOCH

2019-01-09 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7786. Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 > Fast update

[jira] [Resolved] (KAFKA-6833) KafkaProducer throws "Invalid partition given with record" exception

2019-01-09 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6833. Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 > KafkaProdu

[jira] [Resolved] (KAFKA-7799) Fix flaky test RestServerTest.testCORSEnabled

2019-01-09 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7799. Resolution: Fixed > Fix flaky test RestServerTest.testCORSEnab

[jira] [Created] (KAFKA-7799) Fix flaky test RestServerTest.testCORSEnabled

2019-01-08 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-7799: -- Summary: Fix flaky test RestServerTest.testCORSEnabled Key: KAFKA-7799 URL: https://issues.apache.org/jira/browse/KAFKA-7799 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-7051) Improve the efficiency of the ReplicaManager when there are many partitions

2019-01-08 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7051. Resolution: Fixed > Improve the efficiency of the ReplicaManager when there are m

[jira] [Resolved] (KAFKA-7253) The connector type responded by worker is always null when creating connector

2019-01-08 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7253. Resolution: Fixed Fix Version/s: 2.0.2 2.1.1

Re: [DISCUSS] KIP-360: Improve handling of unknown producer

2019-01-07 Thread Jason Gustafson
stream-fail-to-produce-data-after-a-long-time/54029181#54029181 > > Guozhang > > > > On Thu, Nov 29, 2018 at 2:25 PM Jason Gustafson > wrote: > > > Hey Guozhang, > > > > To clarify, the broker does not actually use the ApiVersion API

Re: [VOTE] KIP-345: Introduce static membership protocol to reduce consumer rebalances

2019-01-04 Thread Jason Gustafson
system > > stability and we need to get things rolling right at the start of 2019. > > > > Thank you for your time! > > Boyang > > > > > > From: Jason Gustafson > > Sent: Tuesday, December 18, 2018 7:40 AM > > To:

[jira] [Resolved] (KAFKA-5503) Idempotent producer ignores shutdown while fetching ProducerId

2019-01-02 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-5503. Resolution: Fixed Fix Version/s: 2.0.2 2.1.1 > Idempot

<    5   6   7   8   9   10   11   12   13   14   >