[jira] [Created] (KAFKA-8596) Kafka topic pre-creation error message needs to be passed to application as an exception

ASHISH M VYAS created KAFKA-8596: Summary: Kafka topic pre-creation error message needs to be passed to application as an exception Key: KAFKA-8596 URL: https://issues.apache.org/jira/browse/KAFKA-8596

Build failed in Jenkins: kafka-trunk-jdk11 #657

See Changes: [manikumar.reddy] MINOR: add unit test for Utils.murmur2 (#5926) -- [...truncated 2.88 MB...] org.apache.kafka.connect.json.JsonConverterTest >

Build failed in Jenkins: kafka-trunk-jdk8 #3749

See Changes: [wangguoz] HOTFIX: removed extra footnote (#6996) -- [...truncated 2.53 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

[jira] [Created] (KAFKA-8597) Give access to the Dead Letter Queue APIs to Kafka Connect Developers

Andrea Santurbano created KAFKA-8597: Summary: Give access to the Dead Letter Queue APIs to Kafka Connect Developers Key: KAFKA-8597 URL: https://issues.apache.org/jira/browse/KAFKA-8597 Project:

Build failed in Jenkins: kafka-trunk-jdk8 #3750

See Changes: [manikumar.reddy] MINOR: add unit test for Utils.murmur2 (#5926) -- [...truncated 4.69 MB...]

[jira] [Resolved] (KAFKA-8390) Replace CreateDelegationToken request/response with automated protocol

[ https://issues.apache.org/jira/browse/KAFKA-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-8390. -- Resolution: Fixed Fix Version/s: 2.4.0 > Replace CreateDelegationToken request/response with

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

Hi All, I have added another improvement to this, which is to limit the parallel leader movements. I think I'll soon (maybe late this week or early next) start a vote on this too if there are no additional feedback. Thanks, Viktor On Mon, Apr 29, 2019 at 1:26 PM Viktor Somogyi-Vass wrote: >

Re: [DISCUSS] KIP-477: Add PATCH method for connector config in Connect REST API

Hi, Since Kafka 2.3 has just been release and more people may have time to look at this now, I'd like to bump this discussion. Thanks. Ivan On Thu, 13 Jun 2019 at 17:20, Ivan Yurchenko wrote: > Hello, > > I'd like to start the discussion of KIP-477: Add PATCH method for > connector config in

[jira] [Created] (KAFKA-8601) Producer Improvement: Sticky Partitioner

Justine Olshan created KAFKA-8601: - Summary: Producer Improvement: Sticky Partitioner Key: KAFKA-8601 URL: https://issues.apache.org/jira/browse/KAFKA-8601 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

Hi Boyang, 1. One advantage of retry against on-hold is that it will not tie-up a handler thread (of course the latter could do the same but that involves using a purgatory which is more complicated), and also it is less likely to violate request timeout. So I think there are some rationales to

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

> > The question I have is that whether we should name the interface > `GroupAssignment` instead, so > that Connect later could also extend on the same interface, just to echo > Guozhang's point here, Are you referring to the API used for initTransactions? There would be no reason to use a more

[jira] [Created] (KAFKA-8598) Replace RenewDelegationToken request/response with automated protocol

Mickael Maison created KAFKA-8598: - Summary: Replace RenewDelegationToken request/response with automated protocol Key: KAFKA-8598 URL: https://issues.apache.org/jira/browse/KAFKA-8598 Project: Kafka

Jenkins build is back to normal : kafka-trunk-jdk11 #658

See

[jira] [Created] (KAFKA-8600) Replace DescribeDelegationToken request/response with automated protocol

Mickael Maison created KAFKA-8600: - Summary: Replace DescribeDelegationToken request/response with automated protocol Key: KAFKA-8600 URL: https://issues.apache.org/jira/browse/KAFKA-8600 Project:

[jira] [Created] (KAFKA-8599) Replace ExpireDelegationToken request/response with automated protocol

Mickael Maison created KAFKA-8599: - Summary: Replace ExpireDelegationToken request/response with automated protocol Key: KAFKA-8599 URL: https://issues.apache.org/jira/browse/KAFKA-8599 Project:

Build failed in Jenkins: kafka-trunk-jdk8 #3751

See Changes: [manikumar.reddy] KAFKA-8390: Use automatic RPC generation in CreateDelegationToken -- [...truncated 2.55 MB...] kafka.security.auth.PermissionTypeTest >

[ANNOUNCE] Apache Kafka 2.3.0

The Apache Kafka community is pleased to announce the release for Apache Kafka 2.3.0. This release includes several new features, including: - There have been several improvements to the Kafka Connect REST API. - Kafka Connect now supports incremental cooperative rebalancing. - Kafka Streams

Contribution access to Kafka Confluence page

Hi, Could I get access to create KIPs on the Kafka Confluence page? Email: av...@confleunt.io UserID: avela Thanks, Anastasia

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

> > I think co-locating does have some merits here, i.e. letting the > ConsumerCoordinator which has the source-of-truth of assignment to act as > the TxnCoordinator as well; but I agree there's also some cons of coupling > them together. I'm still a bit inclining towards colocation but if there >

Re: [ANNOUNCE] Apache Kafka 2.3.0

Thanks to everyone who reviewed the Apache blog post about 2.3. It's live now at https://blogs.apache.org/kafka/date/20190624 Plus, Tim Berglund made a video about what's new in this release. https://www.youtube.com/watch?v=sNqwJT2WguQ Finally, check out Stéphane Maarek's video about 2.3

Re: [DISCUSS] KIP-480 : Sticky Partitioner

Thanks Justine! You might want to update the "JIRA" link on the KIP so that it links to this. cheers, Colin On Tue, Jun 25, 2019, at 11:59, Justine Olshan wrote: > Thank you for looking at my KIP! > > I will get to work on these changes. > > In addition, here is the JIRA ticket: >

[jira] [Created] (KAFKA-8602) StreamThread Dies Because Restore Consumer is not Subscribed to Any Topic

Bruno Cadonna created KAFKA-8602: Summary: StreamThread Dies Because Restore Consumer is not Subscribed to Any Topic Key: KAFKA-8602 URL: https://issues.apache.org/jira/browse/KAFKA-8602 Project:

Re: [RESULT] [VOTE] 2.3.0 RC3

Thanks for the hard work Colin, and everyone else who helped get this out! On Mon, Jun 24, 2019 at 1:38 PM Colin McCabe wrote: > Hi all, > > This vote passes with 6 +1 votes (3 of which are binding) and no 0 or -1 > votes. Thanks to everyone who voted. > > +1 votes > PMC Members: > * Ismael

Re: [DISCUSS] KIP-480 : Sticky Partitioner

Hi Justine, The KIP discusses adding a new method to the partitioner interface. > default public Integer onNewBatch(String topic, Cluster cluster) { ... } However, this new method doesn't give the partitioner access to the key and value of the message. While this works for the case described

Re: [DISCUSS] KIP-480 : Sticky Partitioner

Thank you for looking at my KIP! I will get to work on these changes. In addition, here is the JIRA ticket: https://issues.apache.org/jira/browse/KAFKA-8601 Thanks again, Justine On Tue, Jun 25, 2019 at 11:55 AM Colin McCabe wrote: > Hi Justine, > > The KIP discusses adding a new method to

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

Hi Viktor, Now that the 2.3 release is over, we're going to be turning our attention back to working on KIP-455, which provides an API for partition reassignment, and also solves the incremental reassignment problem. Sorry about the pause, but I had to focus on the stuff that was going into

Re: [DISCUSS] KIP-480 : Sticky Partitioner

I also just noticed that if we want to use this method on the keyed record case, I will need to move the method outside of the sticky (no key, no set partition) check. Not a big problem, but something to keep in mind. Perhaps, we should encapsulate the sticky vs. not behavior inside the method?

Re: [DISCUSS] KIP-480 : Sticky Partitioner

Well, this is a generic partitioner method, so it shouldn't dictate any particular behavior. Colin On Tue, Jun 25, 2019, at 12:04, Justine Olshan wrote: > I also just noticed that if we want to use this method on the keyed record > case, I will need to move the method outside of the sticky (no

Re: [DISCUSS] KIP-480 : Sticky Partitioner

I came up with a good solution for this and will push the commit soon. The repartition will be called only when a partition is not manually sent. On Tue, Jun 25, 2019 at 1:39 PM Colin McCabe wrote: > Well, this is a generic partitioner method, so it shouldn't dictate any > particular behavior.

Re: [DISCUSS] KIP-480 : Sticky Partitioner

Also apologies on the late link to the jira, but apparently https links do not work and it kept defaulting to an image on my desktop even when it looked like I put the correct link in. Weird... On Tue, Jun 25, 2019 at 1:41 PM Justine Olshan wrote: > I came up with a good solution for this and

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

Thank you Guozhang, some of my understandings are inline below. On Tue, Jun 25, 2019 at 11:05 AM Jason Gustafson wrote: > > > > I think co-locating does have some merits here, i.e. letting the > > ConsumerCoordinator which has the source-of-truth of assignment to act as > > the TxnCoordinator

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

Here it is: https://youtu.be/YutjYKSGd64 Thanks in advance!! On Tue., 25 Jun. 2019, 12:39 am Colin McCabe, wrote: > Hi Stephane, > > Sounds interesting! Do you have a link to the video you made for 2.3? > > best, > Colin > > > On Sun, Jun 23, 2019, at 15:10, Stephane Maarek wrote: > > The

Re: [DISCUSS] KIP-480 : Sticky Partitioner

No worries. Thanks for fixing it! C. On Tue, Jun 25, 2019, at 13:47, Justine Olshan wrote: > Also apologies on the late link to the jira, but apparently https links do > not work and it kept defaulting to an image on my desktop even when it > looked like I put the correct link in. Weird... > >

Re: Contribution access to Kafka Confluence page

Hi Anastasia, You're all set now! -Bill On Tue, Jun 25, 2019 at 1:50 PM Anastasia Vela wrote: > Hi, > > Could I get access to create KIPs on the Kafka Confluence page? > Email: av...@confleunt.io > UserID: avela > > Thanks, > Anastasia >

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

Actually, after a second thought, I think it actually makes sense to support auto upgrade through admin client to help use get api version from broker. A draft KIP is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-483%3A++Add+Broker+Version+API+in+Admin+Client Boyang On Tue, Jun 25,

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

kafka.api.ApiVersion is an internal class, not suitable to exposing through AdminClient. That class is not even accessible without having the broker jars on your CLASSPATH. Another question is, what is the groupId parameter doing in the call? The API versions are the same no matter what

Re: [VOTE] KIP-476: Add Java AdminClient interface

+1 (binding). C. On Mon, Jun 24, 2019, at 08:10, Andy Coates wrote: > Hi all, > > KIP updated: > - No deprecation > - Factory method back onto Admin interface > > I'd like to kick off another round of voting please. > > Thanks, > > Andy > > On Mon, 24 Jun 2019 at 16:03, Andy Coates wrote:

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

Thank you for the context Colin. The groupId was indeed a copy-paste error. Our use case here for 447 is (Quoted from Guozhang): ''' I think if we can do something else to avoid this config though, for example we can use the embedded AdminClient to send the APIVersion request upon starting up, and

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

For reference, we have BrokerApiVersionCommand already as a public interface. We have a bit of tech debt at the moment because it uses a custom AdminClient. It would be nice to clean that up. In general, I think it is reasonable to expose from AdminClient. It can be used by management tools to

Re: [VOTE] KIP-476: Add Java AdminClient interface

+1 (non-binding) On Wed, Jun 26, 2019 at 8:37 AM Satish Duggana wrote: > > +1 Matthias/Andy. > IMHO, interface is about the contract, it should not have/expose any > implementation. I am fine with either way as it is more of taste or > preference. > > Agree with Ismael/Colin/Ryanne on not

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

On Wed, Jun 19, 2019, at 03:36, Stanislav Kozlovski wrote: > Hey there Colin, > > Thanks for the work on this KIP. It is a much-needed improvement and I'm > excited to see it. Sorry for coming in so late to the discussion, I have > one question to better understand the change and a small

[DISCUSS] KIP-484: Expose metrics for group and transaction metadata loading duration

Hi all, I'd like to discuss KIP-484: https://cwiki.apache.org/confluence/display/KAFKA/KIP-484%3A+Expose+metrics+for+group+and+transaction+metadata+loading+duration Let me know what you think! Thanks, Anastasia

Re: [VOTE] KIP-476: Add Java AdminClient interface

+1 Matthias/Andy. IMHO, interface is about the contract, it should not have/expose any implementation. I am fine with either way as it is more of taste or preference. Agree with Ismael/Colin/Ryanne on not deprecating for good reasons. On Mon, Jun 24, 2019 at 8:33 PM Andy Coates wrote: > > I

[jira] [Created] (KAFKA-8603) Document upgrade path

Sophie Blee-Goldman created KAFKA-8603: -- Summary: Document upgrade path Key: KAFKA-8603 URL: https://issues.apache.org/jira/browse/KAFKA-8603 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

Hi Colin, Took another pass on the KIP. Looks good overall. A few questions below: 1. I wasn't clear why `currentReplicas` is an optional field. Wouldn't we always have a current set of replicas? 2. Seems the only option is to list all active partition reassignments? I think we have tended to

Build failed in Jenkins: kafka-trunk-jdk8 #3752

See Changes: [github] MINOR: Fix flaky test case for compact/delete topics (#6975) -- [...truncated 4.82 MB...]