Re: Way to check if custom SMT has been added to the classpath or even if it i working.

Hi, connector-plugins endpoint does not list the transformations classes currently. However if you are using the latest Kafka version ( >= 0.11.0) one way to see if your transform is discovered during startup in the given classpath is to notice whether a log message such as the one below is

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Congratulations Ismael! On Fri, Jul 7, 2017 at 8:26 AM Eno Thereska wrote: > Congrats! > > Eno > > On 7 Jul 2017, at 16:13, Kamal C wrote: > > > > Congratulations Ismael ! > > > > On 06-Jul-2017 14:11, "Ismael Juma"

Re: [ANNOUNCE] New Kafka PMC member Jason Gustafson

Congratulations Jason! On Wed, Jul 12, 2017 at 10:54 AM, James Cheng wrote: > Congrats Jason! > > -James > > > On Jul 11, 2017, at 10:32 PM, Guozhang Wang wrote: > > > > Hi Everyone, > > > > Jason Gustafson has been very active in contributing to the

[DISCUSS] KIP-145: Classloading Isolation in Connect

Hi everyone, we aim to address dependency conflicts in Kafka Connect soon by applying class loading isolation. Feel free to take a look at KIP-145 here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-145+Classloading+Isolation+in+Connect which describes minimal required changes in the

[DISCUSS] KIP-146: Classloading Isolation in Connect

* Because of KIP number collision, please disregard my previous KIP announcement and use this thread for discussion instead * Hi everyone, we aim to address dependency conflicts in Kafka Connect soon by applying class loading isolation. Feel free to take a look at KIP-146 here:

Re: [DISCUSS] KIP-145: Classloading Isolation in Connect

at 10:20 AM, Colin McCabe <cmcc...@apache.org> wrote: > +1 (non-binding) > > This would be a nice improvement. > > C. > > On Fri, Apr 28, 2017, at 11:03, Konstantine Karantasis wrote: > > Hi everyone, > > > > we aim to address dependency conflicts in Kaf

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

verage, have the fewest > extra dependencies. Connectors have the most, followed by Converters, then > Transformations. But I think Konstantine has mentioned these in the KIP -- > if it could be clearer in the proposed changes, perhaps you could suggest > where to add details? >

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

Thanks Ewen. I'm replying inline as well. On Tue, May 2, 2017 at 11:24 AM, Ewen Cheslack-Postava <e...@confluent.io> wrote: > Thanks for the KIP. > > A few responses inline, followed by additional comments. > > On Mon, May 1, 2017 at 9:50 PM, Konstantine Karantasis < &g

Re: [VOTE] KIP-146: Isolation of dependencies and classes in Kafka Connect (restarted voting thread)

> > Gwen, `plugin.path` sounds good to me. > > In any case, I will leave it to you all to decide. :) > > Ismael > > On Wed, May 10, 2017 at 8:11 PM, Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > Thank you Ismael for your vote as well as

Re: [VOTE] KIP-146: Isolation of dependencies and classes in Kafka Connect (restarted voting thread)

PM, Konstantine Karantasis < konstant...@confluent.io> wrote: > The KIP description has been updated to reflect the use of the term > plugin.path instead. > > -Konstantine > > > > > On Wed, May 10, 2017 at 2:10 PM, Ismael Juma <ism...@juma.me.uk> wrot

Re: [VOTE] KIP-146: Isolation of dependencies and classes in Kafka Connect (restarted voting thread)

ears in 2 different modules, it is an error). What we are proposing > is > > > quite different and perhaps it may make sense to use a different name > to > > > avoid confusion. > > > > > > Ismael > > > > > > [1] https://www.infoq.com/articl

Re: [ANNOUNCE] New committer: Damian Guy

Congrats! On Mon, Jun 12, 2017 at 12:48 PM, Michael Noll wrote: > Congratulations, Damian! > > On Mon, Jun 12, 2017 at 9:07 AM, Molnár Bálint > wrote: > > > Congrats, Damien! > > > > 2017-06-12 8:44 GMT+02:00 Rajini Sivaram

Re: [DISCUSS] KIP 151 - Expose Connector type in REST API

Nice. Thanks! -Konstantine On Sat, May 6, 2017 at 10:43 PM, dan <dan.norw...@gmail.com> wrote: > thanks for the feedback, it all sounds good. i have made the changes to the > pr and the kip. > > dan > > On Fri, May 5, 2017 at 9:29 AM, Konstantine Karantasis < > k

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

the module.path set and therefore classpath > isolation > > will simply not work by default? > > > > In that case, why don't we simply use the existence of non-empty > > module.path as an indicator of whether isolation should work or not? seem > > simpler and intuitive to m

Re: [DISCUSS] KIP 151 - Expose Connector type in REST API

Thank you for the KIP. It's a nice improvement. Two small suggestions: 1) Let's not use all caps to describe the type of the connector. "Source" and "Sink" seem more appropriate (but even all lower case would be better). 2) It's been discussed in other contexts recently, but I believe finally

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

, Konstantine Karantasis < konstant...@confluent.io> wrote: > Dear all, > > KIP-146 has been updated to incorporate the changes that were suggested > during the discussion so far. You may find the updated version here: > > https://cwiki.apache.org/confluence/display/KAFKA/K

Re: [VOTE] KIP-146: Classloading Isolation in Connect

ngg...@gmail.com> wrote: > +1 > > On Mon, May 8, 2017 at 11:04 AM, Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > Hi all, > > > > Given that the comments during the discussion seem to have been > addressed, > > I'm pleased to br

[VOTE] KIP-146: Isolation of dependencies and classes in Kafka Connect (restarted voting thread)

** Restarting the voting thread here, with a different title to avoid collapsing this thread's messages with the discussion thread's messages in mail clients. Apologies for the inconvenience. ** Hi all, Given that the comments during the discussion seem to have been addressed, I'm pleased to

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

e, > what do we do with the built-in ones? > > On Sat, Apr 29, 2017 at 9:16 AM, Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > * Because of KIP number collision, please disregard my previous KIP > > announcement and use this thread for discussion i

[VOTE] KIP-146: Classloading Isolation in Connect

Hi all, Given that the comments during the discussion seem to have been addressed, I'm pleased to bring KIP-146: Classloading Isolation in Connect https://cwiki.apache.org/confluence/display/KAFKA/KIP-146+-+Classloading+Isolation+in+Connect up for voting. Again, this KIP aims to bring the

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

anilla > lightweight connect. Anyway, discussions for later. > > > > On 4/5/17, 4:17 am, "Konstantine Karantasis" <konstant...@confluent.io> > wrote: > > Thank you Stephane, > > your comments bring interesting and useful subjects to the discussion.

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

o the same API. That would allow a clear audit of where the > code is being loaded. :-) > > Mathieu > > > On Sat, Apr 29, 2017 at 10:16 AM, Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > * Because of KIP number collision, please disregard my previo

Re: [VOTE] KIP-154 Add Kafka Connect configuration properties for creating internal topics

+1 (non binding) On Mon, May 8, 2017 at 1:33 PM, Stephane Maarek < steph...@simplemachines.com.au> wrote: > +1 (non binding) > > > > On 9/5/17, 5:51 am, "Randall Hauch" wrote: > > Hi, everyone. > > Given the simple and non-controversial nature of the KIP, I would like

Re: [DISCUSS] KIP-154 Add Kafka Connect configuration properties for creating internal topics

Thanks a lot for the KIP Randall. This improvement should simplify both regular deployments and testing! A minor comment. Maybe it would be nice to add a note about why there's no need for the property: config.storage.partitions I'm mentioning this for the sake of completeness, in case someone

Re: [VOTE] KIP-151: Expose Connector type in REST API (first attempt :)

+1 (non-binding) On Mon, May 8, 2017 at 3:39 PM, dan wrote: > i'd like to begin voting on > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 151+Expose+Connector+type+in+REST+API > > discussion should remain on >

Re: [VOTE] KIP-238: Expose Kafka cluster ID in Connect REST API

+1 Nice KIP. Thanks! - Konstantine On Sat, Dec 16, 2017 at 6:19 AM, Gwen Shapira wrote: > +1 (binding). Thanks! > On Fri, Dec 15, 2017 at 10:55 AM Ted Yu wrote: > > > +1 > > > > On Fri, Dec 15, 2017 at 10:49 AM, Ewen Cheslack-Postava < >

Re: [VOTE] KIP-215: Add topic regex support for Connect sinks

Nice addition! +1 (non-binding) Konstantine On Fri, Nov 3, 2017 at 4:52 PM, Jeff Klukas wrote: > So sorry for skirting the process there. I wasn't aware of the 72 hour > window and I don't see that mentioned in in >

Re: [DISCUSS] KIP-298: Error Handling in Connect

Thanks Arjun for your quick response. Adding an example for the failure log improves things, but I think it'd be better to also add the schema definition of these Json entries. And I'll agree with Magesh that this format should be public API. Also, does the current example have a copy/paste

Re: [DISCUSS] KIP-298: Error Handling in Connect

Arjun, it's exciting to see a KIP around better handling of bad-data and errors in Kafka Connect. I have only a few comments below, which I hope you'll find helpful. 1. I think it'd be useful to describe a bit more in detail how someone can extract the raw data of a Kafka record that failed to

Re: [VOTE] KIP-285: Connect Rest Extension Plugin

+1 (non-binding) - Konstantine On Thu, May 17, 2018 at 10:05 PM, Ewen Cheslack-Postava wrote: > +1 (binding) > > Thanks, > Ewen > > On Thu, May 17, 2018 at 12:16 PM Ted Yu wrote: > > > +1 > > Original message From: Gwen Shapira

Re: [DISCUSS] KIP-298: Error Handling in Connect

fused. > > > > -Ewen > > > > > > On Thu, May 17, 2018 at 8:53 PM Arjun Satish <arjun.sat...@gmail.com> > > wrote: > > > > > Konstantine, > > > > > > Thanks for pointing out the typos. Fixed them. > > > &

Re: [VOTE] KIP-298: Error Handling in Connect kafka

+1 (non-binding) - Konstantine On Thu, May 17, 2018 at 12:58 AM, Arjun Satish wrote: > All, > > Many thanks for all the feedback on KIP-298. Highly appreciate the time and > effort you all put into it. > > I've updated the KIP accordingly, and would like to start to

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

gt; > Sounds reasonable to me too. > > > > Regards, > > > > Rajini > > > > On Mon, Jun 11, 2018 at 7:55 PM, Robert Yokota > wrote: > > > > > Hi Konstantine, > > > > > > Sounds reasonable! > > > > > > Tha

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

Hi everyone, after fixing an issue with a regular expression in Connect's class loading isolation of the new component type ConfigProvider here: https://github.com/apache/kafka/pull/5177 I noticed that the new interface ConfigProvider, along with its first implementation FileConfigProvider, have

Re: [ANNOUNCE] New committer: Matthias J. Sax

Matthias! Congratulations! Konstantine On Mon, Jan 15, 2018 at 4:18 AM, Edoardo Comar wrote: > Congratulations Matthias ! > -- > > Edoardo Comar > > IBM Message Hub > > IBM UK Ltd, Hursley Park, SO21 2JN > > > > From: UMESH

Re: [ANNOUNCE] New Kafka PMC Member: Rajini Sivaram

Congrats Rajini! -Konstantine On Wed, Jan 17, 2018 at 2:18 PM, Becket Qin wrote: > Congratulations, Rajini! > > On Wed, Jan 17, 2018 at 1:52 PM, Ismael Juma wrote: > > > Congratulations Rajini! > > > > On 17 Jan 2018 10:49 am, "Gwen Shapira"

Re: [VOTE] KIP-212: Enforce set of legal characters for connector names

Thanks for the KIP! +1 (non-binding) Konstantine On Tue, Jan 23, 2018 at 10:48 AM, Ewen Cheslack-Postava wrote: > +1 (binding) > > On Tue, Jan 23, 2018 at 7:28 AM, Matt Farmer wrote: > > > +1 from me (non-binding) =) > > > > > On Jan 22, 2018, at 7:35 PM,

Re: [VOTE] KIP-145: Expose Record Headers in Kafka Connect

Great addition! +1 (non-binding) Konstantine On Sun, Jan 21, 2018 at 7:26 PM, Ewen Cheslack-Postava wrote: > +1 (binding) > > Thanks for the work on this -- not a small upgrade to the Connect APIs! > > -Ewen > > On Fri, Jan 19, 2018 at 3:37 PM, Randall Hauch

Re: [DISCUSS] KIP-381 Connect: Tell about records that had their offsets flushed in callback

This is a significant improvement to the semantics of source connectors. I'm expecting that it will facilitate source connector implementations and even enrich the application uses cases that we see. I only have a few minor suggestions at the moment. I believe that Acked is a common abbreviation

Re: [DISCUSS] KIP-381 Connect: Tell about records that had their offsets flushed in callback

this is about. Some > times you only really understand what something is about, when you are > forced to write about it (at least that is my excuse ). > > Regards, Per Steffensen > > On 16/10/2018 05.57, Konstantine Karantasis wrote: > > This is a significant improvement to the sem

Re: [EXTERNAL] Incremental Cooperative Rebalancing

scenariostoconsider > [4] > > https://cwiki.apache.org/confluence/display/KAFKA/Incremental+Cooperative+Rebalancing+for+Streams > > > On Thu, Oct 4, 2018 at 12:16 PM, McCaig, Rhys > wrote: > > > This is fantastic. Im really excited to see the work on this. >

Incremental Cooperative Rebalancing

Hey everyone, I'd like to bring to your attention a general design document that was just published in Apache Kafka's wiki space: https://cwiki.apache.org/confluence/display/KAFKA/Incremental+Cooperative+Rebalancing%3A+Support+and+Policies It deals with the subject of Rebalancing of groups in

Re: [ANNOUNCE] New committer: Colin McCabe

Well deserved! Congratulations Colin. -Konstantine On Wed, Sep 26, 2018 at 4:57 AM Srinivas Reddy wrote: > Congratulations Colin  > > - > Srinivas > > - Typed on tiny keys. pls ignore typos.{mobile app} > > On Tue 25 Sep, 2018, 16:39 Ismael Juma, wrote: > > > Hi all, > > > > The PMC for

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

Hi Rhys, thanks for the proposal and apologies for the late feedback. Utilizing Connect to mirror Kafka topics is definitely a plausible proposal for a very useful use case. However, I don't think the apache/kafka repository is the right place to host such a Connector. Currently, no

Re: [VOTE] KIP-310: Add a Kafka Source Connector to Kafka Connect

Hi Rhys, I just replied in the discussion thread for KIP-310 with my concerns. Thanks for submitting this proposal. -Konstantine On Tue, Sep 11, 2018 at 12:53 AM McCaig, Rhys wrote: > Hi All, > > Bumping this again. > Can I get feedback from some binding vote holders on this KIP? I think its

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

t there > should be support in the project - and MirrorMaker is, as you mention, > showing its age. > > Cheers, > Rhys > > > > > > On Sep 26, 2018, at 10:42 AM, Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > > Hi Rhys, > > &

Re: [DISCUSS] KIP-411: Add option to make Kafka Connect task client ID values unique

Hi Paul. I second Ewen and I intended to give similar feedback: 1) Can we avoid a config altogether? 2) If we prefer to add a config anyways, can we use a set of allowed values instead of a boolean, even if initially these values are only two? As the discussion on Jira highlights, there is a

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

ault value == "eager" until this policy is deprecated. Also, let me note that I have amended the examples and have added the Connect specific detail I promised above. Let me know additional changes are required. Cheers, Konstantine > cc Mayuresh on this thread. > >

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

nnect > > protocol is enabled (version 1 or higher). > > > > 4. For the compatibility change, I'm wondering whether we could just use > 2 > > connect protocols instead of 3. Because the user knows when all the > workers > > all upgraded to version 1, we could just use `compatible` for the first > &g

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

f the optimization as to not > send the assignment that the worker already knows about? > > I enjoyed reading the rebalancing examples. As a small readability > improvement, could I suggest we clarify which Worker (W1,W2,W3) is the > leader in the "Initial group and assignment&quo

Re: [DISCUSS] KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

Thanks for considering removal Alex. I totally agree with your assessment. Still, I'd be in favor of making KIP-404 a small KIP that describes that this option is now being disabled. (If I'm not mistaken, one place I've noticed this feature being used is in Connect's unit tests for the rest

Re: [DISCUSS] KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

Hi Alex, thanks for working on this and identifying the need to address this issue. I see under Rejected Alternatives that the option to disable this feature has been considered already. However, I'd like to return a bit to this option and highlight the following: * Exposing this request was

Re: [DISCUSS] KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

be > introduced in the KIP, > it was never used anywhere else than in unit tests, as a part of PR for the > KIP. > So no public interface changes are needed if we proceed without exposing > the option. > WDYT? > > Regards, Alex. > > On Mon, Dec 17, 2018 a

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Hi Boyang. Thanks for preparing this KIP! It is making good progress and will be a great improvement for stateful Kafka applications. Apologies for my late reply, I was away for a while. Lots of great comments so far, so I'll probably second most of them in what I suggest below at this point.

[DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Hi all, I just published KIP-415: Incremental Cooperative Rebalancing in Kafka Connect on the wiki here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals This is the first KIP to suggest an implementation of incremental and cooperative rebalancing in the context of

Re: [DISCUSS] KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

a PR and backporting -- the WADL operation was never documented nor > > described in any of the prior KIPs, and it seems like an implementation > > detail that leaked out accidentally. > > > > Randall > > > > On Tue, Dec 18, 2018 at 4:15 PM Konstantine Karanta

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

case there's an objection to the more reactive approach that I'm missing right now. Thanks for your comments Jason. Glad you are finding this KIP draft a good start. Konstantine > Thanks, > Jason > > > On Tue, Jan 22, 2019 at 6:26 PM Konstantine Karantasis < > konstant...@confl

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

ith incremental cooperative rebalancing in the future to improve assignment of resources (e.g. topic-partitions or connectors and tasks). Therefore, even if they are combined for the same group at some point, I don't foresee an immediate need to deprecate one over the other. Cheers, Konstantine

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

ds, is it safe to allow > > > tasks to continue executing when a rebalance begins even if the > > coordinator > > > ultimate decides to use the "eager" protocol? If it is, then I think > the > > > value of the `connect.protocol` configuration is diminish

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

comments so far! Konstantine On Tue, Mar 5, 2019 at 11:20 AM Konstantine Karantasis < konstant...@confluent.io> wrote: > > Hi Guozhang, > thanks for your detailed comments! > > I thought it'd be a good idea to have some code supporting my replies. The > PR for KIP-

[VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

I'd like to open the vote on KIP-415: Incremental Cooperative Rebalancing in Kafka Connect https://cwiki.apache.org/confluence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect a proposal that will allow Kafka Connect to scale significantly the number of connectors

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

has been updated to use the existing Kafka protocol types. Thanks! Konstantine > > On Wed, Feb 6, 2019 at 9:58 PM Boyang Chen wrote: > > > Thanks Konstantine for the great summary! +1 for having a separate KIP > > discussing the trade-offs for using a new serialization f

Re: [VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

ence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect > > > and it lgtm. > > > > > > I'm +1 on the KIP. > > > > > > > > > Guozhang > > > > > > > > > On Thu, Mar 7, 2019 at 2:35 PM Konstantine Karantasis

Re: [VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

hanks for the great KIP Konstantine! > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > Robert > > > > > > > > > > > > On Thu, Mar 7, 2019 at 2:56 PM Guozhang Wang > > > > > wrot

Re: [ANNOUNCE] New Committer: Bill Bejeck

Congrats Bill! -Konstantine On Wed, Feb 13, 2019 at 8:42 PM Srinivas Reddy wrote: > Congratulations Bill  > > Well deserved!! > > - > Srinivas > > - Typed on tiny keys. pls ignore typos.{mobile app} > > On Thu, 14 Feb, 2019, 11:21 Ismael Juma > > Congratulations Bill! > > > > On Wed, Feb

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

nks for the KIP Konstantine. Quick question: introducing a new > serialization format (ie flatbuffers) has major implications. Have we > considered json? If so, why did we reject it? > > Ismael > > On Fri, Jan 11, 2019, 3:44 PM Konstantine Karantasis < > konstant...@confluent.io wro

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

the changes of this new rebalancing protocol to Kafka clients, beginning with Kafka Connect, in a more applicable and less disruptive way. I'll change the schema descriptions by end of day. Looking forward to your next comments! Konstantine On Mon, Jan 28, 2019 at 5:22 PM Konstantine Karantasis

Re: [DISCUSS] KIP-449: Add connector contexts to Connect worker logs

urable: you can control it via your own logging configuration file > (rather than optionally including it in the "worker" scope on some of the > log messages). Thoughts? What would the "%{worker.id}" MDC value be set to > if "client.id" is not set? > > Final

Re: [DISCUSS] KIP-449: Add connector contexts to Connect worker logs

Thanks for the KIP Randall. It might not be obvious right away, but this is a great improvement when running Connect with multiple connectors or when debugging Connect, for instance in integration tests or system tests. KIP looks good to me overall, I have just a few comments below: 1. In the

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Thanks Cyrus, this is a nice and straightforward addition. I'm +1 too, but I'd like to return with a question here as well regarding whether the unassigned tasks will be taken into account. Especially after KIP-415 we might start seeing this status for specific periods of time. Therefore, I

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

Thanks Colin. Great initiative! Here's a small correction (between **) for KIP-415 with a small suggestion as well (between _ _): In Kafka Connect, worker tasks are distributed among the available worker nodes. When a connector is reconfigured or a new connector is deployed _as well as when a

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

ts -- blog and video -- for every > release > > > would be helpful as different people have different preferences. > > > > > > Ron > > > > > > On Tue, Jun 18, 2019 at 8:20 PM Colin McCabe > wrote: > > > > > >> Thanks, Kons

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

Hi Chris, Thanks for considering my suggestion regarding default implementations for the new methods. However, given that these implementations don't seem to have sane defaults and throw UnsupportedOperationException, I think we'll be better without defaults. Seems that a compile time error is

Re: [DISCUSS] KIP-440: Extend Connect Converter to support headers

ll > > > > yet > > > > > be serialized by the `HeaderConverter`. Alternatively, the common ` > > > > > org.apache.kafka.common.header.Headers` would correspond to the raw > > > > header > > > > > pairs from the underlying

Re: [VOT] KIP-449: Add connector contexts to Connect worker logs (vote thread)

Great improvement for multi-tenancy. Thanks Randall! +1 (non-binding) Konstantine On Tue, Apr 30, 2019 at 9:18 PM Chris Egerton wrote: > +1 (non-binding) > > Really looking forward to this. Thanks, Randall! > > On Tue, Apr 30, 2019, 20:47 Magesh Nandakumar > wrote: > > > This will make

Re: [VOTE] KIP-437: Custom replacement for MaskField SMT

I think is is a useful improvement proposal. Thanks Valeria! I'm +1 (non-binding) on this KIP Konstantine On Mon, Apr 15, 2019 at 2:04 AM Valeria Vasylieva < valeria.vasyli...@gmail.com> wrote: > Hi all, > > Since there are no more objections/proposals I would like to start the > vote on

Re: [VOTE] KIP-454: Expansion of the ConnectClusterState interface

/KAFKA/KIP-454%3A+Expansion+of+the+ConnectClusterState+interface > > > > > > The discussion thread can be found at > > > https://www.mail-archive.com/dev@kafka.apache.org/msg96911.html > > > > > > Thanks to Konstantine Karantasis and Magesh Nandakumar for their > > thoughtful > > > feedback! > > > > > > Cheers, > > > > > > Chris > > > > > >

Re: [DISCUSS] KIP-465: Add Consolidated Connector Endpoint to Connect REST API

This is a nice improvement, both from an operational standpoint as well as for testing purposes, as we scale the number of connectors that a connect cluster can run. LGTM, thanks for the KIP Dan! On Fri, May 3, 2019 at 2:50 PM Alex Liu wrote: > Good question, > > `info` is probably the best

Re: [VOTE] KIP-465: Add Consolidated Connector Endpoint to Connect REST API

Useful and concise KIP +1 (non-binding) Konstantine On Mon, May 6, 2019 at 10:43 AM Randall Hauch wrote: > Thanks, Dan. As mentioned on the discussion, this is really a nice little > addition that was alway missing from the API. > > +1 (binding) > > Randall > > On Mon, May 6, 2019 at 9:23 AM

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

Thanks for the KIP Magesh, it's quite useful towards the goals for more general multi-tenancy in Connect. Couple of comments from me too: I think the fact that the default policy is 'null' (no implementation) should be mentioned on the table next to the available implementations. Currently the

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

Resending an email I sent this Monday but didn't make it to the mailing list: Hi Chris, thanks for the KIP! I have the following initial comments: 1. In its current form, the code snippet gives the impression that this is a new interface or an interface that completely replaces the

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

gt; as with the current ConnectClusterStateImpl class but with a few method > calls changed here and there. > > Thanks again for your thoughts! > > Cheers, > > Chris > > On Fri, Apr 19, 2019 at 11:24 AM Konstantine Karantasis < > konstant...@confluent.io> wrote: &

Re: [DISCUSS] KIP-440: Extend Connect Converter to support headers

Thanks for the KIP Yaroslav! Apologies for the late comment. However, after reading the KIP it's still not very clear to me what happens to the existing HeaderConverter interface and what's the expectation for existing code implementing this interface out there. Looking at the PR I see that the

Re: [VOTE] KIP-481: SerDe Improvements for Connect Decimal type in JSON

Thanks Almog! Nicely designed and concise KIP. +1 non-binding Konstantine On Tue, Aug 6, 2019 at 11:44 PM Almog Gavra wrote: > Hello Everyone, > > After discussions on > > https://lists.apache.org/thread.html/fa665a6dc59f73ca294a00bcbef2eaa3ad00cc69626e91c516fa4fca@%3Cdev.kafka.apache.org%3E

Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON

ased on your suggestion (e.g. s/Consumer/Sink > Converter/g and s/Producer/Source Converter/g). > > Cheers, > Almog > > On Wed, Aug 14, 2019 at 8:13 AM Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > Thanks Almog for preparing this KIP! > >

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

PM Konstantine Karantasis < konstant...@confluent.io> wrote: > > Thanks Arjun for tackling the need to support this very useful feature. > > One thing I noticed while reading the KIP is that I would have loved to > see more info regarding how this proposal depends on the unde

Re: [VOTE] KIP-495: Dynamically Adjust Log Levels in Connect

Thanks for the KIP Arjun. FYI, I left a few comments on the discussion thread, but mentioning here too since I noticed that the vote started a few hours ago. Konstantine On Wed, Aug 14, 2019 at 12:13 AM Cyrus Vafadari wrote: > I am excited to see this implemented +1 nonbinding > > On Tue,

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

Thanks Arjun for tackling the need to support this very useful feature. One thing I noticed while reading the KIP is that I would have loved to see more info regarding how this proposal depends on the underlying logging APIs and implementations. For instance, my understanding is that slf4j can

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

s way, > but since it never returned null, this should be fine. > > re: example usage, I've added some screenshot on how this feature would be > used with jconsole. > > Hope this works! > > Thanks very much, > Arjun > > On Wed, Aug 14, 2019 at 6:42 AM Konstantine

Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON

Thanks Almog for preparing this KIP! I think it will improve usability and troubleshooting with JSON data a lot. The finalized plan seems quite concrete now. I also liked that some implementation specific implications (such as setting the ObjectMapper to deserialize floating point as BigDecimal)

Re: [VOTE] KIP-495: Dynamically Adjust Log Levels in Connect

Thanks for the updates on the KIP Arjun! +1 by me (non-binding) Konstantine On Wed, Aug 14, 2019 at 6:46 AM Konstantine Karantasis < konstant...@confluent.io> wrote: > > Thanks for the KIP Arjun. > > FYI, I left a few comments on the discussion thread, but mentioning

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

Thanks for the KIP Chris! This proposal will fill a gap in Kafka Connect deployments and will strengthen their production readiness in even more diverse environments, in which current workarounds are less practical. I've read the discussions regarding why the rebalancing protocol is used here

KIP-521: Enable redirection of Connect's log4j messages to a file by default

Hi all. While we are in the midst of some very interesting KIP discussions, I'd like to bring a brief and useful KIP on the table as well. It's about enabling redirection of log4j logging to a file for Kafka Connect by default, in a way similar to how this is done for Kafka brokers today. You

[DISCUSS] KIP-521: Enable redirection of Connect's log4j messages to a file by default

*** Missed the [DISCUSS] tag in the previous email. Reposting here, please reply in this thread instead *** Hi all. While we are in the midst of some very interesting KIP discussions, I'd like to bring a brief and useful KIP on the table as well. It's about enabling redirection of log4j logging

Re: [DISCUSS] KIP-521: Enable redirection of Connect's log4j messages to a file by default

dea. It will greatly improve the ops experience. Can't believe > we didn't do it before. > > On Wed, Sep 11, 2019 at 2:07 PM Konstantine Karantasis > wrote: > > > > *** Missed the [DISCUSS] tag in the previous email. Reposting here, > please > > reply in th

Re: [DISCUSS] KIP-521: Enable redirection of Connect's log4j messages to a file by default

the console would be preferred, and to have users change it in > one place. WDYT? > > Randall > > On Wed, Sep 11, 2019 at 7:06 PM Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > Thanks Gwen! > > Indeed, it's a common setup and it's been missing for so

Re: Stopping All Tasks When a New Connector Added

Liu feel free to share your jira account id on a separate email, so one of the committers can add you to the project. Then you or someone else will be able to assign this ticket to you. I'll review the fix some time this week. Thanks! Konstantine On Mon, Jul 22, 2019 at 5:13 AM Liu Luying

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

Quite useful KIP from an operational standpoint and I also like it in its most recent revised form. +1 (non-binding). Konstantine On Fri, Sep 20, 2019 at 9:55 AM Gwen Shapira wrote: > +1 (binding) > > Thanks for the KIP, David and for the help with the design, Colin. I > think it looks great

[VOTE] KIP-521: Redirect Connect log4j messages also to a file by default

I'd like to start the vote on KIP-521. The proposal seems straightforward and no major concerns came up during its recent brief discussions. I'd appreciate your votes and, of course, your comments are still welcome. Hopefully we could meet the forthcoming KIP deadline for this simple yet useful

Re: [VOTE] KIP-507: Securing Internal Connect REST Endpoints

Nicely done! +1 (non-binding) Konstantine On Tue, Sep 24, 2019 at 9:10 AM Rajini Sivaram wrote: > Thanks for the KIP, Chris! > > +1 (binding) > > Regards, > > Rajini > > On Tue, Sep 24, 2019 at 3:12 PM Randall Hauch wrote: > > > Thanks, Chris! > > > > +1 (binding) > > > > On Thu, Sep 19,

  1   2   3   4   >