t thought through how this new endpoint will
> evolve and look as a top level endpoint for the various plugin types.
>
> Konstantine
>
>
>
> On Mon, Jun 7, 2021 at 5:02 PM Cyrus Vafadari wrote:
>
> > Since it is just two endpoints, I agree it's fine to move them over.
&g
like.
>
> Wdyt?
>
> Konstantine
>
>
> On Mon, Jun 7, 2021 at 4:18 PM Cyrus Vafadari wrote:
>
> > I wasn't trying to suggest deprecating all verbs and endpoints prefixed
> > with `/connector-plugins`, just the `GET: /connector-plugins` as I
> detailed
>
mentioning that the two endpoints will be equivalent
> with respect to existing functionality in the upgrade section.
>
> Konstantine
>
> On Mon, Jun 7, 2021 at 10:57 AM Cyrus Vafadari
> wrote:
>
> > I will move to vote
> >
> > On Mon, Jun 7, 2021 at 10:44 AM Cy
I will move to vote
On Mon, Jun 7, 2021 at 10:44 AM Cyrus Vafadari wrote:
> Thanks for the feedback,
>
> I added an example request/response for SMTs, and I thought about your
> suggestion re:deprecation and am now explicitly proposing to mark the
> existing endpoint as depre
Hi all!
I'd like to kick off voting for KIP-494 --
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=120740150
Discussion Thread --
https://lists.apache.org/x/thread.html/aa8c6eb080cae48046bd6c5af0f5c98dc4866c3f073f535c517a4921@%3Cdev.kafka.apache.org%3E
Cyrus
nd that deprecation doesn't
> necessarily mean imminent removal. The next opportunity to remove this end
> point would be the next major release at the earliest.
>
> Regards,
> Konstantine
>
>
> On Mon, Apr 19, 2021 at 10:13 AM Cyrus Vafadari
> wrote:
>
> > Thanks!
Thanks! Anyone else from the community with final thoughts before going to
vote?
On Mon, Apr 19, 2021 at 4:16 AM Tom Bentley wrote:
> Hi Cyrus,
>
> That seems reasonable to me.
>
> On Thu, Apr 15, 2021 at 6:44 PM Cyrus Vafadari
> wrote:
>
> > What do you thin
in types, but so the json for those
> plugins should omit that property rather than have type=null.
>
> Apart from that it seems reasonable to me. Thanks again,
>
> Tom
>
> On Thu, Apr 15, 2021 at 3:49 PM Cyrus Vafadari
> wrote:
>
> > Hi Tom,
> >
> &g
precate in the future maybe we
> should add a /plugins/connector/config/validate endpoint now?
>
> Many thanks,
>
> Tom
>
> On Tue, Apr 13, 2021 at 6:46 PM Cyrus Vafadari
> wrote:
>
> > Hello all,
> >
> > As the number of connect plugins, SMT's, etc hav
:56 PM Cyrus Vafadari wrote:
> Thanks for the feedback, Chris.
>
> WRT Worker-plugins, I see your point that they aren't very useful for
> people trying to create connectors to see what exists already, but I would
> find them useful for things like making assertions against
for SMTs and other plugins. This will
> > make it easier to see what SMTs you've loaded.
> >
> > Hoping for some good discussion!
> >
> > Thanks, all
> >
> > On Sat, Jul 20, 2019 at 9:07 PM Cyrus Vafadari
> wrote:
> >
> > > Hello, all,
Cyrus Vafadari created KAFKA-9258:
-
Summary: Connect ConnectorStatusMetricsGroup sometimes NPE
Key: KAFKA-9258
URL: https://issues.apache.org/jira/browse/KAFKA-9258
Project: Kafka
Issue Type
bleList instead of
> ArrayList or discover some unknown more efficient implementation than
> ArrayList).
>
> Almog
>
> On Sat, Aug 3, 2019 at 7:01 PM Cyrus Vafadari wrote:
>
> > Hi all,
> >
> > I've written a KIP to update the SinkTask abstract cl
I'd like to bump the thread to get review on the PR.
https://github.com/apache/kafka/pull/6843
Thanks, all!
On Wed, Jul 3, 2019 at 9:33 AM Cyrus Vafadari wrote:
> Thanks for the input, everyone! I'm marking this KIP as accepted with
> 3 binding votes (Gwen, Guozhang, Randall)
> 3 n
I've updated the KIP here to simplify and clarify the goal -- it's a pretty
simple KIP to add a REST endpoint for SMTs and other plugins. This will
make it easier to see what SMTs you've loaded.
Hoping for some good discussion!
Thanks, all
On Sat, Jul 20, 2019 at 9:07 PM Cyrus Vafadari wrote
I am excited to see this implemented +1 nonbinding
On Tue, Aug 13, 2019 at 2:01 PM Chris Egerton wrote:
> Nice stuff, Arjun! +1 (non-binding)
>
> On Tue, Aug 13, 2019 at 1:55 PM Arjun Satish
> wrote:
>
> > Hey everyone,
> >
> > I'd like to start a vote for KIP-495 (
> >
> >
>
This looks like a useful feature, the strategy makes sense, and the KIP is
thorough and nicely written. Thanks!
Cyrus
On Thu, Aug 1, 2019, 12:40 PM Chris Egerton wrote:
> Thanks Arjun! Looks good to me.
>
> On Thu, Aug 1, 2019 at 12:33 PM Arjun Satish
> wrote:
>
> > Thanks for the feedback,
Hi all,
I've written a KIP to update the SinkTask abstract class to specify that
the `put` method will take ArrayList. I think this will greatly simplify
connector development, so you aren't limited to the simplest iterations. It
will also harden the ordering guarantees of the API.
Looking
Cyrus Vafadari created KAFKA-8749:
-
Summary: Connect SinkTask.put(...) to specify
ArrayList in Signature
Key: KAFKA-8749
URL: https://issues.apache.org/jira/browse/KAFKA-8749
Project: Kafka
Hello, all,
I'd like to start discussion on a new KIP I'm proposing to add HTTP
endpoints to Kafka Connect to give us some more insight into other
non-connector plugins, like Simple Message Transformations (SMTs). The KIP
would not change how Connect works in any meaningful way except to expose
Cyrus Vafadari created KAFKA-8694:
-
Summary: Connect REST Endpoint for Transformations (SMTs) and
other Plugins
Key: KAFKA-8694
URL: https://issues.apache.org/jira/browse/KAFKA-8694
Project: Kafka
+1 (non-binding). Nice KIP.
>
> On 23/06/2019, 17:27, "Gwen Shapira" wrote:
>
> +1 from me too
>
> On Tue, Jun 18, 2019, 10:30 AM Guozhang Wang
> wrote:
>
> > Cyrus, thanks for the updates, +1.
> >
> > On Mon,
Cyrus Vafadari created KAFKA-8605:
-
Summary: Warn users when they have same connector in their
plugin-path more than once
Key: KAFKA-8605
URL: https://issues.apache.org/jira/browse/KAFKA-8605
Project
Many thanks for the feedback. Per Gwen's suggestion, I've updated the KIP
to specify that the task count will be per-worker (no additional MBean tag,
since each process is a worker) and per-connector (MBean tag).
On Mon, Jun 17, 2019 at 8:24 PM Cyrus Vafadari wrote:
> I meant to write:
>
I meant to write:
I've also updated the KIP to clarify that every task must have exactly one
non-null *status* at all times.
On Mon, Jun 17, 2019 at 6:55 PM Cyrus Vafadari wrote:
> Guozhang,
>
> Both of Kafka's implementations of "StatusBackingStore" immediately del
ntness of the proposed
> > metrics. For example, is running+paused+failed = total? Can a task be
> > failed and destroyed and therefore count in 2 of those metrics?
> >
> > On Thu, Jun 6, 2019 at 12:29 PM Cyrus Vafadari
> wrote:
> > >
> > > Konstantine,
> &
dall Hauch
> >
> > On Wed, Jun 5, 2019 at 10:36 AM Andrew Schofield <
> > andrew_schofi...@live.com>
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > Andrew Schofield
> > >
> > > On 05/06/2019, 14:04, "Ryanne Dolan"
Hi all,
Like like to start voting in the following KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-475%3A+New+Metric+to+Measure+Number+of+Tasks+on+a+Connector
Discussion thread:
inally proposed. Thanks
> for taking my suggestion into account -- the updated KIP looks great.
>
> Thanks for contributing this improvement, Cyrus!
>
> Best regards,
>
> Randall
>
> On Tue, Jun 4, 2019 at 6:35 PM Cyrus Vafadari wrote:
>
> > Randall,
> >
Randall,
I've updated the KIP to include all of your recommendations!
Cyrus
On Tue, Jun 4, 2019 at 2:55 PM Cyrus Vafadari wrote:
> Randall,
>
> I plan to update the public details section and the performance impact as
> you recommended.
>
> Regarding state-specific
dall
>
> On Sun, Jun 2, 2019 at 10:05 PM Ryanne Dolan
> wrote:
>
> > Cyrus, I agree this would be useful.
> >
> > Ryanne
> >
> > On Fri, May 31, 2019, 7:10 PM Oleksandr Diachenko >
> > wrote:
> >
> > >
> > >
> > &g
Hello Dev,
I'd like to start the discussion of KIP-475: New Metric to Measure Number
of Tasks on a Connector.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-475%3A+New+Metric+to+Measure+Number+of+Tasks+on+a+Connector
The proposal is pretty straightforward -- to add a new metric to Connect
Cyrus Vafadari created KAFKA-8447:
-
Summary: Add Connector-level metric to count tasks
Key: KAFKA-8447
URL: https://issues.apache.org/jira/browse/KAFKA-8447
Project: Kafka
Issue Type: New
Hello,
I'd like to request permission to create a new KIP in the Apache Kafka
project!
Thanks,
Cyrus Vafadari
Allen,
I think this is a good fix -- it is often intuitive for users to define
consumer and producer configs on a Connector level as opposed to a worker
level, since each connector-task gets its own producer/consumer. I think we
should re-start the discussion.
I think this KIP is actually
"scheduled.rebalance.max.delay.ms" -- "...the actual delay used by the
leader to hold off redistribution of connectors and tasks and maintain
imbalance may be less or equal to this value." How is the actual delay
determined and specified in the AssignmentFlatBuffer? I see this defaults
to a max
I see there are no "Rejected Alternatives" in the KIP -- What is the reason
you can't run a distributed herder as a single node? Is the goal here to
run multiple standalone workers with the same group.id?
37 matches
Mail list logo
