t;
> > >> > > > >>> > > // proposed replica assignment in this request, or
> null.
> > >> For
> > >> > > > >>> adding new
> > >> > > > >>> > > partitions request, this is proposed replica assignment
> for
> > >> new
> > >> > > > >>> parti
gt; >> > > > >>> > >
> >> > > > >>> > > I did not spend much time on my AlterTopicRequest interface
> >> > > > >>> proposal, but
> >> > > > >>> >
t; > and figure it out from inspecting the
>> > replicasAssignments().
>> > > It
>> > > > >>> would
>> > > > >>> > >> be
>> > > > >>> > >> > much better for the policy writer to just be able to
&
t;>> making
> > > > >>> > >> policy-violating delete topics or delete records request.
> > > > >>> > >>
> > > > >>> > >> If no further comments a forthcoming in the next day or two
> > > then I
> > > > >>> will
> > >
point about how the
> > proposed
> > > >>> API
> > > >>> > >> > should behave.
> > > >>> > >> >
> > > >>> > >> > The current CreateTopicPolicy gets passed either the request
> > > >>> partition
> &
gt; > >> >>> Ismael
> > >>> > >> >>>
> > >>> > >> >>> On Wed, Oct 4, 2017 at 9:20 AM, Tom Bentley <
> > >>> t.j.bent...@gmail.com>
> > >>> > >> >>> wrote:
&g
the current state available from the
> >>> > >> > ClusterState, and also less symmetric between its use for
> >>> createTopic()
> >>> > >> and
> >>> > >> > for alterTopic(). This can make it harder to write a policy. For
> &
t; > >> > and figure it out from inspecting the replicasAssignments(). It
>>> would
>>> > >> be
>>> > >> > much better for the policy writer to just be able to write:
>>> > >> >
>>> >
t; ------------------
>> > >> >>> > >
>> > >> >>> > > Edoardo Comar
>> > >> >>> > >
>> > >> >>> > > IBM Message Hub
>> > >> >>> > >
>&g
017 at 11:06, Tom Bentley
> wrote:
> > >> >
> > >> >> Good point. Then I guess I can do those items too. I would also
> need to
> > >> >> do the same changes for DeleteRecordsRequest and Response.
> > >> >>
> > >> &g
> >> >>> 1. A policy that can't send errors to clients is much less useful
> >> >>> 2. Testing policies is much easier with `validateOnly`
> >> >>>
> >> >>> Ismael
> >> >>>
> >> >>> On Wed, Oct 4, 2017 at 9:20 AM, Tom Bentley
> >
t;> >>> > DeleteTopicRequest and adding an error message to
>> DeleteTopicResponse.
>> >>> > Since those are not policy-related I think they're best left out of
>> >>> > KIP-201. I suppose it is up to you and Mickael whether to narrow the
&
gt; >>> > > Thanks Tom,
> >>> > > looks got to me and KIP-201 could supersede KIP-170
> >>> > > but could you please add a missing motivation bullet that was
> behind
> >>> > > KIP-170:
> >>> > >
> >>> &
authorizer
>>> used
>>> > for
>>> > > > that (delete on topic) but fine grained control could be better
>>> (even
>>> > > > already happens for topic deletion).
>>> > > > 2. I know about the problem of restarting broker
> > not just against the request metadata but also
>> > > against the current amount of resources already used in the cluster
>> (eg
>> > > number of partitions).
>> > >
>> > > thanks
>> > > Edo
>> > >
> > request
> > >
> > > not just against the request metadata but also
> > > against the current amount of resources already used in the cluster (eg
> > > number of partitions).
> > >
> > > thanks
> > > Edo
> > > ----------------------
> > >
> > > E
; > against the current amount of resources already used in the cluster (eg
> > number of partitions).
> >
> > thanks
> > Edo
> > --
> >
> > Edoardo Comar
> >
> > IBM Message Hub
> >
> > IBM UK Ltd, Hu
> number of partitions).
>
> thanks
> Edo
> --
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
>
>
>
> From: Tom Bentley
> To: dev@kafka.apache.org
> Date: 02/10/2017 15:15
> Subject:Re: [
: [DISCUSS] KIP-201: Rationalising Policy interfaces
Hi All,
I've updated KIP-201 again so there is now a single policy interface (and
thus a single key by which to configure it) for topic creation,
modification, deletion and record deletion, which each have their own
validation method.
Ther
g : DevExperience<http://paolopatierno.wordpress.com/>
>
>
>
> From: isma...@gmail.com on behalf of Ismael Juma <
> ism...@juma.me.uk>
> Sent: Wednesday, September 27, 2017 10:30 AM
> To: dev@kafka.apache.org
> Subject: Re: [DISCUSS] KIP-201
esday, September 27, 2017 10:30 AM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-201: Rationalising Policy interfaces
A couple of questions:
1. Is this a concrete requirement from a user or is it hypothetical?
2. You sure you would want to do this in the broker instead of the clients?
I
Hi Ismael,
Thanks for looking at the KIP and explaining the thinking behind the
original API.
Looking at the updated KIP, I notice that we actually have a
> TopicDeletionPolicy with a separate config. That seems to be a halfway
> house. Not sure about that.
>
I can certainly see that point of vi
t; From: isma...@gmail.com on behalf of Ismael Juma <
> ism...@juma.me.uk>
> Sent: Wednesday, September 27, 2017 10:18 AM
> To: dev@kafka.apache.org
> Subject: Re: [DISCUSS] KIP-201: Rationalising Policy interfaces
>
> A couple more comments:
>
> 1. "If this KIP is ac
nesday, September 27, 2017 9:47 AM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-201: Rationalising Policy interfaces
I have updated the KIP to add a common policy interface for topic and
message deletion. This included pulling ClusterState and TopicState
interfaces up to the top level so that t
vExperience<http://paolopatierno.wordpress.com/>
From: isma...@gmail.com on behalf of Ismael Juma
Sent: Wednesday, September 27, 2017 10:18 AM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-201: Rationalising Policy interfaces
A couple more comments:
1. &q
p into MetadataCache. No need for big upfront
> > allocations.
> >
> > ciao,
> > Edo
> > --
> >
> > Edoardo Comar
> >
> > IBM Message Hub
> >
> > IBM UK Ltd, Hursley Park, SO21 2JN
>
}
>> >
>> > I think that the implementation of ClusterState that the server will
>> pass
>> > to the policy.validate method
>> > would just lazily tap into MetadataCache. No need for big upfront
>> > allocations.
>> >
>>
>
> From: Tom Bentley
> To: dev@kafka.apache.org
> Date: 26/09/2017 17:39
> Subject:Re: [DISCUSS] KIP-201: Rationalising Policy interfaces
>
>
>
> Hi Edoardo,
>
>
> what about a single method in ClusterState
> >
> > interface Cluste
r big upfront
allocations.
ciao,
Edo
--
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
From: Tom Bentley
To: dev@kafka.apache.org
Date: 26/09/2017 17:39
Subject:Re: [DISCUSS] KIP-201: Rationalising Policy inter
Hi Edoardo,
what about a single method in ClusterState
>
> interface ClusterState {
> public Map topicsState();
>
> }
>
> which could return a read-only snapshot of the cluster metadata ?
>
Sure that would work too. A concern with that is that we end up allocating
a potentially r
atierno>
Blog : DevExperience<http://paolopatierno.wordpress.com/>
From: Edoardo Comar
Sent: Tuesday, September 26, 2017 4:18 PM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-201: Rationalising Policy interfaces
Thanks Tom,
what about a single
, Hursley Park, SO21 2JN
From: Tom Bentley
To: dev@kafka.apache.org
Date: 26/09/2017 16:41
Subject:Re: [DISCUSS] KIP-201: Rationalising Policy interfaces
Hi Edoardo and Ismael,
Thanks for the comments. If we're going to have an new ClusterState
parameter via which we can quer
discussed in the KIP-204 thread.
> --
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
>
>
>
> From: Edoardo Comar
> To: dev@kafka.apache.org
> Date: 26/09/2017 14:01
&g
/09/2017 14:01
Subject:Re: [DISCUSS] KIP-201: Rationalising Policy interfaces
Hi Tom,
yes it makes sense to have a single KIP for enhancing these policies.
As Mickael pointed out, we need that the create/alter topic policy are
able to assess the current cluster metadata.
KIP-170
7;m happy if you incorporate the enhancements to create/alter that allow a
> check against the cluster metadata
> and leave out - to anther KIP, or maybe I'll rewrite 170 the changes to
> delete.
>
> thanks
> Edo
>
> ------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
>
Hub
IBM UK Ltd, Hursley Park, SO21 2JN
From: Tom Bentley
To: dev@kafka.apache.org
Date: 25/09/2017 18:11
Subject:Re: [DISCUSS] KIP-201: Rationalising Policy interfaces
Hi Mickael,
Thanks for the reply.
Thanks for the KIP. Is this meant to superseed KIP-170 ?
> If so,
Hi Ismael,
On 25 September 2017 at 17:51, Ismael Juma wrote:
> We don't have this policy today for what it's worth.
>
Thanks for the clarification. On re-reading I realise I misinterpreted
Guozhang Wang's suggestion when 1.0.0 was first mooted:
> Just to clarify, my proposal is that moving fo
Hi Mickael,
Thanks for the reply.
Thanks for the KIP. Is this meant to superseed KIP-170 ?
> If so, one of our key requirements was to be able to access the
> topics/partitions list from the policy, so an administrator could
> enforce a partition limit for example.
>
It's not meant to replace KI
On Mon, Sep 25, 2017 at 5:32 PM, Tom Bentley wrote:
> > bq. If this KIP is accepted for Kafka 1.1.0 this removal could happen in
> > Kafka 3.0.0
> >
> > There would be no Kafka 2.0 ?
> >
>
> As I understand it, a deprecation has to exist for a complete major version
> number cycle before the feat
Hi Tom,
Thanks for the KIP. Is this meant to superseed KIP-170 ?
If so, one of our key requirements was to be able to access the
topics/partitions list from the policy, so an administrator could
enforce a partition limit for example.
Also instead of simply having the Java Principal object, could
bq. deprecations that are added in 1.x (x>0) have to remain in all 2.y
Makes sense.
It is fine to exclude KIP-113 from your KIP.
Thanks
On Mon, Sep 25, 2017 at 9:32 AM, Tom Bentley wrote:
> Hi Ted,
>
> Thanks for the feedback!
>
> bq. topic.action.policy.class.name
> >
> > Since the policy wo
Hi Ted,
Thanks for the feedback!
bq. topic.action.policy.class.name
>
> Since the policy would cover more than one action, how about using actions
> for the second word ?
>
Good point, done.
> For TopicState interface, the abstract modifier for its methods are not
> needed.
>
Fixed.
bq. KIP-
bq. topic.action.policy.class.name
Since the policy would cover more than one action, how about using actions
for the second word ?
For TopicState interface, the abstract modifier for its methods are not
needed.
bq. KIP-113
Mind adding more to the above bullet ?
bq. If this KIP is accepted for
Hi,
I'd like to start a discussion for KIP-201. The basic point is that new
AdminClient APIs for modifying topics should have a configurable policy to
allow the administrator to veto the modifications. Just adding a
ModifyTopicPolicy would make for awkwardness by having separate policies
for creat
44 matches
Mail list logo
