Re: [VOTE] KIP-4 Create Topics Schema

[Grant Henke]

Thanks to all who voted. The KIP-4 Create Topics changes passed with +4
(binding), and +4 (non-binding).

There is a patch available for review here:
https://github.com/apache/kafka/pull/1489

On Tue, Jun 21, 2016 at 1:11 AM, Manikumar Reddy 
wrote:

> +1 (non-binding)
>
> On Tue, Jun 21, 2016 at 9:16 AM, Ewen Cheslack-Postava 
> wrote:
>
> > +1 (binding) and thanks for the work on this Grant!
> >
> > -Ewen
> >
> > On Mon, Jun 20, 2016 at 12:18 PM, Gwen Shapira 
> wrote:
> >
> > > +1 (binding)
> > >
> > > On Mon, Jun 20, 2016 at 12:13 PM, Tom Crayford 
> > > wrote:
> > > > +1 (non-binding)
> > > >
> > > > On Mon, Jun 20, 2016 at 8:07 PM, Harsha  wrote:
> > > >
> > > >> +1 (binding)
> > > >> -Harsha
> > > >>
> > > >> On Mon, Jun 20, 2016, at 11:33 AM, Ismael Juma wrote:
> > > >> > +1 (binding)
> > > >> >
> > > >> > On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers <
> dana.pow...@gmail.com
> > >
> > > >> > wrote:
> > > >> >
> > > >> > > +1 -- thanks for the update
> > > >> > >
> > > >> > > On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke <
> > ghe...@cloudera.com>
> > > >> wrote:
> > > >> > > > I have update the patch and wiki based on the feedback in the
> > > >> discussion
> > > >> > > > thread. The only change is that instead of logging and
> > > disconnecting
> > > >> in
> > > >> > > the
> > > >> > > > case of invalid messages (duplicate topics or both arguments)
> we
> > > now
> > > >> > > return
> > > >> > > > and InvalidRequest error back to the client for that topic.
> > > >> > > >
> > > >> > > > I would like to restart the vote now including that change. If
> > you
> > > >> have
> > > >> > > > already voted, please revote in this thread.
> > > >> > > >
> > > >> > > > Thank you,
> > > >> > > > Grant
> > > >> > > >
> > > >> > > > On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava <
> > > >> > > e...@confluent.io>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > >> Don't necessarily want to add noise here, but I'm -1 based on
> > the
> > > >> > > >> disconnect part. See discussion in other thread. (I'm +1
> > > otherwise,
> > > >> and
> > > >> > > >> happy to have my vote applied assuming we clean up that one
> > > issue.)
> > > >> > > >>
> > > >> > > >> -Ewen
> > > >> > > >>
> > > >> > > >> On Thu, Jun 16, 2016 at 6:05 PM, Harsha 
> > wrote:
> > > >> > > >>
> > > >> > > >> > +1 (binding)
> > > >> > > >> > Thanks,
> > > >> > > >> > Harsha
> > > >> > > >> >
> > > >> > > >> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> > > >> > > >> > > +1.
> > > >> > > >> > >
> > > >> > > >> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma <
> > > ism...@juma.me.uk
> > > >> >
> > > >> > > >> wrote:
> > > >> > > >> > >
> > > >> > > >> > > > +1 (binding)
> > > >> > > >> > > >
> > > >> > > >> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke <
> > > >> > > ghe...@cloudera.com>
> > > >> > > >> > wrote:
> > > >> > > >> > > >
> > > >> > > >> > > > > I would like to initiate the voting process for the
> > > "KIP-4
> > > >> > > Create
> > > >> > > >> > Topics
> > > >> > > >> > > > > Schema changes". This is not a vote for all of KIP-4,
> > but
> > > >> > > >> > specifically
> > > >> > > >> > > > for
> > > >> > > >> > > > > the create topics changes. I have included the exact
> > > changes
> > > >> > > below
> > > >> > > >> > for
> > > >> > > >> > > > > clarity:
> > > >> > > >> > > > > >
> > > >> > > >> > > > > > Create Topics Request (KAFKA-2945
> > > >> > > >> > > > > >  >)
> > > >> > > >> > > > > >
> > > >> > > >> > > > > > CreateTopics Request (Version: 0) =>
> > > >> [create_topic_requests]
> > > >> > > >> > timeout
> > > >> > > >> > > > > >   create_topic_requests => topic num_partitions
> > > >> > > >> replication_factor
> > > >> > > >> > > > > [replica_assignment] [configs]
> > > >> > > >> > > > > > topic => STRING
> > > >> > > >> > > > > > num_partitions => INT32
> > > >> > > >> > > > > > replication_factor => INT16
> > > >> > > >> > > > > > replica_assignment => partition_id [replicas]
> > > >> > > >> > > > > >   partition_id => INT32
> > > >> > > >> > > > > >   replicas => INT32
> > > >> > > >> > > > > > configs => config_key config_value
> > > >> > > >> > > > > >   config_key => STRING
> > > >> > > >> > > > > >   config_value => STRING
> > > >> > > >> > > > > >   timeout => INT32
> > > >> > > >> > > > > >
> > > >> > > >> > > > > > CreateTopicsRequest is a batch request to initiate
> > > topic
> > > >> > > creation
> > > >> > > >> > with
> > > >> > > >> > > > > > either predefined or automatic replica assignment
> and
> > > >> > > optionally
> > > >> > > >> > topic
> > > >> > > >> > > > > > configuration.
> > > >> > > >> > > > > >
> > > >> > > >> > > > > > Request semantics:
> > > >> > > >> > > > > >
> > > >> > > >> > > > > >1. Must be sent to the controller broker
> > > >> > > >> > > > > >

Re: [VOTE] KIP-4 Create Topics Schema

[Manikumar Reddy]

+1 (non-binding)

On Tue, Jun 21, 2016 at 9:16 AM, Ewen Cheslack-Postava 
wrote:

> +1 (binding) and thanks for the work on this Grant!
>
> -Ewen
>
> On Mon, Jun 20, 2016 at 12:18 PM, Gwen Shapira  wrote:
>
> > +1 (binding)
> >
> > On Mon, Jun 20, 2016 at 12:13 PM, Tom Crayford 
> > wrote:
> > > +1 (non-binding)
> > >
> > > On Mon, Jun 20, 2016 at 8:07 PM, Harsha  wrote:
> > >
> > >> +1 (binding)
> > >> -Harsha
> > >>
> > >> On Mon, Jun 20, 2016, at 11:33 AM, Ismael Juma wrote:
> > >> > +1 (binding)
> > >> >
> > >> > On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers  >
> > >> > wrote:
> > >> >
> > >> > > +1 -- thanks for the update
> > >> > >
> > >> > > On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke <
> ghe...@cloudera.com>
> > >> wrote:
> > >> > > > I have update the patch and wiki based on the feedback in the
> > >> discussion
> > >> > > > thread. The only change is that instead of logging and
> > disconnecting
> > >> in
> > >> > > the
> > >> > > > case of invalid messages (duplicate topics or both arguments) we
> > now
> > >> > > return
> > >> > > > and InvalidRequest error back to the client for that topic.
> > >> > > >
> > >> > > > I would like to restart the vote now including that change. If
> you
> > >> have
> > >> > > > already voted, please revote in this thread.
> > >> > > >
> > >> > > > Thank you,
> > >> > > > Grant
> > >> > > >
> > >> > > > On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava <
> > >> > > e...@confluent.io>
> > >> > > > wrote:
> > >> > > >
> > >> > > >> Don't necessarily want to add noise here, but I'm -1 based on
> the
> > >> > > >> disconnect part. See discussion in other thread. (I'm +1
> > otherwise,
> > >> and
> > >> > > >> happy to have my vote applied assuming we clean up that one
> > issue.)
> > >> > > >>
> > >> > > >> -Ewen
> > >> > > >>
> > >> > > >> On Thu, Jun 16, 2016 at 6:05 PM, Harsha 
> wrote:
> > >> > > >>
> > >> > > >> > +1 (binding)
> > >> > > >> > Thanks,
> > >> > > >> > Harsha
> > >> > > >> >
> > >> > > >> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> > >> > > >> > > +1.
> > >> > > >> > >
> > >> > > >> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma <
> > ism...@juma.me.uk
> > >> >
> > >> > > >> wrote:
> > >> > > >> > >
> > >> > > >> > > > +1 (binding)
> > >> > > >> > > >
> > >> > > >> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke <
> > >> > > ghe...@cloudera.com>
> > >> > > >> > wrote:
> > >> > > >> > > >
> > >> > > >> > > > > I would like to initiate the voting process for the
> > "KIP-4
> > >> > > Create
> > >> > > >> > Topics
> > >> > > >> > > > > Schema changes". This is not a vote for all of KIP-4,
> but
> > >> > > >> > specifically
> > >> > > >> > > > for
> > >> > > >> > > > > the create topics changes. I have included the exact
> > changes
> > >> > > below
> > >> > > >> > for
> > >> > > >> > > > > clarity:
> > >> > > >> > > > > >
> > >> > > >> > > > > > Create Topics Request (KAFKA-2945
> > >> > > >> > > > > > )
> > >> > > >> > > > > >
> > >> > > >> > > > > > CreateTopics Request (Version: 0) =>
> > >> [create_topic_requests]
> > >> > > >> > timeout
> > >> > > >> > > > > >   create_topic_requests => topic num_partitions
> > >> > > >> replication_factor
> > >> > > >> > > > > [replica_assignment] [configs]
> > >> > > >> > > > > > topic => STRING
> > >> > > >> > > > > > num_partitions => INT32
> > >> > > >> > > > > > replication_factor => INT16
> > >> > > >> > > > > > replica_assignment => partition_id [replicas]
> > >> > > >> > > > > >   partition_id => INT32
> > >> > > >> > > > > >   replicas => INT32
> > >> > > >> > > > > > configs => config_key config_value
> > >> > > >> > > > > >   config_key => STRING
> > >> > > >> > > > > >   config_value => STRING
> > >> > > >> > > > > >   timeout => INT32
> > >> > > >> > > > > >
> > >> > > >> > > > > > CreateTopicsRequest is a batch request to initiate
> > topic
> > >> > > creation
> > >> > > >> > with
> > >> > > >> > > > > > either predefined or automatic replica assignment and
> > >> > > optionally
> > >> > > >> > topic
> > >> > > >> > > > > > configuration.
> > >> > > >> > > > > >
> > >> > > >> > > > > > Request semantics:
> > >> > > >> > > > > >
> > >> > > >> > > > > >1. Must be sent to the controller broker
> > >> > > >> > > > > >2. If there are multiple instructions for the same
> > >> topic in
> > >> > > >> one
> > >> > > >> > > > > >request an InvalidRequestException will be logged
> on
> > >> the
> > >> > > >> broker
> > >> > > >> > and
> > >> > > >> > > > > the
> > >> > > >> > > > > >client will be disconnected.
> > >> > > >> > > > > >   - This is because the list of topics is modeled
> > >> server
> > >> > > side
> > >> > > >> > as a
> > >> > > >> > > > > >   map with TopicName as the key
> > >> > > >> > > > > >3. The principal must be authorized to the

Re: [VOTE] KIP-4 Create Topics Schema

[Ewen Cheslack-Postava]

+1 (binding) and thanks for the work on this Grant!

-Ewen

On Mon, Jun 20, 2016 at 12:18 PM, Gwen Shapira  wrote:

> +1 (binding)
>
> On Mon, Jun 20, 2016 at 12:13 PM, Tom Crayford 
> wrote:
> > +1 (non-binding)
> >
> > On Mon, Jun 20, 2016 at 8:07 PM, Harsha  wrote:
> >
> >> +1 (binding)
> >> -Harsha
> >>
> >> On Mon, Jun 20, 2016, at 11:33 AM, Ismael Juma wrote:
> >> > +1 (binding)
> >> >
> >> > On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers 
> >> > wrote:
> >> >
> >> > > +1 -- thanks for the update
> >> > >
> >> > > On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke 
> >> wrote:
> >> > > > I have update the patch and wiki based on the feedback in the
> >> discussion
> >> > > > thread. The only change is that instead of logging and
> disconnecting
> >> in
> >> > > the
> >> > > > case of invalid messages (duplicate topics or both arguments) we
> now
> >> > > return
> >> > > > and InvalidRequest error back to the client for that topic.
> >> > > >
> >> > > > I would like to restart the vote now including that change. If you
> >> have
> >> > > > already voted, please revote in this thread.
> >> > > >
> >> > > > Thank you,
> >> > > > Grant
> >> > > >
> >> > > > On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava <
> >> > > e...@confluent.io>
> >> > > > wrote:
> >> > > >
> >> > > >> Don't necessarily want to add noise here, but I'm -1 based on the
> >> > > >> disconnect part. See discussion in other thread. (I'm +1
> otherwise,
> >> and
> >> > > >> happy to have my vote applied assuming we clean up that one
> issue.)
> >> > > >>
> >> > > >> -Ewen
> >> > > >>
> >> > > >> On Thu, Jun 16, 2016 at 6:05 PM, Harsha  wrote:
> >> > > >>
> >> > > >> > +1 (binding)
> >> > > >> > Thanks,
> >> > > >> > Harsha
> >> > > >> >
> >> > > >> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> >> > > >> > > +1.
> >> > > >> > >
> >> > > >> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma <
> ism...@juma.me.uk
> >> >
> >> > > >> wrote:
> >> > > >> > >
> >> > > >> > > > +1 (binding)
> >> > > >> > > >
> >> > > >> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke <
> >> > > ghe...@cloudera.com>
> >> > > >> > wrote:
> >> > > >> > > >
> >> > > >> > > > > I would like to initiate the voting process for the
> "KIP-4
> >> > > Create
> >> > > >> > Topics
> >> > > >> > > > > Schema changes". This is not a vote for all of KIP-4, but
> >> > > >> > specifically
> >> > > >> > > > for
> >> > > >> > > > > the create topics changes. I have included the exact
> changes
> >> > > below
> >> > > >> > for
> >> > > >> > > > > clarity:
> >> > > >> > > > > >
> >> > > >> > > > > > Create Topics Request (KAFKA-2945
> >> > > >> > > > > > )
> >> > > >> > > > > >
> >> > > >> > > > > > CreateTopics Request (Version: 0) =>
> >> [create_topic_requests]
> >> > > >> > timeout
> >> > > >> > > > > >   create_topic_requests => topic num_partitions
> >> > > >> replication_factor
> >> > > >> > > > > [replica_assignment] [configs]
> >> > > >> > > > > > topic => STRING
> >> > > >> > > > > > num_partitions => INT32
> >> > > >> > > > > > replication_factor => INT16
> >> > > >> > > > > > replica_assignment => partition_id [replicas]
> >> > > >> > > > > >   partition_id => INT32
> >> > > >> > > > > >   replicas => INT32
> >> > > >> > > > > > configs => config_key config_value
> >> > > >> > > > > >   config_key => STRING
> >> > > >> > > > > >   config_value => STRING
> >> > > >> > > > > >   timeout => INT32
> >> > > >> > > > > >
> >> > > >> > > > > > CreateTopicsRequest is a batch request to initiate
> topic
> >> > > creation
> >> > > >> > with
> >> > > >> > > > > > either predefined or automatic replica assignment and
> >> > > optionally
> >> > > >> > topic
> >> > > >> > > > > > configuration.
> >> > > >> > > > > >
> >> > > >> > > > > > Request semantics:
> >> > > >> > > > > >
> >> > > >> > > > > >1. Must be sent to the controller broker
> >> > > >> > > > > >2. If there are multiple instructions for the same
> >> topic in
> >> > > >> one
> >> > > >> > > > > >request an InvalidRequestException will be logged on
> >> the
> >> > > >> broker
> >> > > >> > and
> >> > > >> > > > > the
> >> > > >> > > > > >client will be disconnected.
> >> > > >> > > > > >   - This is because the list of topics is modeled
> >> server
> >> > > side
> >> > > >> > as a
> >> > > >> > > > > >   map with TopicName as the key
> >> > > >> > > > > >3. The principal must be authorized to the "Create"
> >> > > Operation
> >> > > >> > on the
> >> > > >> > > > > >"Cluster" resource to create topics.
> >> > > >> > > > > >   - Unauthorized requests will receive a
> >> > > >> > > > > ClusterAuthorizationException
> >> > > >> > > > > >4.
> >> > > >> > > > > >
> >> > > >> > > > > >Only one from ReplicaAssignment or (num_partitions +
> >> > > >> > > > > 

Re: [VOTE] KIP-4 Create Topics Schema

[Gwen Shapira]

+1 (binding)

On Mon, Jun 20, 2016 at 12:13 PM, Tom Crayford  wrote:
> +1 (non-binding)
>
> On Mon, Jun 20, 2016 at 8:07 PM, Harsha  wrote:
>
>> +1 (binding)
>> -Harsha
>>
>> On Mon, Jun 20, 2016, at 11:33 AM, Ismael Juma wrote:
>> > +1 (binding)
>> >
>> > On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers 
>> > wrote:
>> >
>> > > +1 -- thanks for the update
>> > >
>> > > On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke 
>> wrote:
>> > > > I have update the patch and wiki based on the feedback in the
>> discussion
>> > > > thread. The only change is that instead of logging and disconnecting
>> in
>> > > the
>> > > > case of invalid messages (duplicate topics or both arguments) we now
>> > > return
>> > > > and InvalidRequest error back to the client for that topic.
>> > > >
>> > > > I would like to restart the vote now including that change. If you
>> have
>> > > > already voted, please revote in this thread.
>> > > >
>> > > > Thank you,
>> > > > Grant
>> > > >
>> > > > On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava <
>> > > e...@confluent.io>
>> > > > wrote:
>> > > >
>> > > >> Don't necessarily want to add noise here, but I'm -1 based on the
>> > > >> disconnect part. See discussion in other thread. (I'm +1 otherwise,
>> and
>> > > >> happy to have my vote applied assuming we clean up that one issue.)
>> > > >>
>> > > >> -Ewen
>> > > >>
>> > > >> On Thu, Jun 16, 2016 at 6:05 PM, Harsha  wrote:
>> > > >>
>> > > >> > +1 (binding)
>> > > >> > Thanks,
>> > > >> > Harsha
>> > > >> >
>> > > >> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
>> > > >> > > +1.
>> > > >> > >
>> > > >> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma > >
>> > > >> wrote:
>> > > >> > >
>> > > >> > > > +1 (binding)
>> > > >> > > >
>> > > >> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke <
>> > > ghe...@cloudera.com>
>> > > >> > wrote:
>> > > >> > > >
>> > > >> > > > > I would like to initiate the voting process for the "KIP-4
>> > > Create
>> > > >> > Topics
>> > > >> > > > > Schema changes". This is not a vote for all of KIP-4, but
>> > > >> > specifically
>> > > >> > > > for
>> > > >> > > > > the create topics changes. I have included the exact changes
>> > > below
>> > > >> > for
>> > > >> > > > > clarity:
>> > > >> > > > > >
>> > > >> > > > > > Create Topics Request (KAFKA-2945
>> > > >> > > > > > )
>> > > >> > > > > >
>> > > >> > > > > > CreateTopics Request (Version: 0) =>
>> [create_topic_requests]
>> > > >> > timeout
>> > > >> > > > > >   create_topic_requests => topic num_partitions
>> > > >> replication_factor
>> > > >> > > > > [replica_assignment] [configs]
>> > > >> > > > > > topic => STRING
>> > > >> > > > > > num_partitions => INT32
>> > > >> > > > > > replication_factor => INT16
>> > > >> > > > > > replica_assignment => partition_id [replicas]
>> > > >> > > > > >   partition_id => INT32
>> > > >> > > > > >   replicas => INT32
>> > > >> > > > > > configs => config_key config_value
>> > > >> > > > > >   config_key => STRING
>> > > >> > > > > >   config_value => STRING
>> > > >> > > > > >   timeout => INT32
>> > > >> > > > > >
>> > > >> > > > > > CreateTopicsRequest is a batch request to initiate topic
>> > > creation
>> > > >> > with
>> > > >> > > > > > either predefined or automatic replica assignment and
>> > > optionally
>> > > >> > topic
>> > > >> > > > > > configuration.
>> > > >> > > > > >
>> > > >> > > > > > Request semantics:
>> > > >> > > > > >
>> > > >> > > > > >1. Must be sent to the controller broker
>> > > >> > > > > >2. If there are multiple instructions for the same
>> topic in
>> > > >> one
>> > > >> > > > > >request an InvalidRequestException will be logged on
>> the
>> > > >> broker
>> > > >> > and
>> > > >> > > > > the
>> > > >> > > > > >client will be disconnected.
>> > > >> > > > > >   - This is because the list of topics is modeled
>> server
>> > > side
>> > > >> > as a
>> > > >> > > > > >   map with TopicName as the key
>> > > >> > > > > >3. The principal must be authorized to the "Create"
>> > > Operation
>> > > >> > on the
>> > > >> > > > > >"Cluster" resource to create topics.
>> > > >> > > > > >   - Unauthorized requests will receive a
>> > > >> > > > > ClusterAuthorizationException
>> > > >> > > > > >4.
>> > > >> > > > > >
>> > > >> > > > > >Only one from ReplicaAssignment or (num_partitions +
>> > > >> > > > > replication_factor
>> > > >> > > > > >), can be defined in one instruction.
>> > > >> > > > > >- If both parameters are specified an
>> > > InvalidRequestException
>> > > >> > will
>> > > >> > > > be
>> > > >> > > > > >   logged on the broker and the client will be
>> > > disconnected.
>> > > >> > > > > >   - In the case ReplicaAssignment is defined number of
>> > > >> > partitions
>> > > >> > > > and
>> > > >> 

Re: [VOTE] KIP-4 Create Topics Schema

[Tom Crayford]

+1 (non-binding)

On Mon, Jun 20, 2016 at 8:07 PM, Harsha  wrote:

> +1 (binding)
> -Harsha
>
> On Mon, Jun 20, 2016, at 11:33 AM, Ismael Juma wrote:
> > +1 (binding)
> >
> > On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers 
> > wrote:
> >
> > > +1 -- thanks for the update
> > >
> > > On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke 
> wrote:
> > > > I have update the patch and wiki based on the feedback in the
> discussion
> > > > thread. The only change is that instead of logging and disconnecting
> in
> > > the
> > > > case of invalid messages (duplicate topics or both arguments) we now
> > > return
> > > > and InvalidRequest error back to the client for that topic.
> > > >
> > > > I would like to restart the vote now including that change. If you
> have
> > > > already voted, please revote in this thread.
> > > >
> > > > Thank you,
> > > > Grant
> > > >
> > > > On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava <
> > > e...@confluent.io>
> > > > wrote:
> > > >
> > > >> Don't necessarily want to add noise here, but I'm -1 based on the
> > > >> disconnect part. See discussion in other thread. (I'm +1 otherwise,
> and
> > > >> happy to have my vote applied assuming we clean up that one issue.)
> > > >>
> > > >> -Ewen
> > > >>
> > > >> On Thu, Jun 16, 2016 at 6:05 PM, Harsha  wrote:
> > > >>
> > > >> > +1 (binding)
> > > >> > Thanks,
> > > >> > Harsha
> > > >> >
> > > >> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> > > >> > > +1.
> > > >> > >
> > > >> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma  >
> > > >> wrote:
> > > >> > >
> > > >> > > > +1 (binding)
> > > >> > > >
> > > >> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke <
> > > ghe...@cloudera.com>
> > > >> > wrote:
> > > >> > > >
> > > >> > > > > I would like to initiate the voting process for the "KIP-4
> > > Create
> > > >> > Topics
> > > >> > > > > Schema changes". This is not a vote for all of KIP-4, but
> > > >> > specifically
> > > >> > > > for
> > > >> > > > > the create topics changes. I have included the exact changes
> > > below
> > > >> > for
> > > >> > > > > clarity:
> > > >> > > > > >
> > > >> > > > > > Create Topics Request (KAFKA-2945
> > > >> > > > > > )
> > > >> > > > > >
> > > >> > > > > > CreateTopics Request (Version: 0) =>
> [create_topic_requests]
> > > >> > timeout
> > > >> > > > > >   create_topic_requests => topic num_partitions
> > > >> replication_factor
> > > >> > > > > [replica_assignment] [configs]
> > > >> > > > > > topic => STRING
> > > >> > > > > > num_partitions => INT32
> > > >> > > > > > replication_factor => INT16
> > > >> > > > > > replica_assignment => partition_id [replicas]
> > > >> > > > > >   partition_id => INT32
> > > >> > > > > >   replicas => INT32
> > > >> > > > > > configs => config_key config_value
> > > >> > > > > >   config_key => STRING
> > > >> > > > > >   config_value => STRING
> > > >> > > > > >   timeout => INT32
> > > >> > > > > >
> > > >> > > > > > CreateTopicsRequest is a batch request to initiate topic
> > > creation
> > > >> > with
> > > >> > > > > > either predefined or automatic replica assignment and
> > > optionally
> > > >> > topic
> > > >> > > > > > configuration.
> > > >> > > > > >
> > > >> > > > > > Request semantics:
> > > >> > > > > >
> > > >> > > > > >1. Must be sent to the controller broker
> > > >> > > > > >2. If there are multiple instructions for the same
> topic in
> > > >> one
> > > >> > > > > >request an InvalidRequestException will be logged on
> the
> > > >> broker
> > > >> > and
> > > >> > > > > the
> > > >> > > > > >client will be disconnected.
> > > >> > > > > >   - This is because the list of topics is modeled
> server
> > > side
> > > >> > as a
> > > >> > > > > >   map with TopicName as the key
> > > >> > > > > >3. The principal must be authorized to the "Create"
> > > Operation
> > > >> > on the
> > > >> > > > > >"Cluster" resource to create topics.
> > > >> > > > > >   - Unauthorized requests will receive a
> > > >> > > > > ClusterAuthorizationException
> > > >> > > > > >4.
> > > >> > > > > >
> > > >> > > > > >Only one from ReplicaAssignment or (num_partitions +
> > > >> > > > > replication_factor
> > > >> > > > > >), can be defined in one instruction.
> > > >> > > > > >- If both parameters are specified an
> > > InvalidRequestException
> > > >> > will
> > > >> > > > be
> > > >> > > > > >   logged on the broker and the client will be
> > > disconnected.
> > > >> > > > > >   - In the case ReplicaAssignment is defined number of
> > > >> > partitions
> > > >> > > > and
> > > >> > > > > >   replicas will be calculated from the supplied
> > > >> > replica_assignment.
> > > >> > > > > >   - In the case of defined (num_partitions +
> > > >> > replication_factor)
> > > >> > > > > >   replica 

Re: [VOTE] KIP-4 Create Topics Schema

[Harsha]

+1 (binding)
-Harsha

On Mon, Jun 20, 2016, at 11:33 AM, Ismael Juma wrote:
> +1 (binding)
> 
> On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers 
> wrote:
> 
> > +1 -- thanks for the update
> >
> > On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke  wrote:
> > > I have update the patch and wiki based on the feedback in the discussion
> > > thread. The only change is that instead of logging and disconnecting in
> > the
> > > case of invalid messages (duplicate topics or both arguments) we now
> > return
> > > and InvalidRequest error back to the client for that topic.
> > >
> > > I would like to restart the vote now including that change. If you have
> > > already voted, please revote in this thread.
> > >
> > > Thank you,
> > > Grant
> > >
> > > On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava <
> > e...@confluent.io>
> > > wrote:
> > >
> > >> Don't necessarily want to add noise here, but I'm -1 based on the
> > >> disconnect part. See discussion in other thread. (I'm +1 otherwise, and
> > >> happy to have my vote applied assuming we clean up that one issue.)
> > >>
> > >> -Ewen
> > >>
> > >> On Thu, Jun 16, 2016 at 6:05 PM, Harsha  wrote:
> > >>
> > >> > +1 (binding)
> > >> > Thanks,
> > >> > Harsha
> > >> >
> > >> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> > >> > > +1.
> > >> > >
> > >> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma 
> > >> wrote:
> > >> > >
> > >> > > > +1 (binding)
> > >> > > >
> > >> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke <
> > ghe...@cloudera.com>
> > >> > wrote:
> > >> > > >
> > >> > > > > I would like to initiate the voting process for the "KIP-4
> > Create
> > >> > Topics
> > >> > > > > Schema changes". This is not a vote for all of KIP-4, but
> > >> > specifically
> > >> > > > for
> > >> > > > > the create topics changes. I have included the exact changes
> > below
> > >> > for
> > >> > > > > clarity:
> > >> > > > > >
> > >> > > > > > Create Topics Request (KAFKA-2945
> > >> > > > > > )
> > >> > > > > >
> > >> > > > > > CreateTopics Request (Version: 0) => [create_topic_requests]
> > >> > timeout
> > >> > > > > >   create_topic_requests => topic num_partitions
> > >> replication_factor
> > >> > > > > [replica_assignment] [configs]
> > >> > > > > > topic => STRING
> > >> > > > > > num_partitions => INT32
> > >> > > > > > replication_factor => INT16
> > >> > > > > > replica_assignment => partition_id [replicas]
> > >> > > > > >   partition_id => INT32
> > >> > > > > >   replicas => INT32
> > >> > > > > > configs => config_key config_value
> > >> > > > > >   config_key => STRING
> > >> > > > > >   config_value => STRING
> > >> > > > > >   timeout => INT32
> > >> > > > > >
> > >> > > > > > CreateTopicsRequest is a batch request to initiate topic
> > creation
> > >> > with
> > >> > > > > > either predefined or automatic replica assignment and
> > optionally
> > >> > topic
> > >> > > > > > configuration.
> > >> > > > > >
> > >> > > > > > Request semantics:
> > >> > > > > >
> > >> > > > > >1. Must be sent to the controller broker
> > >> > > > > >2. If there are multiple instructions for the same topic in
> > >> one
> > >> > > > > >request an InvalidRequestException will be logged on the
> > >> broker
> > >> > and
> > >> > > > > the
> > >> > > > > >client will be disconnected.
> > >> > > > > >   - This is because the list of topics is modeled server
> > side
> > >> > as a
> > >> > > > > >   map with TopicName as the key
> > >> > > > > >3. The principal must be authorized to the "Create"
> > Operation
> > >> > on the
> > >> > > > > >"Cluster" resource to create topics.
> > >> > > > > >   - Unauthorized requests will receive a
> > >> > > > > ClusterAuthorizationException
> > >> > > > > >4.
> > >> > > > > >
> > >> > > > > >Only one from ReplicaAssignment or (num_partitions +
> > >> > > > > replication_factor
> > >> > > > > >), can be defined in one instruction.
> > >> > > > > >- If both parameters are specified an
> > InvalidRequestException
> > >> > will
> > >> > > > be
> > >> > > > > >   logged on the broker and the client will be
> > disconnected.
> > >> > > > > >   - In the case ReplicaAssignment is defined number of
> > >> > partitions
> > >> > > > and
> > >> > > > > >   replicas will be calculated from the supplied
> > >> > replica_assignment.
> > >> > > > > >   - In the case of defined (num_partitions +
> > >> > replication_factor)
> > >> > > > > >   replica assignment will be automatically generated by
> > the
> > >> > server.
> > >> > > > > >   - One or the other must be defined. The existing broker
> > >> side
> > >> > auto
> > >> > > > > >   create defaults will not be used
> > >> > > > > >   (default.replication.factor, num.partitions). The client
> > >> > > > > implementation can
> > >> > > > > >   

Re: [VOTE] KIP-4 Create Topics Schema

[Ismael Juma]

+1 (binding)

On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers  wrote:

> +1 -- thanks for the update
>
> On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke  wrote:
> > I have update the patch and wiki based on the feedback in the discussion
> > thread. The only change is that instead of logging and disconnecting in
> the
> > case of invalid messages (duplicate topics or both arguments) we now
> return
> > and InvalidRequest error back to the client for that topic.
> >
> > I would like to restart the vote now including that change. If you have
> > already voted, please revote in this thread.
> >
> > Thank you,
> > Grant
> >
> > On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava <
> e...@confluent.io>
> > wrote:
> >
> >> Don't necessarily want to add noise here, but I'm -1 based on the
> >> disconnect part. See discussion in other thread. (I'm +1 otherwise, and
> >> happy to have my vote applied assuming we clean up that one issue.)
> >>
> >> -Ewen
> >>
> >> On Thu, Jun 16, 2016 at 6:05 PM, Harsha  wrote:
> >>
> >> > +1 (binding)
> >> > Thanks,
> >> > Harsha
> >> >
> >> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> >> > > +1.
> >> > >
> >> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma 
> >> wrote:
> >> > >
> >> > > > +1 (binding)
> >> > > >
> >> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke <
> ghe...@cloudera.com>
> >> > wrote:
> >> > > >
> >> > > > > I would like to initiate the voting process for the "KIP-4
> Create
> >> > Topics
> >> > > > > Schema changes". This is not a vote for all of KIP-4, but
> >> > specifically
> >> > > > for
> >> > > > > the create topics changes. I have included the exact changes
> below
> >> > for
> >> > > > > clarity:
> >> > > > > >
> >> > > > > > Create Topics Request (KAFKA-2945
> >> > > > > > )
> >> > > > > >
> >> > > > > > CreateTopics Request (Version: 0) => [create_topic_requests]
> >> > timeout
> >> > > > > >   create_topic_requests => topic num_partitions
> >> replication_factor
> >> > > > > [replica_assignment] [configs]
> >> > > > > > topic => STRING
> >> > > > > > num_partitions => INT32
> >> > > > > > replication_factor => INT16
> >> > > > > > replica_assignment => partition_id [replicas]
> >> > > > > >   partition_id => INT32
> >> > > > > >   replicas => INT32
> >> > > > > > configs => config_key config_value
> >> > > > > >   config_key => STRING
> >> > > > > >   config_value => STRING
> >> > > > > >   timeout => INT32
> >> > > > > >
> >> > > > > > CreateTopicsRequest is a batch request to initiate topic
> creation
> >> > with
> >> > > > > > either predefined or automatic replica assignment and
> optionally
> >> > topic
> >> > > > > > configuration.
> >> > > > > >
> >> > > > > > Request semantics:
> >> > > > > >
> >> > > > > >1. Must be sent to the controller broker
> >> > > > > >2. If there are multiple instructions for the same topic in
> >> one
> >> > > > > >request an InvalidRequestException will be logged on the
> >> broker
> >> > and
> >> > > > > the
> >> > > > > >client will be disconnected.
> >> > > > > >   - This is because the list of topics is modeled server
> side
> >> > as a
> >> > > > > >   map with TopicName as the key
> >> > > > > >3. The principal must be authorized to the "Create"
> Operation
> >> > on the
> >> > > > > >"Cluster" resource to create topics.
> >> > > > > >   - Unauthorized requests will receive a
> >> > > > > ClusterAuthorizationException
> >> > > > > >4.
> >> > > > > >
> >> > > > > >Only one from ReplicaAssignment or (num_partitions +
> >> > > > > replication_factor
> >> > > > > >), can be defined in one instruction.
> >> > > > > >- If both parameters are specified an
> InvalidRequestException
> >> > will
> >> > > > be
> >> > > > > >   logged on the broker and the client will be
> disconnected.
> >> > > > > >   - In the case ReplicaAssignment is defined number of
> >> > partitions
> >> > > > and
> >> > > > > >   replicas will be calculated from the supplied
> >> > replica_assignment.
> >> > > > > >   - In the case of defined (num_partitions +
> >> > replication_factor)
> >> > > > > >   replica assignment will be automatically generated by
> the
> >> > server.
> >> > > > > >   - One or the other must be defined. The existing broker
> >> side
> >> > auto
> >> > > > > >   create defaults will not be used
> >> > > > > >   (default.replication.factor, num.partitions). The client
> >> > > > > implementation can
> >> > > > > >   have defaults for these options when generating the
> >> messages.
> >> > > > > >   - The first replica in [replicas] is assumed to be the
> >> > preferred
> >> > > > > >   leader. This matches current behavior elsewhere.
> >> > > > > >5. Setting a timeout > 0 will allow the request to block
> until
> >> > the
> >> > > > > >

Re: [VOTE] KIP-4 Create Topics Schema

[Dana Powers]

+1 -- thanks for the update

On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke  wrote:
> I have update the patch and wiki based on the feedback in the discussion
> thread. The only change is that instead of logging and disconnecting in the
> case of invalid messages (duplicate topics or both arguments) we now return
> and InvalidRequest error back to the client for that topic.
>
> I would like to restart the vote now including that change. If you have
> already voted, please revote in this thread.
>
> Thank you,
> Grant
>
> On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava 
> wrote:
>
>> Don't necessarily want to add noise here, but I'm -1 based on the
>> disconnect part. See discussion in other thread. (I'm +1 otherwise, and
>> happy to have my vote applied assuming we clean up that one issue.)
>>
>> -Ewen
>>
>> On Thu, Jun 16, 2016 at 6:05 PM, Harsha  wrote:
>>
>> > +1 (binding)
>> > Thanks,
>> > Harsha
>> >
>> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
>> > > +1.
>> > >
>> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma 
>> wrote:
>> > >
>> > > > +1 (binding)
>> > > >
>> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke 
>> > wrote:
>> > > >
>> > > > > I would like to initiate the voting process for the "KIP-4 Create
>> > Topics
>> > > > > Schema changes". This is not a vote for all of KIP-4, but
>> > specifically
>> > > > for
>> > > > > the create topics changes. I have included the exact changes below
>> > for
>> > > > > clarity:
>> > > > > >
>> > > > > > Create Topics Request (KAFKA-2945
>> > > > > > )
>> > > > > >
>> > > > > > CreateTopics Request (Version: 0) => [create_topic_requests]
>> > timeout
>> > > > > >   create_topic_requests => topic num_partitions
>> replication_factor
>> > > > > [replica_assignment] [configs]
>> > > > > > topic => STRING
>> > > > > > num_partitions => INT32
>> > > > > > replication_factor => INT16
>> > > > > > replica_assignment => partition_id [replicas]
>> > > > > >   partition_id => INT32
>> > > > > >   replicas => INT32
>> > > > > > configs => config_key config_value
>> > > > > >   config_key => STRING
>> > > > > >   config_value => STRING
>> > > > > >   timeout => INT32
>> > > > > >
>> > > > > > CreateTopicsRequest is a batch request to initiate topic creation
>> > with
>> > > > > > either predefined or automatic replica assignment and optionally
>> > topic
>> > > > > > configuration.
>> > > > > >
>> > > > > > Request semantics:
>> > > > > >
>> > > > > >1. Must be sent to the controller broker
>> > > > > >2. If there are multiple instructions for the same topic in
>> one
>> > > > > >request an InvalidRequestException will be logged on the
>> broker
>> > and
>> > > > > the
>> > > > > >client will be disconnected.
>> > > > > >   - This is because the list of topics is modeled server side
>> > as a
>> > > > > >   map with TopicName as the key
>> > > > > >3. The principal must be authorized to the "Create" Operation
>> > on the
>> > > > > >"Cluster" resource to create topics.
>> > > > > >   - Unauthorized requests will receive a
>> > > > > ClusterAuthorizationException
>> > > > > >4.
>> > > > > >
>> > > > > >Only one from ReplicaAssignment or (num_partitions +
>> > > > > replication_factor
>> > > > > >), can be defined in one instruction.
>> > > > > >- If both parameters are specified an InvalidRequestException
>> > will
>> > > > be
>> > > > > >   logged on the broker and the client will be disconnected.
>> > > > > >   - In the case ReplicaAssignment is defined number of
>> > partitions
>> > > > and
>> > > > > >   replicas will be calculated from the supplied
>> > replica_assignment.
>> > > > > >   - In the case of defined (num_partitions +
>> > replication_factor)
>> > > > > >   replica assignment will be automatically generated by the
>> > server.
>> > > > > >   - One or the other must be defined. The existing broker
>> side
>> > auto
>> > > > > >   create defaults will not be used
>> > > > > >   (default.replication.factor, num.partitions). The client
>> > > > > implementation can
>> > > > > >   have defaults for these options when generating the
>> messages.
>> > > > > >   - The first replica in [replicas] is assumed to be the
>> > preferred
>> > > > > >   leader. This matches current behavior elsewhere.
>> > > > > >5. Setting a timeout > 0 will allow the request to block until
>> > the
>> > > > > >topic metadata is "complete" on the controller node.
>> > > > > >   - Complete means the local topic metadata cache been
>> > completely
>> > > > > >   populated and all partitions have leaders
>> > > > > >  - The topic metadata is updated when the controller
>> sends
>> > out
>> > > > > >  update metadata requests to the brokers
>> > > > > > 

Re: [VOTE] KIP-4 Create Topics Schema

[Grant Henke]

I have update the patch and wiki based on the feedback in the discussion
thread. The only change is that instead of logging and disconnecting in the
case of invalid messages (duplicate topics or both arguments) we now return
and InvalidRequest error back to the client for that topic.

I would like to restart the vote now including that change. If you have
already voted, please revote in this thread.

Thank you,
Grant

On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava 
wrote:

> Don't necessarily want to add noise here, but I'm -1 based on the
> disconnect part. See discussion in other thread. (I'm +1 otherwise, and
> happy to have my vote applied assuming we clean up that one issue.)
>
> -Ewen
>
> On Thu, Jun 16, 2016 at 6:05 PM, Harsha  wrote:
>
> > +1 (binding)
> > Thanks,
> > Harsha
> >
> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> > > +1.
> > >
> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma 
> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke 
> > wrote:
> > > >
> > > > > I would like to initiate the voting process for the "KIP-4 Create
> > Topics
> > > > > Schema changes". This is not a vote for all of KIP-4, but
> > specifically
> > > > for
> > > > > the create topics changes. I have included the exact changes below
> > for
> > > > > clarity:
> > > > > >
> > > > > > Create Topics Request (KAFKA-2945
> > > > > > )
> > > > > >
> > > > > > CreateTopics Request (Version: 0) => [create_topic_requests]
> > timeout
> > > > > >   create_topic_requests => topic num_partitions
> replication_factor
> > > > > [replica_assignment] [configs]
> > > > > > topic => STRING
> > > > > > num_partitions => INT32
> > > > > > replication_factor => INT16
> > > > > > replica_assignment => partition_id [replicas]
> > > > > >   partition_id => INT32
> > > > > >   replicas => INT32
> > > > > > configs => config_key config_value
> > > > > >   config_key => STRING
> > > > > >   config_value => STRING
> > > > > >   timeout => INT32
> > > > > >
> > > > > > CreateTopicsRequest is a batch request to initiate topic creation
> > with
> > > > > > either predefined or automatic replica assignment and optionally
> > topic
> > > > > > configuration.
> > > > > >
> > > > > > Request semantics:
> > > > > >
> > > > > >1. Must be sent to the controller broker
> > > > > >2. If there are multiple instructions for the same topic in
> one
> > > > > >request an InvalidRequestException will be logged on the
> broker
> > and
> > > > > the
> > > > > >client will be disconnected.
> > > > > >   - This is because the list of topics is modeled server side
> > as a
> > > > > >   map with TopicName as the key
> > > > > >3. The principal must be authorized to the "Create" Operation
> > on the
> > > > > >"Cluster" resource to create topics.
> > > > > >   - Unauthorized requests will receive a
> > > > > ClusterAuthorizationException
> > > > > >4.
> > > > > >
> > > > > >Only one from ReplicaAssignment or (num_partitions +
> > > > > replication_factor
> > > > > >), can be defined in one instruction.
> > > > > >- If both parameters are specified an InvalidRequestException
> > will
> > > > be
> > > > > >   logged on the broker and the client will be disconnected.
> > > > > >   - In the case ReplicaAssignment is defined number of
> > partitions
> > > > and
> > > > > >   replicas will be calculated from the supplied
> > replica_assignment.
> > > > > >   - In the case of defined (num_partitions +
> > replication_factor)
> > > > > >   replica assignment will be automatically generated by the
> > server.
> > > > > >   - One or the other must be defined. The existing broker
> side
> > auto
> > > > > >   create defaults will not be used
> > > > > >   (default.replication.factor, num.partitions). The client
> > > > > implementation can
> > > > > >   have defaults for these options when generating the
> messages.
> > > > > >   - The first replica in [replicas] is assumed to be the
> > preferred
> > > > > >   leader. This matches current behavior elsewhere.
> > > > > >5. Setting a timeout > 0 will allow the request to block until
> > the
> > > > > >topic metadata is "complete" on the controller node.
> > > > > >   - Complete means the local topic metadata cache been
> > completely
> > > > > >   populated and all partitions have leaders
> > > > > >  - The topic metadata is updated when the controller
> sends
> > out
> > > > > >  update metadata requests to the brokers
> > > > > >   - If a timeout error occurs, the topic could still be
> created
> > > > > >   successfully at a later time. Its up to the client to query
> > for
> > > > > the state
> > > > > >   at that point.
> > > > > >6. Setting a timeout 

Re: [VOTE] KIP-4 Create Topics Schema

[Ewen Cheslack-Postava]

Don't necessarily want to add noise here, but I'm -1 based on the
disconnect part. See discussion in other thread. (I'm +1 otherwise, and
happy to have my vote applied assuming we clean up that one issue.)

-Ewen

On Thu, Jun 16, 2016 at 6:05 PM, Harsha  wrote:

> +1 (binding)
> Thanks,
> Harsha
>
> On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> > +1.
> >
> > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma  wrote:
> >
> > > +1 (binding)
> > >
> > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke 
> wrote:
> > >
> > > > I would like to initiate the voting process for the "KIP-4 Create
> Topics
> > > > Schema changes". This is not a vote for all of KIP-4, but
> specifically
> > > for
> > > > the create topics changes. I have included the exact changes below
> for
> > > > clarity:
> > > > >
> > > > > Create Topics Request (KAFKA-2945
> > > > > )
> > > > >
> > > > > CreateTopics Request (Version: 0) => [create_topic_requests]
> timeout
> > > > >   create_topic_requests => topic num_partitions replication_factor
> > > > [replica_assignment] [configs]
> > > > > topic => STRING
> > > > > num_partitions => INT32
> > > > > replication_factor => INT16
> > > > > replica_assignment => partition_id [replicas]
> > > > >   partition_id => INT32
> > > > >   replicas => INT32
> > > > > configs => config_key config_value
> > > > >   config_key => STRING
> > > > >   config_value => STRING
> > > > >   timeout => INT32
> > > > >
> > > > > CreateTopicsRequest is a batch request to initiate topic creation
> with
> > > > > either predefined or automatic replica assignment and optionally
> topic
> > > > > configuration.
> > > > >
> > > > > Request semantics:
> > > > >
> > > > >1. Must be sent to the controller broker
> > > > >2. If there are multiple instructions for the same topic in one
> > > > >request an InvalidRequestException will be logged on the broker
> and
> > > > the
> > > > >client will be disconnected.
> > > > >   - This is because the list of topics is modeled server side
> as a
> > > > >   map with TopicName as the key
> > > > >3. The principal must be authorized to the "Create" Operation
> on the
> > > > >"Cluster" resource to create topics.
> > > > >   - Unauthorized requests will receive a
> > > > ClusterAuthorizationException
> > > > >4.
> > > > >
> > > > >Only one from ReplicaAssignment or (num_partitions +
> > > > replication_factor
> > > > >), can be defined in one instruction.
> > > > >- If both parameters are specified an InvalidRequestException
> will
> > > be
> > > > >   logged on the broker and the client will be disconnected.
> > > > >   - In the case ReplicaAssignment is defined number of
> partitions
> > > and
> > > > >   replicas will be calculated from the supplied
> replica_assignment.
> > > > >   - In the case of defined (num_partitions +
> replication_factor)
> > > > >   replica assignment will be automatically generated by the
> server.
> > > > >   - One or the other must be defined. The existing broker side
> auto
> > > > >   create defaults will not be used
> > > > >   (default.replication.factor, num.partitions). The client
> > > > implementation can
> > > > >   have defaults for these options when generating the messages.
> > > > >   - The first replica in [replicas] is assumed to be the
> preferred
> > > > >   leader. This matches current behavior elsewhere.
> > > > >5. Setting a timeout > 0 will allow the request to block until
> the
> > > > >topic metadata is "complete" on the controller node.
> > > > >   - Complete means the local topic metadata cache been
> completely
> > > > >   populated and all partitions have leaders
> > > > >  - The topic metadata is updated when the controller sends
> out
> > > > >  update metadata requests to the brokers
> > > > >   - If a timeout error occurs, the topic could still be created
> > > > >   successfully at a later time. Its up to the client to query
> for
> > > > the state
> > > > >   at that point.
> > > > >6. Setting a timeout <= 0 will validate arguments and trigger
> the
> > > > >create topics and return immediately.
> > > > >   - This is essentially the fully asynchronous mode we have in
> the
> > > > >   Zookeeper tools today.
> > > > >   - The error code in the response will either contain an
> argument
> > > > >   validation exception or a timeout exception. If you receive a
> > > > timeout
> > > > >   exception, because you asked for 0 timeout, you can assume
> the
> > > > message was
> > > > >   valid and the topic creation was triggered.
> > > > >7. The request is not transactional.
> > > > >   1. If an error occurs on one topic, the others could still be
> > > > >   created.
> > > > >   2. Errors are 

Re: [VOTE] KIP-4 Create Topics Schema

[Harsha]

+1 (binding)
Thanks,
Harsha

On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> +1.
> 
> On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma  wrote:
> 
> > +1 (binding)
> >
> > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke  wrote:
> >
> > > I would like to initiate the voting process for the "KIP-4 Create Topics
> > > Schema changes". This is not a vote for all of KIP-4, but specifically
> > for
> > > the create topics changes. I have included the exact changes below for
> > > clarity:
> > > >
> > > > Create Topics Request (KAFKA-2945
> > > > )
> > > >
> > > > CreateTopics Request (Version: 0) => [create_topic_requests] timeout
> > > >   create_topic_requests => topic num_partitions replication_factor
> > > [replica_assignment] [configs]
> > > > topic => STRING
> > > > num_partitions => INT32
> > > > replication_factor => INT16
> > > > replica_assignment => partition_id [replicas]
> > > >   partition_id => INT32
> > > >   replicas => INT32
> > > > configs => config_key config_value
> > > >   config_key => STRING
> > > >   config_value => STRING
> > > >   timeout => INT32
> > > >
> > > > CreateTopicsRequest is a batch request to initiate topic creation with
> > > > either predefined or automatic replica assignment and optionally topic
> > > > configuration.
> > > >
> > > > Request semantics:
> > > >
> > > >1. Must be sent to the controller broker
> > > >2. If there are multiple instructions for the same topic in one
> > > >request an InvalidRequestException will be logged on the broker and
> > > the
> > > >client will be disconnected.
> > > >   - This is because the list of topics is modeled server side as a
> > > >   map with TopicName as the key
> > > >3. The principal must be authorized to the "Create" Operation on the
> > > >"Cluster" resource to create topics.
> > > >   - Unauthorized requests will receive a
> > > ClusterAuthorizationException
> > > >4.
> > > >
> > > >Only one from ReplicaAssignment or (num_partitions +
> > > replication_factor
> > > >), can be defined in one instruction.
> > > >- If both parameters are specified an InvalidRequestException will
> > be
> > > >   logged on the broker and the client will be disconnected.
> > > >   - In the case ReplicaAssignment is defined number of partitions
> > and
> > > >   replicas will be calculated from the supplied replica_assignment.
> > > >   - In the case of defined (num_partitions + replication_factor)
> > > >   replica assignment will be automatically generated by the server.
> > > >   - One or the other must be defined. The existing broker side auto
> > > >   create defaults will not be used
> > > >   (default.replication.factor, num.partitions). The client
> > > implementation can
> > > >   have defaults for these options when generating the messages.
> > > >   - The first replica in [replicas] is assumed to be the preferred
> > > >   leader. This matches current behavior elsewhere.
> > > >5. Setting a timeout > 0 will allow the request to block until the
> > > >topic metadata is "complete" on the controller node.
> > > >   - Complete means the local topic metadata cache been completely
> > > >   populated and all partitions have leaders
> > > >  - The topic metadata is updated when the controller sends out
> > > >  update metadata requests to the brokers
> > > >   - If a timeout error occurs, the topic could still be created
> > > >   successfully at a later time. Its up to the client to query for
> > > the state
> > > >   at that point.
> > > >6. Setting a timeout <= 0 will validate arguments and trigger the
> > > >create topics and return immediately.
> > > >   - This is essentially the fully asynchronous mode we have in the
> > > >   Zookeeper tools today.
> > > >   - The error code in the response will either contain an argument
> > > >   validation exception or a timeout exception. If you receive a
> > > timeout
> > > >   exception, because you asked for 0 timeout, you can assume the
> > > message was
> > > >   valid and the topic creation was triggered.
> > > >7. The request is not transactional.
> > > >   1. If an error occurs on one topic, the others could still be
> > > >   created.
> > > >   2. Errors are reported independently.
> > > >
> > > > QA:
> > > >
> > > >- Why is CreateTopicsRequest a batch request?
> > > >   - Scenarios where tools or admins want to create many topics
> > should
> > > >   be able to with fewer requests
> > > >   - Example: MirrorMaker may want to create the topics downstream
> > > >- What happens if some topics error immediately? Will it
> > > >return immediately?
> > > >   - The request will block until all topics have either been
> > created,
> > > >   errors, 

Re: [VOTE] KIP-4 Create Topics Schema

[Guozhang Wang]

+1.

On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma  wrote:

> +1 (binding)
>
> On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke  wrote:
>
> > I would like to initiate the voting process for the "KIP-4 Create Topics
> > Schema changes". This is not a vote for all of KIP-4, but specifically
> for
> > the create topics changes. I have included the exact changes below for
> > clarity:
> > >
> > > Create Topics Request (KAFKA-2945
> > > )
> > >
> > > CreateTopics Request (Version: 0) => [create_topic_requests] timeout
> > >   create_topic_requests => topic num_partitions replication_factor
> > [replica_assignment] [configs]
> > > topic => STRING
> > > num_partitions => INT32
> > > replication_factor => INT16
> > > replica_assignment => partition_id [replicas]
> > >   partition_id => INT32
> > >   replicas => INT32
> > > configs => config_key config_value
> > >   config_key => STRING
> > >   config_value => STRING
> > >   timeout => INT32
> > >
> > > CreateTopicsRequest is a batch request to initiate topic creation with
> > > either predefined or automatic replica assignment and optionally topic
> > > configuration.
> > >
> > > Request semantics:
> > >
> > >1. Must be sent to the controller broker
> > >2. If there are multiple instructions for the same topic in one
> > >request an InvalidRequestException will be logged on the broker and
> > the
> > >client will be disconnected.
> > >   - This is because the list of topics is modeled server side as a
> > >   map with TopicName as the key
> > >3. The principal must be authorized to the "Create" Operation on the
> > >"Cluster" resource to create topics.
> > >   - Unauthorized requests will receive a
> > ClusterAuthorizationException
> > >4.
> > >
> > >Only one from ReplicaAssignment or (num_partitions +
> > replication_factor
> > >), can be defined in one instruction.
> > >- If both parameters are specified an InvalidRequestException will
> be
> > >   logged on the broker and the client will be disconnected.
> > >   - In the case ReplicaAssignment is defined number of partitions
> and
> > >   replicas will be calculated from the supplied replica_assignment.
> > >   - In the case of defined (num_partitions + replication_factor)
> > >   replica assignment will be automatically generated by the server.
> > >   - One or the other must be defined. The existing broker side auto
> > >   create defaults will not be used
> > >   (default.replication.factor, num.partitions). The client
> > implementation can
> > >   have defaults for these options when generating the messages.
> > >   - The first replica in [replicas] is assumed to be the preferred
> > >   leader. This matches current behavior elsewhere.
> > >5. Setting a timeout > 0 will allow the request to block until the
> > >topic metadata is "complete" on the controller node.
> > >   - Complete means the local topic metadata cache been completely
> > >   populated and all partitions have leaders
> > >  - The topic metadata is updated when the controller sends out
> > >  update metadata requests to the brokers
> > >   - If a timeout error occurs, the topic could still be created
> > >   successfully at a later time. Its up to the client to query for
> > the state
> > >   at that point.
> > >6. Setting a timeout <= 0 will validate arguments and trigger the
> > >create topics and return immediately.
> > >   - This is essentially the fully asynchronous mode we have in the
> > >   Zookeeper tools today.
> > >   - The error code in the response will either contain an argument
> > >   validation exception or a timeout exception. If you receive a
> > timeout
> > >   exception, because you asked for 0 timeout, you can assume the
> > message was
> > >   valid and the topic creation was triggered.
> > >7. The request is not transactional.
> > >   1. If an error occurs on one topic, the others could still be
> > >   created.
> > >   2. Errors are reported independently.
> > >
> > > QA:
> > >
> > >- Why is CreateTopicsRequest a batch request?
> > >   - Scenarios where tools or admins want to create many topics
> should
> > >   be able to with fewer requests
> > >   - Example: MirrorMaker may want to create the topics downstream
> > >- What happens if some topics error immediately? Will it
> > >return immediately?
> > >   - The request will block until all topics have either been
> created,
> > >   errors, or the timeout has been hit
> > >   - There is no "short circuiting" where 1 error stops the other
> > >   topics from being created
> > >- Why implement "partial blocking" instead of fully async or fully
> > >consistent?
> > >   - See Cluster Consistent Blocking
> > >   

Re: [VOTE] KIP-4 Create Topics Schema

[Ismael Juma]

+1 (binding)

On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke  wrote:

> I would like to initiate the voting process for the "KIP-4 Create Topics
> Schema changes". This is not a vote for all of KIP-4, but specifically for
> the create topics changes. I have included the exact changes below for
> clarity:
> >
> > Create Topics Request (KAFKA-2945
> > )
> >
> > CreateTopics Request (Version: 0) => [create_topic_requests] timeout
> >   create_topic_requests => topic num_partitions replication_factor
> [replica_assignment] [configs]
> > topic => STRING
> > num_partitions => INT32
> > replication_factor => INT16
> > replica_assignment => partition_id [replicas]
> >   partition_id => INT32
> >   replicas => INT32
> > configs => config_key config_value
> >   config_key => STRING
> >   config_value => STRING
> >   timeout => INT32
> >
> > CreateTopicsRequest is a batch request to initiate topic creation with
> > either predefined or automatic replica assignment and optionally topic
> > configuration.
> >
> > Request semantics:
> >
> >1. Must be sent to the controller broker
> >2. If there are multiple instructions for the same topic in one
> >request an InvalidRequestException will be logged on the broker and
> the
> >client will be disconnected.
> >   - This is because the list of topics is modeled server side as a
> >   map with TopicName as the key
> >3. The principal must be authorized to the "Create" Operation on the
> >"Cluster" resource to create topics.
> >   - Unauthorized requests will receive a
> ClusterAuthorizationException
> >4.
> >
> >Only one from ReplicaAssignment or (num_partitions +
> replication_factor
> >), can be defined in one instruction.
> >- If both parameters are specified an InvalidRequestException will be
> >   logged on the broker and the client will be disconnected.
> >   - In the case ReplicaAssignment is defined number of partitions and
> >   replicas will be calculated from the supplied replica_assignment.
> >   - In the case of defined (num_partitions + replication_factor)
> >   replica assignment will be automatically generated by the server.
> >   - One or the other must be defined. The existing broker side auto
> >   create defaults will not be used
> >   (default.replication.factor, num.partitions). The client
> implementation can
> >   have defaults for these options when generating the messages.
> >   - The first replica in [replicas] is assumed to be the preferred
> >   leader. This matches current behavior elsewhere.
> >5. Setting a timeout > 0 will allow the request to block until the
> >topic metadata is "complete" on the controller node.
> >   - Complete means the local topic metadata cache been completely
> >   populated and all partitions have leaders
> >  - The topic metadata is updated when the controller sends out
> >  update metadata requests to the brokers
> >   - If a timeout error occurs, the topic could still be created
> >   successfully at a later time. Its up to the client to query for
> the state
> >   at that point.
> >6. Setting a timeout <= 0 will validate arguments and trigger the
> >create topics and return immediately.
> >   - This is essentially the fully asynchronous mode we have in the
> >   Zookeeper tools today.
> >   - The error code in the response will either contain an argument
> >   validation exception or a timeout exception. If you receive a
> timeout
> >   exception, because you asked for 0 timeout, you can assume the
> message was
> >   valid and the topic creation was triggered.
> >7. The request is not transactional.
> >   1. If an error occurs on one topic, the others could still be
> >   created.
> >   2. Errors are reported independently.
> >
> > QA:
> >
> >- Why is CreateTopicsRequest a batch request?
> >   - Scenarios where tools or admins want to create many topics should
> >   be able to with fewer requests
> >   - Example: MirrorMaker may want to create the topics downstream
> >- What happens if some topics error immediately? Will it
> >return immediately?
> >   - The request will block until all topics have either been created,
> >   errors, or the timeout has been hit
> >   - There is no "short circuiting" where 1 error stops the other
> >   topics from being created
> >- Why implement "partial blocking" instead of fully async or fully
> >consistent?
> >   - See Cluster Consistent Blocking
> >   <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking
> >
> >below
> >- Why require the request to go to the controller?
> >   

[VOTE] KIP-4 Create Topics Schema

[Grant Henke]

I would like to initiate the voting process for the "KIP-4 Create Topics
Schema changes". This is not a vote for all of KIP-4, but specifically for
the create topics changes. I have included the exact changes below for
clarity:
>
> Create Topics Request (KAFKA-2945
> )
>
> CreateTopics Request (Version: 0) => [create_topic_requests] timeout
>   create_topic_requests => topic num_partitions replication_factor 
> [replica_assignment] [configs]
> topic => STRING
> num_partitions => INT32
> replication_factor => INT16
> replica_assignment => partition_id [replicas]
>   partition_id => INT32
>   replicas => INT32
> configs => config_key config_value
>   config_key => STRING
>   config_value => STRING
>   timeout => INT32
>
> CreateTopicsRequest is a batch request to initiate topic creation with
> either predefined or automatic replica assignment and optionally topic
> configuration.
>
> Request semantics:
>
>1. Must be sent to the controller broker
>2. If there are multiple instructions for the same topic in one
>request an InvalidRequestException will be logged on the broker and the
>client will be disconnected.
>   - This is because the list of topics is modeled server side as a
>   map with TopicName as the key
>3. The principal must be authorized to the "Create" Operation on the
>"Cluster" resource to create topics.
>   - Unauthorized requests will receive a ClusterAuthorizationException
>4.
>
>Only one from ReplicaAssignment or (num_partitions + replication_factor
>), can be defined in one instruction.
>- If both parameters are specified an InvalidRequestException will be
>   logged on the broker and the client will be disconnected.
>   - In the case ReplicaAssignment is defined number of partitions and
>   replicas will be calculated from the supplied replica_assignment.
>   - In the case of defined (num_partitions + replication_factor)
>   replica assignment will be automatically generated by the server.
>   - One or the other must be defined. The existing broker side auto
>   create defaults will not be used
>   (default.replication.factor, num.partitions). The client implementation 
> can
>   have defaults for these options when generating the messages.
>   - The first replica in [replicas] is assumed to be the preferred
>   leader. This matches current behavior elsewhere.
>5. Setting a timeout > 0 will allow the request to block until the
>topic metadata is "complete" on the controller node.
>   - Complete means the local topic metadata cache been completely
>   populated and all partitions have leaders
>  - The topic metadata is updated when the controller sends out
>  update metadata requests to the brokers
>   - If a timeout error occurs, the topic could still be created
>   successfully at a later time. Its up to the client to query for the 
> state
>   at that point.
>6. Setting a timeout <= 0 will validate arguments and trigger the
>create topics and return immediately.
>   - This is essentially the fully asynchronous mode we have in the
>   Zookeeper tools today.
>   - The error code in the response will either contain an argument
>   validation exception or a timeout exception. If you receive a timeout
>   exception, because you asked for 0 timeout, you can assume the message 
> was
>   valid and the topic creation was triggered.
>7. The request is not transactional.
>   1. If an error occurs on one topic, the others could still be
>   created.
>   2. Errors are reported independently.
>
> QA:
>
>- Why is CreateTopicsRequest a batch request?
>   - Scenarios where tools or admins want to create many topics should
>   be able to with fewer requests
>   - Example: MirrorMaker may want to create the topics downstream
>- What happens if some topics error immediately? Will it
>return immediately?
>   - The request will block until all topics have either been created,
>   errors, or the timeout has been hit
>   - There is no "short circuiting" where 1 error stops the other
>   topics from being created
>- Why implement "partial blocking" instead of fully async or fully
>consistent?
>   - See Cluster Consistent Blocking
>   
> 
>below
>- Why require the request to go to the controller?
>   - The controller is responsible for the cluster metadata and its
>   propagation
>   - See Request Forwarding
>   
>