[jira] [Commented] (KAFKA-5856) Add AdminClient.createPartitions()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175359#comment-16175359
 ] 

ASF GitHub Bot commented on KAFKA-5856:
---

GitHub user tombentley opened a pull request:

https://github.com/apache/kafka/pull/3937

KAFKA-5856 AdminClient.createPartitions() follow up

Adds support for noop requests. When assignments are given with a request 
that would be a noop we validate that the given assignments match the actual 
ones, so that the state of the partitions after a successful call definitely 
matches what was requested.

I can put the additional Javadoc on the exception in a separate PR if you 
prefer.

The tests have the [improvements 
requested](https://github.com/apache/kafka/pull/3930#issuecomment-331130475) by 
@ijuma

The Javadoc is improved too. Putting the expected exceptions on the 
AdminClient method is rather distant from where they're actually throw (the 
Future from the Map from the Results from the call), but it keeps the 
documentation about the method as a whole in one place. I didn't know whether 
to include the detailed possible causes for each Exception. The more detail the 
harder it is to maintain, but the more useful to the client.

/cc @ijuma 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tombentley/kafka 
KAFKA-5856-AdminClient.createPartitions-follow-up

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3937.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3937


commit 1fad83a82bd1477cf6b46ac26755bfcb37b4914a
Author: Tom Bentley 
Date:   2017-09-21T13:09:55Z

End exception messages with a period

commit cfcf1dbd59d3707035e15f2e6a3f857d22aadcdd
Author: Tom Bentley 
Date:   2017-09-21T13:18:12Z

Javadoc the difference between InvalidTopic and UknownTopicOrPartition

commit 3b2bb532a1097ffc58b8ca519dd42fcee71b63de
Author: Tom Bentley 
Date:   2017-09-21T16:38:05Z

Handling for noop requests

commit 08625b187c7713b41a49f5a12c2ca5221a24fb38
Author: Tom Bentley 
Date:   2017-09-21T17:08:32Z

Javadoc, plus change a couple of exceptions to be more consistent

commit 512308b563a192aa2a924ae732ef9b14e0b1b49a
Author: Tom Bentley 
Date:   2017-09-21T19:46:46Z

Improve tests

Ensure state doesn't change for errors cases whether or not 
validateOnly=true

Add a mixed success and failure case




> Add AdminClient.createPartitions()
> --
>
> Key: KAFKA-5856
> URL: https://issues.apache.org/jira/browse/KAFKA-5856
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Tom Bentley
>Assignee: Tom Bentley
>  Labels: kip
> Fix For: 1.0.0
>
>
> It should be possible to increase the partition count using the AdminClient. 
> See 
> [KIP-195|https://cwiki.apache.org/confluence/display/KAFKA/KIP-195%3A+AdminClient.increasePartitions]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5856) Add AdminClient.createPartitions()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174842#comment-16174842
 ] 

ASF GitHub Bot commented on KAFKA-5856:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/3930


> Add AdminClient.createPartitions()
> --
>
> Key: KAFKA-5856
> URL: https://issues.apache.org/jira/browse/KAFKA-5856
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Tom Bentley
>Assignee: Tom Bentley
>  Labels: kip
> Fix For: 1.0.0
>
>
> It should be possible to increase the partition count using the AdminClient. 
> See 
> [KIP-195|https://cwiki.apache.org/confluence/display/KAFKA/KIP-195%3A+AdminClient.increasePartitions]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5856) Add AdminClient.createPartitions()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174258#comment-16174258
 ] 

ASF GitHub Bot commented on KAFKA-5856:
---

GitHub user ijuma opened a pull request:

https://github.com/apache/kafka/pull/3930

KAFKA-5856; AdminClient.createPartitions() follow-up (KIP-195)

- Remove DelayedCreatePartitions to reduce code duplication
- Avoid unnecessary ZK calls (call it once per request instead
of once per topic, if possible)
- Simplify code
- A few minor clean-ups

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ijuma/kafka 
kafka-5856-admin-client-creation-partitions

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3930.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3930






> Add AdminClient.createPartitions()
> --
>
> Key: KAFKA-5856
> URL: https://issues.apache.org/jira/browse/KAFKA-5856
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Tom Bentley
>Assignee: Tom Bentley
>  Labels: kip
> Fix For: 1.0.0
>
>
> It should be possible to increase the partition count using the AdminClient. 
> See 
> [KIP-195|https://cwiki.apache.org/confluence/display/KAFKA/KIP-195%3A+AdminClient.increasePartitions]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5856) Add AdminClient.createPartitions()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174245#comment-16174245
 ] 

ASF GitHub Bot commented on KAFKA-5856:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/3870


> Add AdminClient.createPartitions()
> --
>
> Key: KAFKA-5856
> URL: https://issues.apache.org/jira/browse/KAFKA-5856
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Tom Bentley
>Assignee: Tom Bentley
>  Labels: kip
>
> It should be possible to increase the partition count using the AdminClient. 
> See 
> [KIP-195|https://cwiki.apache.org/confluence/display/KAFKA/KIP-195%3A+AdminClient.increasePartitions]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5856) Add AdminClient.createPartitions()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167712#comment-16167712
 ] 

ASF GitHub Bot commented on KAFKA-5856:
---

GitHub user tombentley opened a pull request:

https://github.com/apache/kafka/pull/3870

KAFKA-5856: Add AdminClient.createPartitions()

See KIP-195.

The contribution is my original work and I license the work to the project 
under the project's open source license.

This patch adds AdminClient.createPartitions() and the network protocol is
uses. The broker-side algorithm is as follows:

1. KafkaApis makes some initial checks on the request, then delegates to the
   new AdminManager.createPartitions() method.
2. AdminManager.createPartitions() performs some validation then delegates 
to
   AdminUtils.addPartitions().

Aside: I felt it was safer to add the extra validation in
AdminManager.createPartitions() than in AdminUtils.addPartitions() since the
latter is used on other code paths which might fail differently with the
introduction of extra checks.

3. AdminUtils.addPartitions() does its own checks and adds the partitions.
4. AdminManager then uses the existing topic purgatory to wait for the
   PartitionInfo available from the metadata cache to become consistent with
   the new total number of partitions.

The messages of exceptions thrown in AdminUtils affecting this new API have
been made consistent with initial capital letter and terminating period.
A few have been reworded for clarity. I've also standardized on using
String.format().

cc @ijuma

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tombentley/kafka 
KAFKA-5856-AdminClient.createPartitions

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3870.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3870


commit ab07f15a794c385cbfdecd33a5a44c7725e8d103
Author: Tom Bentley 
Date:   2017-09-15T09:50:59Z

KAFKA-5856: Add AdminClient.createPartitions()

See KIP-195.

This patch adds AdminClient.createPartitions() and the network protocol is
uses. The broker-side algorithm is as follows:

1. KafkaApis makes some initial checks on the request, then delegates to the
   new AdminManager.createPartitions() method.
2. AdminManager.createPartitions() performs some validation then delegates 
to
   AdminUtils.addPartitions().

Aside: I felt it was safer to add the extra validation in
AdminManager.createPartitions() than in AdminUtils.addPartitions() since the
latter is used on other code paths which might fail differently with the
introduction of extra checks.

3. AdminUtils.addPartitions() does its own checks and adds the partitions.
4. AdminManager then uses the existing topic purgatory to wait for the
   PartitionInfo available from the metadata cache to become consistent with
   the new total number of partitions.

The messages of exceptions thrown in AdminUtils affecting this new API have
been made consistent with initial capital letter and terminating period.
A few have been reworded for clarity. I've also standardized on using
String.format().




> Add AdminClient.createPartitions()
> --
>
> Key: KAFKA-5856
> URL: https://issues.apache.org/jira/browse/KAFKA-5856
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Tom Bentley
>Assignee: Tom Bentley
>  Labels: kip
>
> It should be possible to increase the partition count using the AdminClient. 
> See 
> [KIP-195|https://cwiki.apache.org/confluence/display/KAFKA/KIP-195%3A+AdminClient.increasePartitions]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)