Re: [kafka-clients] Re: [VOTE] 1.0.0 RC1

2017-10-16 Thread Guozhang Wang 
Thanks Jun,

I will roll out a new RC for this PR then.


Guozhang

On Mon, Oct 16, 2017 at 2:19 PM, Jun Rao  wrote:

> Hi, Guozhang,
>
> Onur found an existing performance bug in the controller when there are
> lots of partitions. The fix is simple (https://github.com/apache/
> kafka/pull/4075) and reduces the controlled shutdown time from 6.5 mins
> to 30 secs, with
> 25K partitions, RF=2 and 5 brokers.
>
> It would be useful to include this fix in 1.0.0.
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 16, 2017 at 9:55 AM, Guozhang Wang  wrote:
>
>> Hi Tom,
>>
>> Thanks for pointing it out. I meant to say Oct. 17th, Tuesday, for a 72
>> hours period.
>>
>> That being said, we need to have a lazy majority to accept a release RC
>> according to our bylaws (https://cwiki.apache.org/conf
>> luence/display/KAFKA/Bylaws). And if we cannot achieve that via thorough
>> testing within the period we will automatically extend the voting process.
>>
>>
>> Guozhang
>>
>>
>>
>> On Mon, Oct 16, 2017 at 5:09 AM, Thomas Crayford <
>> tcrayf...@salesforce.com> wrote:
>>
>>> Hi Guozhang,
>>>
>>> This says the due date on the testing is October 13th, which was the day
>>> this email was sent. Is that accurate, or is it meant to read October
>>> 17th,
>>> which is next Tuesday?
>>>
>>> I feel like this short a testing window for a 1.0 RC is a little low, as
>>> 1.0 is clearly a big announcement of stability, and folk should be given
>>> enough time to do thorough testing.
>>>
>>> Thanks
>>>
>>> Tom
>>>
>>> On Fri, Oct 13, 2017 at 9:12 PM, Guozhang Wang 
>>> wrote:
>>>
>>> > Hello Kafka users, developers and client-developers,
>>> >
>>> > This is the second candidate for release of Apache Kafka 1.0.0.
>>> >
>>> > It's worth noting that starting in this version we are using a
>>> different
>>> > version protocol with three digits: *major.minor.bug-fix*
>>> >
>>> > Any and all testing is welcome, but the following areas are worth
>>> > highlighting:
>>> >
>>> > 1. Client developers should verify that their clients can
>>> produce/consume
>>> > to/from 1.0.0 brokers (ideally with compressed and uncompressed data).
>>> > 2. Performance and stress testing. Heroku and LinkedIn have helped with
>>> > this in the past (and issues have been found and fixed).
>>> > 3. End users can verify that their apps work correctly with the new
>>> > release.
>>> >
>>> > This is a major version release of Apache Kafka. It includes 29 new
>>> KIPs.
>>> > See the release notes and release plan
>>> > (*https://cwiki.apache.org/confluence/pages/viewpage.
>>> > action?pageId=71764913
>>> > >> ageId=71764913
>>> > >*)
>>> > for more details. A few feature highlights:
>>> >
>>> > * Java 9 support with significantly faster TLS and CRC32C
>>> implementations
>>> > (KIP)
>>> > * JBOD improvements: disk failure only disables failed disk but not the
>>> > broker (KIP-112/KIP-113)
>>> > * Newly added metrics across all the modules (KIP-164, KIP-168,
>>> KIP-187,
>>> > KIP-188, KIP-196)
>>> > * Kafka Streams API improvements (KIP-120 / 130 / 138 / 150 / 160 /
>>> 161),
>>> > and drop compatibility "Evolving" annotations
>>> >
>>> > Release notes for the 1.0.0 release:
>>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/RELEASE_NOTES.html
>>> > *
>>> >
>>> >
>>> >
>>> > *** Please download, test and vote by Tuesday, October 13, 8pm PT
>>> >
>>> > Kafka's KEYS file containing PGP keys we use to sign the release:
>>> > http://kafka.apache.org/KEYS
>>> >
>>> > * Release artifacts to be voted upon (source and binary):
>>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>>> > *
>>> >
>>> > * Maven artifacts to be voted upon:
>>> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>> >
>>> > * Javadoc:
>>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/javadoc/
>>> > *
>>> >
>>> > * Tag to be voted upon (off 1.0 branch) is the 1.0.0-rc1 tag:
>>> >
>>> > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=
>>> > 9424d29dbf0a3c538215b0b98b1e6b956481e4d5
>>> >
>>> > * Documentation:
>>> > Note the documentation can't be pushed live due to changes that will
>>> not go
>>> > live until the release. You can manually verify by downloading
>>> > http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>>> > kafka_2.11-1.0.0-site-docs.tgz
>>> >
>>> > * Successful Jenkins builds for the 1.0.0 branch:
>>> > Unit/integration tests: https://builds.apache.org/job/
>>> kafka-1.0-jdk7/31/
>>> > System test: https://jenkins.confluent.io/job/system-test-kafka-1.0/1/
>>> >
>>> >
>>> > /**
>>> >
>>> >
>>> > Thanks,
>>> > -- Guozhang
>>> >
>>>
>>
>>
>>
>> --
>> -- Guozhang
>>
>> --
>> You received this message because you are

Re: [kafka-clients] Re: [VOTE] 1.0.0 RC1

2017-10-16 Thread Jun Rao 
Hi, Guozhang,

Onur found an existing performance bug in the controller when there are
lots of partitions. The fix is simple (
https://github.com/apache/kafka/pull/4075) and reduces the controlled
shutdown time from 6.5 mins to 30 secs, with
25K partitions, RF=2 and 5 brokers.

It would be useful to include this fix in 1.0.0.

Thanks,

Jun


On Mon, Oct 16, 2017 at 9:55 AM, Guozhang Wang  wrote:

> Hi Tom,
>
> Thanks for pointing it out. I meant to say Oct. 17th, Tuesday, for a 72
> hours period.
>
> That being said, we need to have a lazy majority to accept a release RC
> according to our bylaws (https://cwiki.apache.org/
> confluence/display/KAFKA/Bylaws). And if we cannot achieve that via
> thorough testing within the period we will automatically extend the voting
> process.
>
>
> Guozhang
>
>
>
> On Mon, Oct 16, 2017 at 5:09 AM, Thomas Crayford  > wrote:
>
>> Hi Guozhang,
>>
>> This says the due date on the testing is October 13th, which was the day
>> this email was sent. Is that accurate, or is it meant to read October
>> 17th,
>> which is next Tuesday?
>>
>> I feel like this short a testing window for a 1.0 RC is a little low, as
>> 1.0 is clearly a big announcement of stability, and folk should be given
>> enough time to do thorough testing.
>>
>> Thanks
>>
>> Tom
>>
>> On Fri, Oct 13, 2017 at 9:12 PM, Guozhang Wang 
>> wrote:
>>
>> > Hello Kafka users, developers and client-developers,
>> >
>> > This is the second candidate for release of Apache Kafka 1.0.0.
>> >
>> > It's worth noting that starting in this version we are using a different
>> > version protocol with three digits: *major.minor.bug-fix*
>> >
>> > Any and all testing is welcome, but the following areas are worth
>> > highlighting:
>> >
>> > 1. Client developers should verify that their clients can
>> produce/consume
>> > to/from 1.0.0 brokers (ideally with compressed and uncompressed data).
>> > 2. Performance and stress testing. Heroku and LinkedIn have helped with
>> > this in the past (and issues have been found and fixed).
>> > 3. End users can verify that their apps work correctly with the new
>> > release.
>> >
>> > This is a major version release of Apache Kafka. It includes 29 new
>> KIPs.
>> > See the release notes and release plan
>> > (*https://cwiki.apache.org/confluence/pages/viewpage.
>> > action?pageId=71764913
>> > > pageId=71764913
>> > >*)
>> > for more details. A few feature highlights:
>> >
>> > * Java 9 support with significantly faster TLS and CRC32C
>> implementations
>> > (KIP)
>> > * JBOD improvements: disk failure only disables failed disk but not the
>> > broker (KIP-112/KIP-113)
>> > * Newly added metrics across all the modules (KIP-164, KIP-168, KIP-187,
>> > KIP-188, KIP-196)
>> > * Kafka Streams API improvements (KIP-120 / 130 / 138 / 150 / 160 /
>> 161),
>> > and drop compatibility "Evolving" annotations
>> >
>> > Release notes for the 1.0.0 release:
>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/RELEASE_NOTES.html
>> > *
>> >
>> >
>> >
>> > *** Please download, test and vote by Tuesday, October 13, 8pm PT
>> >
>> > Kafka's KEYS file containing PGP keys we use to sign the release:
>> > http://kafka.apache.org/KEYS
>> >
>> > * Release artifacts to be voted upon (source and binary):
>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>> > *
>> >
>> > * Maven artifacts to be voted upon:
>> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> >
>> > * Javadoc:
>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/javadoc/
>> > *
>> >
>> > * Tag to be voted upon (off 1.0 branch) is the 1.0.0-rc1 tag:
>> >
>> > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=
>> > 9424d29dbf0a3c538215b0b98b1e6b956481e4d5
>> >
>> > * Documentation:
>> > Note the documentation can't be pushed live due to changes that will
>> not go
>> > live until the release. You can manually verify by downloading
>> > http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>> > kafka_2.11-1.0.0-site-docs.tgz
>> >
>> > * Successful Jenkins builds for the 1.0.0 branch:
>> > Unit/integration tests: https://builds.apache.org/job/
>> kafka-1.0-jdk7/31/
>> > System test: https://jenkins.confluent.io/job/system-test-kafka-1.0/1/
>> >
>> >
>> > /**
>> >
>> >
>> > Thanks,
>> > -- Guozhang
>> >
>>
>
>
>
> --
> -- Guozhang
>
> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscr...@googlegroups.com.
> To post to this group, send email to kafka-clie...@googlegroups.com.
> Visit this group at