Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[James Cheng]

Congrats and great job, everyone! Thanks Rajini for driving the release!

-James

Sent from my iPhone

> On Jul 30, 2018, at 3:25 AM, Rajini Sivaram  wrote:
> 
> The Apache Kafka community is pleased to announce the release for
> 
> Apache Kafka 2.0.0.
> 
> 
> 
> 
> 
> This is a major release and includes significant new features from
> 
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
> 
> a few critical bugs. Here is a summary of some notable changes:
> 
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
> 
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
> 
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
> 
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
> 
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
> 
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
> 
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
> 
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
> 
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
> 
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
> 
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably regarding
> Serdes with new implicit Serdes.
> 
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
> 
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
> 
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
> 
> 
> 
> 
> 
> All of the changes in this release can be found in the release notes:
> 
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
> 
> 
> 
> 
> 
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
> 
> https://kafka.apache.org/downloads#2.0.0
> 
> 
> 
> 
> ---
> 
> 
> 
> 
> 
> Apache Kafka is a distributed streaming platform with four core APIs:
> 
> 
> 
> ** The Producer API allows an application to publish a stream records to
> 
> one or more Kafka topics.
> 
> 
> 
> ** The Consumer API allows an application to subscribe to one or more
> 
> 

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Ismael Juma]

Thanks to everyone who contributed to the release (including testing and
bug reports)! And thank you Rajini for managing the release.

Ismael

On Mon, 30 Jul 2018, 03:25 Rajini Sivaram,  wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> 
>
>
>
>
> ---
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics 

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Guozhang Wang]

Great news! Thanks for driving the release Rajini.


Guozhang

On Mon, Jul 30, 2018 at 8:33 AM, Vahid S Hashemian <
vahidhashem...@us.ibm.com> wrote:

> Such a good news on a Monday morning ...
>
> Thank you Rajini for driving the release!
>
> --Vahid
>
>
>
>
> From:   Mickael Maison 
> To: Users 
> Cc: dev , annou...@apache.org, kafka-clients
> 
> Date:   07/30/2018 04:37 AM
> Subject:Re: [ANNOUNCE] Apache Kafka 2.0.0 Released
>
>
>
> Great news! Thanks for running the release
>
> On Mon, Jul 30, 2018 at 12:20 PM, Manikumar 
> wrote:
> > Thanks for driving the release!
> >
> >
> >
> > On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram 
> wrote:
> >
> >> The Apache Kafka community is pleased to announce the release for
> >>
> >> Apache Kafka 2.0.0.
> >>
> >>
> >>
> >>
> >>
> >> This is a major release and includes significant new features from
> >>
> >> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
> >>
> >> a few critical bugs. Here is a summary of some notable changes:
> >>
> >> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> >> management in large secure deployments. Bulk access to topics,
> >> consumer groups or transactional ids with a prefix can now be granted
> >> using a single rule. Access control for topic creation has also been
> >> improved to enable access to be granted to create specific topics or
> >> topics with a prefix.
> >>
> >> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> >> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> >> customizable using callbacks for token retrieval and validation.
> >>
> >> **Host name verification is now enabled by default for SSL connections
> >> to ensure that the default SSL configuration is not susceptible to
> >> man-in-the middle attacks. You can disable this verification for
> >> deployments where validation is performed using other mechanisms.
> >>
> >> ** You can now dynamically update SSL trust stores without broker
> restart.
> >> You can also configure security for broker listeners in ZooKeeper
> before
> >> starting brokers, including SSL key store and trust store passwords and
> >> JAAS configuration for SASL. With this new feature, you can store
> sensitive
> >> password configs in encrypted form in ZooKeeper rather than in
> cleartext
> >> in the broker properties file.
> >>
> >> ** The replication protocol has been improved to avoid log divergence
> >> between leader and follower during fast leader failover. We have also
> >> improved resilience of brokers by reducing the memory footprint of
> >> message down-conversions. By using message chunking, both memory
> >> usage and memory reference time have been reduced to avoid
> >> OutOfMemory errors in brokers.
> >>
> >> ** Kafka clients are now notified of throttling before any throttling
> is
> >> applied
> >> when quotas are enabled. This enables clients to distinguish between
> >> network errors and large throttle times when quotas are exceeded.
> >>
> >> ** We have added a configuration option for Kafka consumer to avoid
> >> indefinite blocking in the consumer.
> >>
> >> ** We have dropped support for Java 7 and removed the previously
> >> deprecated Scala producer and consumer.
> >>
> >> ** Kafka Connect includes a number of improvements and features.
> >> KIP-298 enables you to control how errors in connectors,
> transformations
> >> and converters are handled by enabling automatic retries and
> controlling
> >> the
> >> number of errors that are tolerated before the connector is stopped.
> More
> >> contextual information can be included in the logs to help diagnose
> >> problems
> >> and problematic messages consumed by sink connectors can be sent to a
> >> dead letter queue rather than forcing the connector to stop.
> >>
> >> ** KIP-297 adds a new extension point to move secrets out of connector
> >> configurations and integrate with any external key management system.
> >> The placeholders in connector configurations are only resolved before
> >> sending the configuration to the connector, ensuring that secrets are
> >> stored
> >> and managed securely in your preferred key management system and
> >> not expo

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Vahid S Hashemian]

Such a good news on a Monday morning ...

Thank you Rajini for driving the release!

--Vahid




From:   Mickael Maison 
To: Users 
Cc: dev , annou...@apache.org, kafka-clients 

Date:   07/30/2018 04:37 AM
Subject:Re: [ANNOUNCE] Apache Kafka 2.0.0 Released



Great news! Thanks for running the release

On Mon, Jul 30, 2018 at 12:20 PM, Manikumar  
wrote:
> Thanks for driving the release!
>
>
>
> On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram  
wrote:
>
>> The Apache Kafka community is pleased to announce the release for
>>
>> Apache Kafka 2.0.0.
>>
>>
>>
>>
>>
>> This is a major release and includes significant new features from
>>
>> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>>
>> a few critical bugs. Here is a summary of some notable changes:
>>
>> ** KIP-290 adds support for prefixed ACLs, simplifying access control
>> management in large secure deployments. Bulk access to topics,
>> consumer groups or transactional ids with a prefix can now be granted
>> using a single rule. Access control for topic creation has also been
>> improved to enable access to be granted to create specific topics or
>> topics with a prefix.
>>
>> ** KIP-255 adds a framework for authenticating to Kafka brokers using
>> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
>> customizable using callbacks for token retrieval and validation.
>>
>> **Host name verification is now enabled by default for SSL connections
>> to ensure that the default SSL configuration is not susceptible to
>> man-in-the middle attacks. You can disable this verification for
>> deployments where validation is performed using other mechanisms.
>>
>> ** You can now dynamically update SSL trust stores without broker 
restart.
>> You can also configure security for broker listeners in ZooKeeper 
before
>> starting brokers, including SSL key store and trust store passwords and
>> JAAS configuration for SASL. With this new feature, you can store 
sensitive
>> password configs in encrypted form in ZooKeeper rather than in 
cleartext
>> in the broker properties file.
>>
>> ** The replication protocol has been improved to avoid log divergence
>> between leader and follower during fast leader failover. We have also
>> improved resilience of brokers by reducing the memory footprint of
>> message down-conversions. By using message chunking, both memory
>> usage and memory reference time have been reduced to avoid
>> OutOfMemory errors in brokers.
>>
>> ** Kafka clients are now notified of throttling before any throttling 
is
>> applied
>> when quotas are enabled. This enables clients to distinguish between
>> network errors and large throttle times when quotas are exceeded.
>>
>> ** We have added a configuration option for Kafka consumer to avoid
>> indefinite blocking in the consumer.
>>
>> ** We have dropped support for Java 7 and removed the previously
>> deprecated Scala producer and consumer.
>>
>> ** Kafka Connect includes a number of improvements and features.
>> KIP-298 enables you to control how errors in connectors, 
transformations
>> and converters are handled by enabling automatic retries and 
controlling
>> the
>> number of errors that are tolerated before the connector is stopped. 
More
>> contextual information can be included in the logs to help diagnose
>> problems
>> and problematic messages consumed by sink connectors can be sent to a
>> dead letter queue rather than forcing the connector to stop.
>>
>> ** KIP-297 adds a new extension point to move secrets out of connector
>> configurations and integrate with any external key management system.
>> The placeholders in connector configurations are only resolved before
>> sending the configuration to the connector, ensuring that secrets are
>> stored
>> and managed securely in your preferred key management system and
>> not exposed over the REST APIs or in log files.
>>
>> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
>> which provides better type inference and better type safety during 
compile
>> time. Scala users can have less boilerplate in their code, notably
>> regarding
>> Serdes with new implicit Serdes.
>>
>> ** Message headers are now supported in the Kafka Streams Processor 
API,
>> allowing users to add and manipulate headers read from the source 
topics
>> and propagate them to the sink topics.
>>
>> ** Windowed aggregations performance in Kafka Streams has been largely
>> improved (

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Mickael Maison]

Great news! Thanks for running the release

On Mon, Jul 30, 2018 at 12:20 PM, Manikumar  wrote:
> Thanks for driving the release!
>
>
>
> On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram  wrote:
>
>> The Apache Kafka community is pleased to announce the release for
>>
>> Apache Kafka 2.0.0.
>>
>>
>>
>>
>>
>> This is a major release and includes significant new features from
>>
>> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>>
>> a few critical bugs. Here is a summary of some notable changes:
>>
>> ** KIP-290 adds support for prefixed ACLs, simplifying access control
>> management in large secure deployments. Bulk access to topics,
>> consumer groups or transactional ids with a prefix can now be granted
>> using a single rule. Access control for topic creation has also been
>> improved to enable access to be granted to create specific topics or
>> topics with a prefix.
>>
>> ** KIP-255 adds a framework for authenticating to Kafka brokers using
>> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
>> customizable using callbacks for token retrieval and validation.
>>
>> **Host name verification is now enabled by default for SSL connections
>> to ensure that the default SSL configuration is not susceptible to
>> man-in-the middle attacks. You can disable this verification for
>> deployments where validation is performed using other mechanisms.
>>
>> ** You can now dynamically update SSL trust stores without broker restart.
>> You can also configure security for broker listeners in ZooKeeper before
>> starting brokers, including SSL key store and trust store passwords and
>> JAAS configuration for SASL. With this new feature, you can store sensitive
>> password configs in encrypted form in ZooKeeper rather than in cleartext
>> in the broker properties file.
>>
>> ** The replication protocol has been improved to avoid log divergence
>> between leader and follower during fast leader failover. We have also
>> improved resilience of brokers by reducing the memory footprint of
>> message down-conversions. By using message chunking, both memory
>> usage and memory reference time have been reduced to avoid
>> OutOfMemory errors in brokers.
>>
>> ** Kafka clients are now notified of throttling before any throttling is
>> applied
>> when quotas are enabled. This enables clients to distinguish between
>> network errors and large throttle times when quotas are exceeded.
>>
>> ** We have added a configuration option for Kafka consumer to avoid
>> indefinite blocking in the consumer.
>>
>> ** We have dropped support for Java 7 and removed the previously
>> deprecated Scala producer and consumer.
>>
>> ** Kafka Connect includes a number of improvements and features.
>> KIP-298 enables you to control how errors in connectors, transformations
>> and converters are handled by enabling automatic retries and controlling
>> the
>> number of errors that are tolerated before the connector is stopped. More
>> contextual information can be included in the logs to help diagnose
>> problems
>> and problematic messages consumed by sink connectors can be sent to a
>> dead letter queue rather than forcing the connector to stop.
>>
>> ** KIP-297 adds a new extension point to move secrets out of connector
>> configurations and integrate with any external key management system.
>> The placeholders in connector configurations are only resolved before
>> sending the configuration to the connector, ensuring that secrets are
>> stored
>> and managed securely in your preferred key management system and
>> not exposed over the REST APIs or in log files.
>>
>> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
>> which provides better type inference and better type safety during compile
>> time. Scala users can have less boilerplate in their code, notably
>> regarding
>> Serdes with new implicit Serdes.
>>
>> ** Message headers are now supported in the Kafka Streams Processor API,
>> allowing users to add and manipulate headers read from the source topics
>> and propagate them to the sink topics.
>>
>> ** Windowed aggregations performance in Kafka Streams has been largely
>> improved (sometimes by an order of magnitude) thanks to the new
>> single-key-fetch API.
>>
>> ** We have further improved unit testibility of Kafka Streams with the
>> kafka-streams-testutil artifact.
>>
>>
>>
>>
>>
>> All of the changes in this release can be found in the release notes:
>>
>> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>>
>>
>>
>>
>>
>> You can download the source and binary release (Scala 2.11 and Scala 2.12)
>> from:
>>
>> https://kafka.apache.org/downloads#2.0.0
>> 
>>
>>
>>
>>
>> ---
>>
>>
>>
>>
>>
>> Apache Kafka is a distributed streaming platform with four core APIs:
>>
>>
>>
>> ** The Producer API allows an application to publish a stream records to

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Manikumar]

Thanks for driving the release!



On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram  wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> 
>
>
>
>
> ---
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to 

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Damian Guy]

Excellent! Thanks for running the release Rajini!

On Mon, 30 Jul 2018 at 11:25 Rajini Sivaram  wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> 
>
>
>
>
> ---
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an 

[ANNOUNCE] Apache Kafka 2.0.0 Released

[Rajini Sivaram]

The Apache Kafka community is pleased to announce the release for

Apache Kafka 2.0.0.





This is a major release and includes significant new features from

40 KIPs. It contains fixes and improvements from 246 JIRAs, including

a few critical bugs. Here is a summary of some notable changes:

** KIP-290 adds support for prefixed ACLs, simplifying access control
management in large secure deployments. Bulk access to topics,
consumer groups or transactional ids with a prefix can now be granted
using a single rule. Access control for topic creation has also been
improved to enable access to be granted to create specific topics or
topics with a prefix.

** KIP-255 adds a framework for authenticating to Kafka brokers using
OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
customizable using callbacks for token retrieval and validation.

**Host name verification is now enabled by default for SSL connections
to ensure that the default SSL configuration is not susceptible to
man-in-the middle attacks. You can disable this verification for
deployments where validation is performed using other mechanisms.

** You can now dynamically update SSL trust stores without broker restart.
You can also configure security for broker listeners in ZooKeeper before
starting brokers, including SSL key store and trust store passwords and
JAAS configuration for SASL. With this new feature, you can store sensitive
password configs in encrypted form in ZooKeeper rather than in cleartext
in the broker properties file.

** The replication protocol has been improved to avoid log divergence
between leader and follower during fast leader failover. We have also
improved resilience of brokers by reducing the memory footprint of
message down-conversions. By using message chunking, both memory
usage and memory reference time have been reduced to avoid
OutOfMemory errors in brokers.

** Kafka clients are now notified of throttling before any throttling is
applied
when quotas are enabled. This enables clients to distinguish between
network errors and large throttle times when quotas are exceeded.

** We have added a configuration option for Kafka consumer to avoid
indefinite blocking in the consumer.

** We have dropped support for Java 7 and removed the previously
deprecated Scala producer and consumer.

** Kafka Connect includes a number of improvements and features.
KIP-298 enables you to control how errors in connectors, transformations
and converters are handled by enabling automatic retries and controlling the
number of errors that are tolerated before the connector is stopped. More
contextual information can be included in the logs to help diagnose problems
and problematic messages consumed by sink connectors can be sent to a
dead letter queue rather than forcing the connector to stop.

** KIP-297 adds a new extension point to move secrets out of connector
configurations and integrate with any external key management system.
The placeholders in connector configurations are only resolved before
sending the configuration to the connector, ensuring that secrets are stored
and managed securely in your preferred key management system and
not exposed over the REST APIs or in log files.

** We have added a thin Scala wrapper API for our Kafka Streams DSL,
which provides better type inference and better type safety during compile
time. Scala users can have less boilerplate in their code, notably regarding
Serdes with new implicit Serdes.

** Message headers are now supported in the Kafka Streams Processor API,
allowing users to add and manipulate headers read from the source topics
and propagate them to the sink topics.

** Windowed aggregations performance in Kafka Streams has been largely
improved (sometimes by an order of magnitude) thanks to the new
single-key-fetch API.

** We have further improved unit testibility of Kafka Streams with the
kafka-streams-testutil artifact.





All of the changes in this release can be found in the release notes:

https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html





You can download the source and binary release (Scala 2.11 and Scala 2.12)
from:

https://kafka.apache.org/downloads#2.0.0




---





Apache Kafka is a distributed streaming platform with four core APIs:



** The Producer API allows an application to publish a stream records to

one or more Kafka topics.



** The Consumer API allows an application to subscribe to one or more

topics and process the stream of records produced to them.



** The Streams API allows an application to act as a stream processor,

consuming an input stream from one or more topics and producing an

output stream to one or more output topics, effectively transforming the

input streams to output streams.



** The Connector API allows building and running reusable producers or

consumers that connect Kafka