[GitHub] jvrao commented on a change in pull request #402: Issue 356: Release notes 4.5.0
jvrao commented on a change in pull request #402: Issue 356: Release notes 4.5.0 URL: https://github.com/apache/bookkeeper/pull/402#discussion_r131711992 ## File path: site/docs/latest/releaseNotes.md ## @@ -0,0 +1,483 @@ +--- +title: Apache BookKeeper 4.5.0 Release Notes +layout: community +--- + +This is the fifth release of BookKeeper as an Apache Top Level Project! + +The 4.5.0 release incorporates hundreds of new fixes, improvements, and features since previous major release, 4.4.0, +which was released over a year ago. It is a big milestone in Apache BookKeeper community, converging from three +main branches (Salesforce, Twitter and Yahoo). + +Apache BookKeeper users are encouraged to upgrade to `4.5.0`. The technical details of this release are summarized +below. + +## Highlights + +The main features in 4.5.0 cover are around four areas: + +- Security +- Public API +- Performance +- Operations + +### Security + +Prior to this release, Apache BookKeeper only supports simple `DIGEST-MD5` type authentication. + +With this release of Apache BookKeeper, a number of feature are introduced that can be used, together of separately, +to secure a BookKeeper cluster. + +The following security features are currently supported. + +- Authentication of connections to bookies from clients, using either `TLS` or `SASL (Kerberos). +- Authentication of connections from clients, bookies, autorecovery daemons to `ZooKeeper`, when using zookeeper +based ledger managers. +- Encryption of data transferred between bookies and clients, between bookies and autorecovery daemons using `TLS`. + +It's worth noting that those security features are optional - non-secured clusters are supported, as well as a mix +of authenticated, unauthenticated, encrypted and non-encrypted clients. + +For more details, have a look at [BookKeeper Security](../security). + +### Public API + +There are multiple new client features introduced in 4.5.0. + + LedgerHandleAdv + +The [Ledger API] is the low level API provides by BookKeeper for interacting with `ledgers` in a bookkeeper cluster. +It is simple but not flexible on ledger id or entry id generation. Apache BookKeeper introduces `LedgerHandleAdv` +as an extension of existing `LedgerHandle` for advanced usage. The new `LedgerHandleAdv` allows applications providing +its own `ledger-id` and assigning `entry-id` on adding entries. + +See [Ledger Advanced API](../api/ledger-adv-api) for more details. + + Long Poll + +`Long Poll` is a main feature that [DistributedLog](https://distributedlog.io) uses to achieve low-latency tailing. +This big feature has been merged back in 4.5.0 and available to BookKeeper users. It allows tailing-reads without +polling `LastAddConfirmed` everytime after the readers exhaust known entries. + +Although `Long Poll` brings great latency improvements on tailing reads, it is still a very low-level primitive. +It is still recommended to use high level API (e.g. [DistributedLog API](../api/distributedlog-api)) for tailing and streaming use cases. + +See [Streaming Reads](https://distributedlog.incubator.apache.org/docs/latest/user_guide/design/main.html#streaming-reads) for more details. + + Explicit LAC + +Prior to 4.5.0, the `LAC` is only advanced when subsequent entries are added. If there is no subsequent entries added, +the last entry written will not be visible to readers until the ledger is closed. High-level client (e.g. DistributedLog) or applications +has to work around this by writing some sort of `control records` to advance `LAC`. + +In 4.5.0, a new `explicit lac` feature is introduced to periodically advance `LAC` if there are not subsequent entries added. This feature +can be enabled by setting `explicitLacInterval` to a positive value. + +### Performance + +There are a lot for performance related bug fixes and improvements in 4.5.0. The major performance improvement introduced in 4.5.0, is +upgrading netty from 3.x to [4.x](http://netty.io/wiki/new-and-noteworthy-in-4.0.html). + +For more details, please read [upgrade guide](../upgrade) about the netty related tips when upgrading bookkeeper from 4.4.0 to 4.5.0. + +Besides netty 4 upgrade, there are other performance related changes highlighted as below: + + Delay Ensemble Change + +`Ensemble Change` is a feature that Apache BookKeeper uses to achieve high availability. However it is an expensive metadata operation. +Especially when Apache BookKeeper is deployed in a multiple data-centers environment, losing a data center will cause churn of metadata +operations due to ensemble changes. `Delay Ensemble Change` is introduced in 4.5.0 to overcome this problem. Enabling this feature means +an `Ensemble Change` will only occur when clients can't receive enough valid responses to satisfy `ack-quorum` constraint. This feature +improves the tail latency. + +To enable this feature, please set `delayEnsembleChange` to `true` on your clients. +
[GitHub] jvrao commented on a change in pull request #402: Issue 356: Release notes 4.5.0
jvrao commented on a change in pull request #402: Issue 356: Release notes 4.5.0 URL: https://github.com/apache/bookkeeper/pull/402#discussion_r131708549 ## File path: site/docs/latest/releaseNotes.md ## @@ -0,0 +1,483 @@ +--- +title: Apache BookKeeper 4.5.0 Release Notes +layout: community +--- + +This is the fifth release of BookKeeper as an Apache Top Level Project! + +The 4.5.0 release incorporates hundreds of new fixes, improvements, and features since previous major release, 4.4.0, +which was released over a year ago. It is a big milestone in Apache BookKeeper community, converging from three +main branches (Salesforce, Twitter and Yahoo). + +Apache BookKeeper users are encouraged to upgrade to `4.5.0`. The technical details of this release are summarized +below. + +## Highlights + +The main features in 4.5.0 cover are around four areas: + +- Security +- Public API +- Performance +- Operations + +### Security + +Prior to this release, Apache BookKeeper only supports simple `DIGEST-MD5` type authentication. + +With this release of Apache BookKeeper, a number of feature are introduced that can be used, together of separately, +to secure a BookKeeper cluster. + +The following security features are currently supported. + +- Authentication of connections to bookies from clients, using either `TLS` or `SASL (Kerberos). +- Authentication of connections from clients, bookies, autorecovery daemons to `ZooKeeper`, when using zookeeper +based ledger managers. +- Encryption of data transferred between bookies and clients, between bookies and autorecovery daemons using `TLS`. + +It's worth noting that those security features are optional - non-secured clusters are supported, as well as a mix +of authenticated, unauthenticated, encrypted and non-encrypted clients. + +For more details, have a look at [BookKeeper Security](../security). + +### Public API + +There are multiple new client features introduced in 4.5.0. + + LedgerHandleAdv + +The [Ledger API] is the low level API provides by BookKeeper for interacting with `ledgers` in a bookkeeper cluster. +It is simple but not flexible on ledger id or entry id generation. Apache BookKeeper introduces `LedgerHandleAdv` +as an extension of existing `LedgerHandle` for advanced usage. The new `LedgerHandleAdv` allows applications providing +its own `ledger-id` and assigning `entry-id` on adding entries. + +See [Ledger Advanced API](../api/ledger-adv-api) for more details. + + Long Poll + +`Long Poll` is a main feature that [DistributedLog](https://distributedlog.io) uses to achieve low-latency tailing. +This big feature has been merged back in 4.5.0 and available to BookKeeper users. It allows tailing-reads without +polling `LastAddConfirmed` everytime after the readers exhaust known entries. + +Although `Long Poll` brings great latency improvements on tailing reads, it is still a very low-level primitive. +It is still recommended to use high level API (e.g. [DistributedLog API](../api/distributedlog-api)) for tailing and streaming use cases. + +See [Streaming Reads](https://distributedlog.incubator.apache.org/docs/latest/user_guide/design/main.html#streaming-reads) for more details. + + Explicit LAC + +Prior to 4.5.0, the `LAC` is only advanced when subsequent entries are added. If there is no subsequent entries added, +the last entry written will not be visible to readers until the ledger is closed. High-level client (e.g. DistributedLog) or applications +has to work around this by writing some sort of `control records` to advance `LAC`. + +In 4.5.0, a new `explicit lac` feature is introduced to periodically advance `LAC` if there are not subsequent entries added. This feature +can be enabled by setting `explicitLacInterval` to a positive value. + +### Performance + +There are a lot for performance related bug fixes and improvements in 4.5.0. The major performance improvement introduced in 4.5.0, is +upgrading netty from 3.x to [4.x](http://netty.io/wiki/new-and-noteworthy-in-4.0.html). + +For more details, please read [upgrade guide](../upgrade) about the netty related tips when upgrading bookkeeper from 4.4.0 to 4.5.0. + +Besides netty 4 upgrade, there are other performance related changes highlighted as below: + + Delay Ensemble Change + +`Ensemble Change` is a feature that Apache BookKeeper uses to achieve high availability. However it is an expensive metadata operation. +Especially when Apache BookKeeper is deployed in a multiple data-centers environment, losing a data center will cause churn of metadata +operations due to ensemble changes. `Delay Ensemble Change` is introduced in 4.5.0 to overcome this problem. Enabling this feature means +an `Ensemble Change` will only occur when clients can't receive enough valid responses to satisfy `ack-quorum` constraint. This feature +improves the tail latency. + +To enable this feature, please set `delayEnsembleChange` to `true` on your clients. +
[GitHub] jvrao commented on a change in pull request #402: Issue 356: Release notes 4.5.0
jvrao commented on a change in pull request #402: Issue 356: Release notes 4.5.0 URL: https://github.com/apache/bookkeeper/pull/402#discussion_r131708389 ## File path: site/docs/latest/releaseNotes.md ## @@ -0,0 +1,483 @@ +--- +title: Apache BookKeeper 4.5.0 Release Notes +layout: community +--- + +This is the fifth release of BookKeeper as an Apache Top Level Project! + +The 4.5.0 release incorporates hundreds of new fixes, improvements, and features since previous major release, 4.4.0, +which was released over a year ago. It is a big milestone in Apache BookKeeper community, converging from three +main branches (Salesforce, Twitter and Yahoo). + +Apache BookKeeper users are encouraged to upgrade to `4.5.0`. The technical details of this release are summarized +below. + +## Highlights + +The main features in 4.5.0 cover are around four areas: + +- Security +- Public API +- Performance +- Operations + +### Security + +Prior to this release, Apache BookKeeper only supports simple `DIGEST-MD5` type authentication. + +With this release of Apache BookKeeper, a number of feature are introduced that can be used, together of separately, +to secure a BookKeeper cluster. + +The following security features are currently supported. + +- Authentication of connections to bookies from clients, using either `TLS` or `SASL (Kerberos). +- Authentication of connections from clients, bookies, autorecovery daemons to `ZooKeeper`, when using zookeeper +based ledger managers. +- Encryption of data transferred between bookies and clients, between bookies and autorecovery daemons using `TLS`. + +It's worth noting that those security features are optional - non-secured clusters are supported, as well as a mix +of authenticated, unauthenticated, encrypted and non-encrypted clients. + +For more details, have a look at [BookKeeper Security](../security). + +### Public API + +There are multiple new client features introduced in 4.5.0. + + LedgerHandleAdv + +The [Ledger API] is the low level API provides by BookKeeper for interacting with `ledgers` in a bookkeeper cluster. +It is simple but not flexible on ledger id or entry id generation. Apache BookKeeper introduces `LedgerHandleAdv` +as an extension of existing `LedgerHandle` for advanced usage. The new `LedgerHandleAdv` allows applications providing +its own `ledger-id` and assigning `entry-id` on adding entries. + +See [Ledger Advanced API](../api/ledger-adv-api) for more details. + + Long Poll + +`Long Poll` is a main feature that [DistributedLog](https://distributedlog.io) uses to achieve low-latency tailing. +This big feature has been merged back in 4.5.0 and available to BookKeeper users. It allows tailing-reads without +polling `LastAddConfirmed` everytime after the readers exhaust known entries. + +Although `Long Poll` brings great latency improvements on tailing reads, it is still a very low-level primitive. +It is still recommended to use high level API (e.g. [DistributedLog API](../api/distributedlog-api)) for tailing and streaming use cases. + +See [Streaming Reads](https://distributedlog.incubator.apache.org/docs/latest/user_guide/design/main.html#streaming-reads) for more details. + + Explicit LAC + +Prior to 4.5.0, the `LAC` is only advanced when subsequent entries are added. If there is no subsequent entries added, +the last entry written will not be visible to readers until the ledger is closed. High-level client (e.g. DistributedLog) or applications +has to work around this by writing some sort of `control records` to advance `LAC`. + +In 4.5.0, a new `explicit lac` feature is introduced to periodically advance `LAC` if there are not subsequent entries added. This feature +can be enabled by setting `explicitLacInterval` to a positive value. + +### Performance + +There are a lot for performance related bug fixes and improvements in 4.5.0. The major performance improvement introduced in 4.5.0, is Review comment: I think there are lot more perf improvements related to caching additional metadata, avoiding buffer copies, taking advantage of java8 features, scheduling and threading improvements etc. Ledger recovery improvements. We should discuss these. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jvrao commented on a change in pull request #402: Issue 356: Release notes 4.5.0
jvrao commented on a change in pull request #402: Issue 356: Release notes 4.5.0 URL: https://github.com/apache/bookkeeper/pull/402#discussion_r131706021 ## File path: site/docs/latest/releaseNotes.md ## @@ -0,0 +1,483 @@ +--- Review comment: Can we add all upgrades we did? Upgrade protobuf to 2.6 Codehale to 3.1.0 ZK to version 3.5 Move to JDK 8 SLF4J to 7.25 Netty 4.1 Guava to version 20.0 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
