Jennifer88huang commented on a change in pull request #7756: URL: https://github.com/apache/pulsar/pull/7756#discussion_r474399643
########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -263,23 +247,19 @@ org.apache.pulsar.functions.worker.FunctionWorkerStarter.main(FunctionWorkerStar This is because the broker 2.5.0 supports "bookkeeperMetadataServiceUri" and the admin client returns a `null` field, thus causing the NPE. -For more information about implementation details, see [PR-7528](https://github.com/apache/pulsar/pull/7528). - -#### Improve security setting of Pulsar Functions - -For more information about implementation details, see [PR-7578](https://github.com/apache/pulsar/pull/7578). +For more information about implementation, see [PR-7528](https://github.com/apache/pulsar/pull/7528). ## pulsar-perf -#### Support `tlsAllowInsecureConnection` in pulsar-perf produce/consume/read performance tests +### Support `tlsAllowInsecureConnection` in pulsar-perf produce/consume/read performance tests Add `tlsAllowInsecureConnection` config to the CLI tool **pulsar-perf**, to support produce/consume/read performance tests to clusters with insecure TLS connections. -For more information about implementation details, see [PR-7300](https://github.com/apache/pulsar/pull/7300). +For more information about implementation, see [PR-7300](https://github.com/apache/pulsar/pull/7300). ## More information -- To download Apache Pulsar 2.6.1, click [here](https://pulsar.apache.org/en/download/). +- To download Apache Pulsar 2.6.1, click [download](https://pulsar.apache.org/en/download/). - For more information about Apache Pulsar 2.6.1, see [2.6.1 release notes](https://pulsar.apache.org/release-notes/#2.6.1 and [2.6.1 PR list](https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.6.1+is%3Aclosed). Review comment: ```suggestion - For more information about Apache Pulsar 2.6.1, see [2.6.1 release notes](https://pulsar.apache.org/release-notes/#2.6.1) and [2.6.1 PR list](https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.6.1+is%3Aclosed). ``` ########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -63,82 +64,74 @@ try { return new PersistentSubscription(this, subscriptionName, cursor, false); ``` -Additionally, the NPE leads to the topic usage count to not be decremented, leaking 1 usage increment. At the time of deletion, this will prevent the topic from being deleted, even when using the force flag. +Additionally, the NPE leads to the topic usage count increasing to 1. When deleting a topic, the topic cannot be deleted even if you use the force flag. -For more information about implementation details, see [PR-7355](https://github.com/apache/pulsar/pull/7355). +For more information about implementation, see [PR-7355](https://github.com/apache/pulsar/pull/7355). -#### Avoid the NPE occurs in method `ManagedLedgerImpl.isOffloadedNeedsDelete` +### Avoid an NPE occurs in the `ManagedLedgerImpl.isOffloadedNeedsDelete` method Review comment: ```suggestion ### Avoid an NPE occurs in the `ManagedLedgerImpl.isOffloadedNeedsDelete` method ``` ########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -63,82 +64,74 @@ try { return new PersistentSubscription(this, subscriptionName, cursor, false); ``` -Additionally, the NPE leads to the topic usage count to not be decremented, leaking 1 usage increment. At the time of deletion, this will prevent the topic from being deleted, even when using the force flag. +Additionally, the NPE leads to the topic usage count increasing to 1. When deleting a topic, the topic cannot be deleted even if you use the force flag. -For more information about implementation details, see [PR-7355](https://github.com/apache/pulsar/pull/7355). +For more information about implementation, see [PR-7355](https://github.com/apache/pulsar/pull/7355). -#### Avoid the NPE occurs in method `ManagedLedgerImpl.isOffloadedNeedsDelete` +### Avoid an NPE occurs in the `ManagedLedgerImpl.isOffloadedNeedsDelete` method Review comment: ```suggestion ### Avoid an NPE occurs in the `ManagedLedgerImpl.isOffloadedNeedsDelete` method ``` ########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -148,109 +141,100 @@ The exception is thrown at Line 77, since `getAddress()` returns a `null` given 78 } ``` -The default implementation for the `DnsResolver.useHostName()` is to return true. +The default implementation for the `DnsResolver.useHostName()` returns `true`. -For more information about implementation details, see [PR-7361](https://github.com/apache/pulsar/pull/7361). +For more information about implementation, see [PR-7361](https://github.com/apache/pulsar/pull/7361). ## Java Client -#### Fix the issue that the HTTP header used in Athenz authentication can not be renamed +### Fix the issue that the HTTP header used in Athenz authentication can not be renamed -The authentication plugin for Athenz allows users to change the name of the HTTP header for sending an authentication token to a broker server with a parameter named `roleHeader`. The change will hold the value of the `roleHeader` parameter on the `AuthenticationAthenz` side, and use it directly as the header name. +The authentication plugin for Athenz allows users to change the name of the HTTP header for sending an authentication token to a broker server with a parameter named `roleHeader`. The change uses the value of the `roleHeader` parameter on the `AuthenticationAthenz` side, and uses it directly as the header name. -For more information about implementation details, see [PR-7311](https://github.com/apache/pulsar/pull/7311). +For more information about implementation, see [PR-7311](https://github.com/apache/pulsar/pull/7311). -#### Fix batch ack set recycled multiple times +### Fix the issue that batch ack set is recycled multiple times -Fix batch ack set recycled multiple times. The root cause is a race condition in group ack flush and cumulative Ack. So add recycled state check for the ack set. +The batch ack sets are recycled multiple times, due to race condition in group ack flush and cumulative Ack. So we add a recycled state check for the ack set in PR-7409, and fix the recycle issue. -For more information about implementation details, see [PR-7409](https://github.com/apache/pulsar/pull/7409). +For more information about implementation, see [PR-7409](https://github.com/apache/pulsar/pull/7409). -#### Add authentication client with OAuth2 support +### Add authentication client with OAuth2 support -Pulsar supports authenticating clients using OAuth 2.0 access tokens. You can use tokens to identify a Pulsar client and associate with some "principal" (or "role") that is permitted to do some actions (eg: publish messages to a topic or consume messages from a topic). +Pulsar supports authenticating clients using OAuth 2.0 access tokens. You can use tokens to identify a Pulsar client and associate with some "principal" (or "role") that is permitted to do some actions, for example, publish messages to a topic or consume messages from a topic. -This module is to support Pulsar Client Authentication Plugin for OAuth 2.0 directly. The client communicates with the Oauth 2.0 server, and then the client gets an `access token` from the Oauth 2.0 server, and passes this `access token` to Pulsar broker to do the authentication. +This module is to support Pulsar Client Authentication Plugin for OAuth 2.0 directly. The client communicates with the Oauth 2.0 server, gets an `access token` from the Oauth 2.0 server, and passes the `access token` to Pulsar broker to do the authentication. -So, the broker could still use `org.apache.pulsar.broker.authentication.AuthenticationProviderToken`, +So, the broker can use `org.apache.pulsar.broker.authentication.AuthenticationProviderToken`, and the user can add their own `AuthenticationProvider` to work with this module. -For more information about implementation details, see [PR-7420](https://github.com/apache/pulsar/pull/7420). - -#### Ensure the create subscription can be completed when the operation timeout happens +For more information about implementation, see [PR-7420](https://github.com/apache/pulsar/pull/7420). -Ensure the create subscription can be completed when the operation timeout happens. -For more information about implementation details, see [PR-7522](https://github.com/apache/pulsar/pull/7522). - -#### Do not try to subscribe to the topic if the consumer is closed +### Not subscribe to the topic when the consumer is closed Fix race condition on the closed consumer while reconnecting to the broker. -The race condition happens while the consumer reconnects to the broker. The cnx of the consumer is set to null when the consumer reconnects to the broker. If the consumer is closed at this time, the client will not send close consumer command to the broker. So, when the consumer reconnects to the broker, the consumer will send the subscribe command again. - -This pull request adds state check when connection opened of the consumer is opened. If the consumer is in closing or closed state, we do not need to send the subscribe command. +The race condition happens when the consumer reconnects to the broker. The connection of the consumer is set to `null` when the consumer reconnects to the broker. If the consumer is not connected to broker at this time, the client does not send the consumer command to the broker. So, when the consumer reconnects to the broker, the consumer sends the subscribe command again. -For more information about implementation details, see [PR-7589](https://github.com/apache/pulsar/pull/7589). +This pull request adds a state check when the `connectionOpened()` of the consumer opens. If the consumer is in closing or closed state, the consumer does not send the subscribe command. -#### Make OAuth2 authentication plugin to use AsyncHttpClient +For more information about implementation, see [PR-7589](https://github.com/apache/pulsar/pull/7589). -The OAuth2 client authentication plugin is using Apache HTTP client lib to make request, but it would be better to use AsyncHttpClient as we are using everywhere else in client and broker. Apache HTTP client was only meant to be used for hostname validation and, as explained in [#7612](https://github.com/apache/pulsar/issues/7612) we had better get rid of that dependency. +### OAuth2 authentication plugin uses AsyncHttpClient -For more information about implementation details, see [PR-7615](https://github.com/apache/pulsar/pull/7615). +Previously, the OAuth2 client authentication plugin used Apache HTTP client lib to make requests, Apache HTTP client is used to validate hostname. As suggested in [#7612](https://github.com/apache/pulsar/issues/7612), we get rid of the dependency of using Apache HTTP client. -#### Fix batch index filter issue in Consumer +In PR-7615, OAuth2 client authentication plugin uses AsyncHttpClient, which is used in client and broker. For more information about implementation, see [PR-7615](https://github.com/apache/pulsar/pull/7615). -Fix batch index filter issue in Consumer. - -For more information about implementation details, see [PR-7654](https://github.com/apache/pulsar/pull/7654). ## CPP Client -#### CPP Oauth2 authentication client +### CPP Oauth2 authentication client Pulsar supports authenticating clients using OAuth 2.0 access tokens. You can use tokens to identify a Pulsar client and associate with some "principal" (or "role") that is permitted to do some actions (eg: publish messages to a topic or consume messages from a topic). This change tries to support it in cpp client. -For more information about implementation details, see [PR-7467](https://github.com/apache/pulsar/pull/7467). +For more information about implementation, see [PR-7467](https://github.com/apache/pulsar/pull/7467). -#### Fix partition index error in close callback +### Fix partition index error in close callback -In partitioned producer/consumer's close callback, the partition index is always 0. We need to pass `ProducerImpl/ConsumerImpl`'s internal partition index field to `PartitionedProducerImpl/PartitionedConsumerImpl`'s close callback. +In partitioned producer/consumer's close callback, the partition index is always `0`. The `ProducerImpl/ConsumerImpl`'s internal partition index field should be passed to `PartitionedProducerImpl/PartitionedConsumerImpl`'s close callback. Review comment: ```suggestion In partitioned producer/consumer's close callback, the partition index is always `0`. The `ProducerImpl/ConsumerImpl` internal partition index field should be passed to `PartitionedProducerImpl/PartitionedConsumerImpl` close callback. ``` ########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -63,82 +64,74 @@ try { return new PersistentSubscription(this, subscriptionName, cursor, false); ``` -Additionally, the NPE leads to the topic usage count to not be decremented, leaking 1 usage increment. At the time of deletion, this will prevent the topic from being deleted, even when using the force flag. +Additionally, the NPE leads to the topic usage count increasing to 1. When deleting a topic, the topic cannot be deleted even if you use the force flag. -For more information about implementation details, see [PR-7355](https://github.com/apache/pulsar/pull/7355). +For more information about implementation, see [PR-7355](https://github.com/apache/pulsar/pull/7355). -#### Avoid the NPE occurs in method `ManagedLedgerImpl.isOffloadedNeedsDelete` +### Avoid an NPE occurs in the `ManagedLedgerImpl.isOffloadedNeedsDelete` method -When the default value of the `offload-deletion-lag` is set to null, it will cause an NPE problem. To fix this bug, we add null check in the method `ManagedLedgerImpl.isOffloadedNeedsDelete`. +When the default value of the `offload-deletion-lag` is set to `null`, an NPE occurs. To fix the bug, null check is added in the `ManagedLedgerImpl.isOffloadedNeedsDelete` method. -For more information about implementation details, see [PR-7389](https://github.com/apache/pulsar/pull/7389). +For more information about implementation, see [PR-7389](https://github.com/apache/pulsar/pull/7389). -#### Fix producer stuck issue due to NPE thrown when creating a new ledger +### Fix producer stuck issue due to NPE when creating a new ledger -NPE can be thrown when creating a ledger because the network address is unresolvable. If NPE is thrown before adding the timeout task, the timeout mechanism doesn't work. The unresolvable network address is commonly seen in the Kubernetes environment. It can happen when a bookie pod or a worker node restarts. +NPE occurs when creating a ledger if the network address is unresolvable. If NPE occurs before adding the timeout task, the timeout mechanism does not work. The unresolvable network address is common in the Kubernetes environment. It happens when a bookie pod or a worker node restarts. -This pull request does the followings: +This pull request fixes from the following perspectives: 1. Catch the NPE when creating a new ledger. -2. When the timeout task is triggered, always execute the callback. It is totally fine because we already have the logic to ensure the callback is triggered only once. -3. Add a mechanism to detect the `CreatingLedger` state is not moving. - -For more information about implementation details, see [PR-7401](https://github.com/apache/pulsar/pull/7401). - -#### Avoid NPEs at ledger creation when DNS failures happen - -As a followup to the fix in [PR-7401](https://github.com/apache/pulsar/pull/7401), use try/catch in all places where we're creating new ledgers to cover against NPEs triggered by DNS errors. +2. When the timeout task is triggered, it always executes the callback. It is totally fine because we already have the logic to ensure the callback is triggered only once. +3. Add a mechanism to detect that the `CreatingLedger` state is not moving. -For more information about implementation details, see [PR-7403](https://github.com/apache/pulsar/pull/7403). +For more information about implementation, see [PR-7401](https://github.com/apache/pulsar/pull/7401). -#### Decompress payload if needed in Key_Shared subscription -Decompress payload if needed in `Key_Shared` subscription. +### Fix NPE when using advertisedListeners -For more information about implementation details, see [PR-7416](https://github.com/apache/pulsar/pull/7416). +The broker failed to acquire ownership for the namespace bundle when using `advertisedListeners=internal:pulsar://node1:6650,external:pulsar://node1.external:6650` with external listener name. Correct `BrokerServiceUrlTls` when TLS is not enabled. -#### Fix NPE when using advertisedListeners +For more information about implementation, see [PR-7620](https://github.com/apache/pulsar/pull/7620). -The broker failed to acquire ownership for namespace bundle when using `advertisedListeners=internal:pulsar://node1:6650,external:pulsar://node1.external:6650` with external listener name. Correct `BrokerServiceUrlTls` when tls is not enabled. +### Fix the issue that the deduplication cursor cannot be deleted after message deduplication is disabled -For more information about implementation details, see [PR-7620](https://github.com/apache/pulsar/pull/7620). +When enabling the message deduplication in the `broker.conf` file, disabling it and then restarting the broker, the deduplication cursor is not deleted. -#### Fix the issue that the deduplication cursor cannot be deleted after message deduplication is disabled +This PR fixes the issue, so when you disable message deduplication, you can delete the deduplication cursor. -The issue occurs when enabling the message deduplication at the broker.conf and then disable it and restart the broker. The deduplication cursor cannot be deleted. +For more information about implementation, see [PR-7656](https://github.com/apache/pulsar/pull/7656). -For more information about implementation details, see [PR-7656](https://github.com/apache/pulsar/pull/7656). - -#### Get last entry is trying to read entry -1 - -A return statement is missed in the current code when the entry is set to -1. Therefore, after the code is sent, the response is trying to read the entry and sends a second response: +### Fix the issue that GetLastEntry() reads entry `-1` +Previously, the code does not include a return statement. If the entry is set to `-1`, after sending code, the response reads the entry and sends a second response, as shown in the following example. ``` 16:34:25.779 [pulsar-io-54-7:org.apache.bookkeeper.client.LedgerHandle@748] ERROR org.apache.bookkeeper.client.LedgerHandle - IncorrectParameterException on ledgerId:0 firstEntry:-1 lastEntry:-1 16:34:25.779 [pulsar-client-io-82-1:org.apache.pulsar.client.impl.ConsumerImpl@1986] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://external-repl-prop/pulsar-function-admin/assignment][c-use-fw-localhost-0-function-assignment-initialize-reader-b21f7607c9] Successfully getLastMessageId 0:-1 16:34:25.779 [pulsar-client-io-82-1:org.apache.pulsar.client.impl.ClientCnx@602] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xc78f4a0e, L:/127.0.0.1:55657 - R:localhost/127.0.0.1:55615] Received error from server: Failed to get batch size for entry org.apache.bookkeeper.mledger.ManagedLedgerException: Incorrect parameter input 16:34:25.779 [pulsar-client-io-82-1:org.apache.pulsar.client.impl.ClientCnx@612] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xc78f4a0e, L:/127.0.0.1:55657 - R:localhost/127.0.0.1:55615] Received unknown request id from server: 10 ``` -For more information about implementation details, see [PR-7495](https://github.com/apache/pulsar/pull/7495). +PR-7495 adds a return statement to code, so GetLastEntry() reads the last entry, instead of `-1`. Review comment: ```suggestion PR-7495 adds a return statement to code, so GetLastEntry() reads the last entry, instead of `-1`. ``` ########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -148,109 +141,100 @@ The exception is thrown at Line 77, since `getAddress()` returns a `null` given 78 } ``` -The default implementation for the `DnsResolver.useHostName()` is to return true. +The default implementation for the `DnsResolver.useHostName()` returns `true`. -For more information about implementation details, see [PR-7361](https://github.com/apache/pulsar/pull/7361). +For more information about implementation, see [PR-7361](https://github.com/apache/pulsar/pull/7361). ## Java Client -#### Fix the issue that the HTTP header used in Athenz authentication can not be renamed +### Fix the issue that the HTTP header used in Athenz authentication can not be renamed -The authentication plugin for Athenz allows users to change the name of the HTTP header for sending an authentication token to a broker server with a parameter named `roleHeader`. The change will hold the value of the `roleHeader` parameter on the `AuthenticationAthenz` side, and use it directly as the header name. +The authentication plugin for Athenz allows users to change the name of the HTTP header for sending an authentication token to a broker server with a parameter named `roleHeader`. The change uses the value of the `roleHeader` parameter on the `AuthenticationAthenz` side, and uses it directly as the header name. -For more information about implementation details, see [PR-7311](https://github.com/apache/pulsar/pull/7311). +For more information about implementation, see [PR-7311](https://github.com/apache/pulsar/pull/7311). -#### Fix batch ack set recycled multiple times +### Fix the issue that batch ack set is recycled multiple times -Fix batch ack set recycled multiple times. The root cause is a race condition in group ack flush and cumulative Ack. So add recycled state check for the ack set. +The batch ack sets are recycled multiple times, due to race condition in group ack flush and cumulative Ack. So we add a recycled state check for the ack set in PR-7409, and fix the recycle issue. Review comment: ```suggestion The batch ack sets are recycled multiple times, due to race condition in group ack flush and cumulative Ack. So we add a recycled state check for the ack set in PR-7409, and fix the recycle issue. ``` ########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -148,109 +141,100 @@ The exception is thrown at Line 77, since `getAddress()` returns a `null` given 78 } ``` -The default implementation for the `DnsResolver.useHostName()` is to return true. +The default implementation for the `DnsResolver.useHostName()` returns `true`. -For more information about implementation details, see [PR-7361](https://github.com/apache/pulsar/pull/7361). +For more information about implementation, see [PR-7361](https://github.com/apache/pulsar/pull/7361). ## Java Client -#### Fix the issue that the HTTP header used in Athenz authentication can not be renamed +### Fix the issue that the HTTP header used in Athenz authentication can not be renamed -The authentication plugin for Athenz allows users to change the name of the HTTP header for sending an authentication token to a broker server with a parameter named `roleHeader`. The change will hold the value of the `roleHeader` parameter on the `AuthenticationAthenz` side, and use it directly as the header name. +The authentication plugin for Athenz allows users to change the name of the HTTP header for sending an authentication token to a broker server with a parameter named `roleHeader`. The change uses the value of the `roleHeader` parameter on the `AuthenticationAthenz` side, and uses it directly as the header name. -For more information about implementation details, see [PR-7311](https://github.com/apache/pulsar/pull/7311). +For more information about implementation, see [PR-7311](https://github.com/apache/pulsar/pull/7311). -#### Fix batch ack set recycled multiple times +### Fix the issue that batch ack set is recycled multiple times -Fix batch ack set recycled multiple times. The root cause is a race condition in group ack flush and cumulative Ack. So add recycled state check for the ack set. +The batch ack sets are recycled multiple times, due to race condition in group ack flush and cumulative Ack. So we add a recycled state check for the ack set in PR-7409, and fix the recycle issue. -For more information about implementation details, see [PR-7409](https://github.com/apache/pulsar/pull/7409). +For more information about implementation, see [PR-7409](https://github.com/apache/pulsar/pull/7409). -#### Add authentication client with OAuth2 support +### Add authentication client with OAuth2 support -Pulsar supports authenticating clients using OAuth 2.0 access tokens. You can use tokens to identify a Pulsar client and associate with some "principal" (or "role") that is permitted to do some actions (eg: publish messages to a topic or consume messages from a topic). +Pulsar supports authenticating clients using OAuth 2.0 access tokens. You can use tokens to identify a Pulsar client and associate with some "principal" (or "role") that is permitted to do some actions, for example, publish messages to a topic or consume messages from a topic. -This module is to support Pulsar Client Authentication Plugin for OAuth 2.0 directly. The client communicates with the Oauth 2.0 server, and then the client gets an `access token` from the Oauth 2.0 server, and passes this `access token` to Pulsar broker to do the authentication. +This module is to support Pulsar Client Authentication Plugin for OAuth 2.0 directly. The client communicates with the Oauth 2.0 server, gets an `access token` from the Oauth 2.0 server, and passes the `access token` to Pulsar broker to do the authentication. -So, the broker could still use `org.apache.pulsar.broker.authentication.AuthenticationProviderToken`, +So, the broker can use `org.apache.pulsar.broker.authentication.AuthenticationProviderToken`, and the user can add their own `AuthenticationProvider` to work with this module. -For more information about implementation details, see [PR-7420](https://github.com/apache/pulsar/pull/7420). - -#### Ensure the create subscription can be completed when the operation timeout happens +For more information about implementation, see [PR-7420](https://github.com/apache/pulsar/pull/7420). -Ensure the create subscription can be completed when the operation timeout happens. -For more information about implementation details, see [PR-7522](https://github.com/apache/pulsar/pull/7522). - -#### Do not try to subscribe to the topic if the consumer is closed +### Not subscribe to the topic when the consumer is closed Fix race condition on the closed consumer while reconnecting to the broker. -The race condition happens while the consumer reconnects to the broker. The cnx of the consumer is set to null when the consumer reconnects to the broker. If the consumer is closed at this time, the client will not send close consumer command to the broker. So, when the consumer reconnects to the broker, the consumer will send the subscribe command again. - -This pull request adds state check when connection opened of the consumer is opened. If the consumer is in closing or closed state, we do not need to send the subscribe command. +The race condition happens when the consumer reconnects to the broker. The connection of the consumer is set to `null` when the consumer reconnects to the broker. If the consumer is not connected to broker at this time, the client does not send the consumer command to the broker. So, when the consumer reconnects to the broker, the consumer sends the subscribe command again. -For more information about implementation details, see [PR-7589](https://github.com/apache/pulsar/pull/7589). +This pull request adds a state check when the `connectionOpened()` of the consumer opens. If the consumer is in closing or closed state, the consumer does not send the subscribe command. -#### Make OAuth2 authentication plugin to use AsyncHttpClient +For more information about implementation, see [PR-7589](https://github.com/apache/pulsar/pull/7589). -The OAuth2 client authentication plugin is using Apache HTTP client lib to make request, but it would be better to use AsyncHttpClient as we are using everywhere else in client and broker. Apache HTTP client was only meant to be used for hostname validation and, as explained in [#7612](https://github.com/apache/pulsar/issues/7612) we had better get rid of that dependency. +### OAuth2 authentication plugin uses AsyncHttpClient -For more information about implementation details, see [PR-7615](https://github.com/apache/pulsar/pull/7615). +Previously, the OAuth2 client authentication plugin used Apache HTTP client lib to make requests, Apache HTTP client is used to validate hostname. As suggested in [#7612](https://github.com/apache/pulsar/issues/7612), we get rid of the dependency of using Apache HTTP client. -#### Fix batch index filter issue in Consumer +In PR-7615, OAuth2 client authentication plugin uses AsyncHttpClient, which is used in client and broker. For more information about implementation, see [PR-7615](https://github.com/apache/pulsar/pull/7615). -Fix batch index filter issue in Consumer. - -For more information about implementation details, see [PR-7654](https://github.com/apache/pulsar/pull/7654). ## CPP Client -#### CPP Oauth2 authentication client +### CPP Oauth2 authentication client Pulsar supports authenticating clients using OAuth 2.0 access tokens. You can use tokens to identify a Pulsar client and associate with some "principal" (or "role") that is permitted to do some actions (eg: publish messages to a topic or consume messages from a topic). This change tries to support it in cpp client. -For more information about implementation details, see [PR-7467](https://github.com/apache/pulsar/pull/7467). +For more information about implementation, see [PR-7467](https://github.com/apache/pulsar/pull/7467). -#### Fix partition index error in close callback +### Fix partition index error in close callback -In partitioned producer/consumer's close callback, the partition index is always 0. We need to pass `ProducerImpl/ConsumerImpl`'s internal partition index field to `PartitionedProducerImpl/PartitionedConsumerImpl`'s close callback. +In partitioned producer/consumer's close callback, the partition index is always `0`. The `ProducerImpl/ConsumerImpl`'s internal partition index field should be passed to `PartitionedProducerImpl/PartitionedConsumerImpl`'s close callback. Review comment: ```suggestion In partitioned producer/consumer's close callback, the partition index is always `0`. The `ProducerImpl/ConsumerImpl` internal partition index field should be passed to `PartitionedProducerImpl/PartitionedConsumerImpl` close callback. ``` ########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -263,23 +247,19 @@ org.apache.pulsar.functions.worker.FunctionWorkerStarter.main(FunctionWorkerStar This is because the broker 2.5.0 supports "bookkeeperMetadataServiceUri" and the admin client returns a `null` field, thus causing the NPE. -For more information about implementation details, see [PR-7528](https://github.com/apache/pulsar/pull/7528). - -#### Improve security setting of Pulsar Functions - -For more information about implementation details, see [PR-7578](https://github.com/apache/pulsar/pull/7578). +For more information about implementation, see [PR-7528](https://github.com/apache/pulsar/pull/7528). ## pulsar-perf -#### Support `tlsAllowInsecureConnection` in pulsar-perf produce/consume/read performance tests +### Support `tlsAllowInsecureConnection` in pulsar-perf produce/consume/read performance tests Add `tlsAllowInsecureConnection` config to the CLI tool **pulsar-perf**, to support produce/consume/read performance tests to clusters with insecure TLS connections. -For more information about implementation details, see [PR-7300](https://github.com/apache/pulsar/pull/7300). +For more information about implementation, see [PR-7300](https://github.com/apache/pulsar/pull/7300). ## More information -- To download Apache Pulsar 2.6.1, click [here](https://pulsar.apache.org/en/download/). +- To download Apache Pulsar 2.6.1, click [download](https://pulsar.apache.org/en/download/). - For more information about Apache Pulsar 2.6.1, see [2.6.1 release notes](https://pulsar.apache.org/release-notes/#2.6.1 and [2.6.1 PR list](https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.6.1+is%3Aclosed). Review comment: ```suggestion - For more information about Apache Pulsar 2.6.1, see [2.6.1 release notes](https://pulsar.apache.org/release-notes/#2.6.1) and [2.6.1 PR list](https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.6.1+is%3Aclosed). ``` ########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -148,109 +141,100 @@ The exception is thrown at Line 77, since `getAddress()` returns a `null` given 78 } ``` -The default implementation for the `DnsResolver.useHostName()` is to return true. +The default implementation for the `DnsResolver.useHostName()` returns `true`. -For more information about implementation details, see [PR-7361](https://github.com/apache/pulsar/pull/7361). +For more information about implementation, see [PR-7361](https://github.com/apache/pulsar/pull/7361). ## Java Client -#### Fix the issue that the HTTP header used in Athenz authentication can not be renamed +### Fix the issue that the HTTP header used in Athenz authentication can not be renamed -The authentication plugin for Athenz allows users to change the name of the HTTP header for sending an authentication token to a broker server with a parameter named `roleHeader`. The change will hold the value of the `roleHeader` parameter on the `AuthenticationAthenz` side, and use it directly as the header name. +The authentication plugin for Athenz allows users to change the name of the HTTP header for sending an authentication token to a broker server with a parameter named `roleHeader`. The change uses the value of the `roleHeader` parameter on the `AuthenticationAthenz` side, and uses it directly as the header name. -For more information about implementation details, see [PR-7311](https://github.com/apache/pulsar/pull/7311). +For more information about implementation, see [PR-7311](https://github.com/apache/pulsar/pull/7311). -#### Fix batch ack set recycled multiple times +### Fix the issue that batch ack set is recycled multiple times -Fix batch ack set recycled multiple times. The root cause is a race condition in group ack flush and cumulative Ack. So add recycled state check for the ack set. +The batch ack sets are recycled multiple times, due to race condition in group ack flush and cumulative Ack. So we add a recycled state check for the ack set in PR-7409, and fix the recycle issue. Review comment: ```suggestion The batch ack sets are recycled multiple times, due to race condition in group ack flush and cumulative Ack. So we add a recycled state check for the ack set in PR-7409, and fix the recycle issue. ``` ########## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ########## @@ -63,82 +64,74 @@ try { return new PersistentSubscription(this, subscriptionName, cursor, false); ``` -Additionally, the NPE leads to the topic usage count to not be decremented, leaking 1 usage increment. At the time of deletion, this will prevent the topic from being deleted, even when using the force flag. +Additionally, the NPE leads to the topic usage count increasing to 1. When deleting a topic, the topic cannot be deleted even if you use the force flag. -For more information about implementation details, see [PR-7355](https://github.com/apache/pulsar/pull/7355). +For more information about implementation, see [PR-7355](https://github.com/apache/pulsar/pull/7355). -#### Avoid the NPE occurs in method `ManagedLedgerImpl.isOffloadedNeedsDelete` +### Avoid an NPE occurs in the `ManagedLedgerImpl.isOffloadedNeedsDelete` method -When the default value of the `offload-deletion-lag` is set to null, it will cause an NPE problem. To fix this bug, we add null check in the method `ManagedLedgerImpl.isOffloadedNeedsDelete`. +When the default value of the `offload-deletion-lag` is set to `null`, an NPE occurs. To fix the bug, null check is added in the `ManagedLedgerImpl.isOffloadedNeedsDelete` method. -For more information about implementation details, see [PR-7389](https://github.com/apache/pulsar/pull/7389). +For more information about implementation, see [PR-7389](https://github.com/apache/pulsar/pull/7389). -#### Fix producer stuck issue due to NPE thrown when creating a new ledger +### Fix producer stuck issue due to NPE when creating a new ledger -NPE can be thrown when creating a ledger because the network address is unresolvable. If NPE is thrown before adding the timeout task, the timeout mechanism doesn't work. The unresolvable network address is commonly seen in the Kubernetes environment. It can happen when a bookie pod or a worker node restarts. +NPE occurs when creating a ledger if the network address is unresolvable. If NPE occurs before adding the timeout task, the timeout mechanism does not work. The unresolvable network address is common in the Kubernetes environment. It happens when a bookie pod or a worker node restarts. -This pull request does the followings: +This pull request fixes from the following perspectives: 1. Catch the NPE when creating a new ledger. -2. When the timeout task is triggered, always execute the callback. It is totally fine because we already have the logic to ensure the callback is triggered only once. -3. Add a mechanism to detect the `CreatingLedger` state is not moving. - -For more information about implementation details, see [PR-7401](https://github.com/apache/pulsar/pull/7401). - -#### Avoid NPEs at ledger creation when DNS failures happen - -As a followup to the fix in [PR-7401](https://github.com/apache/pulsar/pull/7401), use try/catch in all places where we're creating new ledgers to cover against NPEs triggered by DNS errors. +2. When the timeout task is triggered, it always executes the callback. It is totally fine because we already have the logic to ensure the callback is triggered only once. +3. Add a mechanism to detect that the `CreatingLedger` state is not moving. -For more information about implementation details, see [PR-7403](https://github.com/apache/pulsar/pull/7403). +For more information about implementation, see [PR-7401](https://github.com/apache/pulsar/pull/7401). -#### Decompress payload if needed in Key_Shared subscription -Decompress payload if needed in `Key_Shared` subscription. +### Fix NPE when using advertisedListeners -For more information about implementation details, see [PR-7416](https://github.com/apache/pulsar/pull/7416). +The broker failed to acquire ownership for the namespace bundle when using `advertisedListeners=internal:pulsar://node1:6650,external:pulsar://node1.external:6650` with external listener name. Correct `BrokerServiceUrlTls` when TLS is not enabled. -#### Fix NPE when using advertisedListeners +For more information about implementation, see [PR-7620](https://github.com/apache/pulsar/pull/7620). -The broker failed to acquire ownership for namespace bundle when using `advertisedListeners=internal:pulsar://node1:6650,external:pulsar://node1.external:6650` with external listener name. Correct `BrokerServiceUrlTls` when tls is not enabled. +### Fix the issue that the deduplication cursor cannot be deleted after message deduplication is disabled -For more information about implementation details, see [PR-7620](https://github.com/apache/pulsar/pull/7620). +When enabling the message deduplication in the `broker.conf` file, disabling it and then restarting the broker, the deduplication cursor is not deleted. -#### Fix the issue that the deduplication cursor cannot be deleted after message deduplication is disabled +This PR fixes the issue, so when you disable message deduplication, you can delete the deduplication cursor. -The issue occurs when enabling the message deduplication at the broker.conf and then disable it and restart the broker. The deduplication cursor cannot be deleted. +For more information about implementation, see [PR-7656](https://github.com/apache/pulsar/pull/7656). -For more information about implementation details, see [PR-7656](https://github.com/apache/pulsar/pull/7656). - -#### Get last entry is trying to read entry -1 - -A return statement is missed in the current code when the entry is set to -1. Therefore, after the code is sent, the response is trying to read the entry and sends a second response: +### Fix the issue that GetLastEntry() reads entry `-1` +Previously, the code does not include a return statement. If the entry is set to `-1`, after sending code, the response reads the entry and sends a second response, as shown in the following example. ``` 16:34:25.779 [pulsar-io-54-7:org.apache.bookkeeper.client.LedgerHandle@748] ERROR org.apache.bookkeeper.client.LedgerHandle - IncorrectParameterException on ledgerId:0 firstEntry:-1 lastEntry:-1 16:34:25.779 [pulsar-client-io-82-1:org.apache.pulsar.client.impl.ConsumerImpl@1986] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://external-repl-prop/pulsar-function-admin/assignment][c-use-fw-localhost-0-function-assignment-initialize-reader-b21f7607c9] Successfully getLastMessageId 0:-1 16:34:25.779 [pulsar-client-io-82-1:org.apache.pulsar.client.impl.ClientCnx@602] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xc78f4a0e, L:/127.0.0.1:55657 - R:localhost/127.0.0.1:55615] Received error from server: Failed to get batch size for entry org.apache.bookkeeper.mledger.ManagedLedgerException: Incorrect parameter input 16:34:25.779 [pulsar-client-io-82-1:org.apache.pulsar.client.impl.ClientCnx@612] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xc78f4a0e, L:/127.0.0.1:55657 - R:localhost/127.0.0.1:55615] Received unknown request id from server: 10 ``` -For more information about implementation details, see [PR-7495](https://github.com/apache/pulsar/pull/7495). +PR-7495 adds a return statement to code, so GetLastEntry() reads the last entry, instead of `-1`. Review comment: ```suggestion PR-7495 adds a return statement to code, so GetLastEntry() reads the last entry, instead of `-1`. ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
